TWI749921B - Method and system for register operating space - Google Patents

Abstract

A system for registering operating space includes a back bracing, a computer system and a robotic arm. The back bracing includes positioning marks which can be imaged in a computer tomography (CT) image, and the back bracing is used to be worn on a patient. The computer system computes image coordinates of the positioning marks in an image space. A base of the robotic arm is connected to the back bracing in an operating space after the CT image is generated. The robotic arm contacts the positioning marks in the operating space and the computer system obtains robotic arm coordinates of the robotic arm in a robotic arm space at this time. The computer system generates conversion models among the image space, the operating space, and the robot arm space according to the image coordinates and the robot arm coordinates.

Description

Surgical space registration system and method

This disclosure is about systems and methods for registering surgical space, imaging space, and robotic arm space.

With the increase in the proportion of the aging population, coupled with the influence of modern life characteristics, such as obesity, sedentary, etc., the proportion of spine lesions has increased year by year; after conservative treatment fails, it is often necessary to rely on implants to help the spine to relieve pain and reduce pain. Maintain basic functions. The spine is responsible for the basic function of protecting the central nervous system, but the implants that can be used are quite narrow, such as pedicle screws, which may damage the central nervous system. Although there are orthopedic minimally invasive surgery on the market, the position of the spine changes with the posture of the person. Therefore, how to accurately track the position of the spine during the operation is a topic of concern to those skilled in the art.

The disclosed embodiment proposes a surgical space registration system, which includes a back frame, a computer system, and a robotic arm. The back frame includes multiple positioning marks, which These positioning marks can be imaged in computer tomography, and the back frame can be worn on the patient. The computer system is used to identify the positioning marks in the computer tomography image, and calculate the multiple image coordinates of the positioning marks in the image space. The robotic arm includes a base, which is used to connect to the back frame in the operating space after the computer tomography image is generated. The robotic arm is used to contact the positioning mark in the operation space, and the computer system is used to obtain the coordinates of multiple robotic arms in a robotic arm space when the robotic arm touches the positioning mark. The computer system is used for generating at least one conversion model among the image space, the operation space, and the robot arm space according to the image coordinates and the robot arm coordinates.

In some embodiments, the back frame includes a plurality of straps, which are used to tie the patient's body part, and each positioning mark is provided on one of the straps.

In some embodiments, the back frame further includes a rigid structure for connecting to the base.

In some embodiments, the computer system is used to provide a navigation interface to convert the real-time arm position of the robotic arm in the robotic arm space to the real-time image position in the image space according to the conversion model, and display information about the real-time image position in the navigation interface. The virtual object of the robotic arm.

In some embodiments, the surgical space registration system further includes a camera, a spine positioning mark, and an arm positioning mark. The spine positioning mark is used to be fixed on the patient's spine, and the arm positioning mark is used to be fixed on the robotic arm. The camera is used to shoot images, and the computer system is used to identify the spine positioning mark and the arm positioning mark in the image to calculate the position of the spine positioning mark relative to the robotic arm in the operating space.

From another perspective, the embodiment of the present disclosure proposes a surgical space registration method, which is suitable for a surgical space registration system, and the surgical space registration system includes a computer system. The surgical space registration method includes: obtaining a computerized tomographic image, identifying multiple positioning marks in the computerized tomographic image, and calculating multiple image coordinates of the positioning marks in the image space, wherein the positioning marks are set on the back frame, and the back frame is used for wearing On the patient; control the robotic arm to contact the positioning mark in the surgical space, and obtain multiple coordinates of the robotic arm in the robotic arm space when the robotic arm touches the positioning mark. The robotic arm includes the base, and the robotic arm touches after the computer tomography is generated. The base before the positioning mark is connected to the back frame in the operation space; and a conversion model between the image space, the operation space and the robot arm space is generated according to the image coordinates and the robot arm coordinates.

In some embodiments, the above-mentioned surgical space registration method further includes: providing a navigation interface; converting the real-time arm position of the robotic arm in the robotic arm space to the real-time image position in the image space according to the conversion model; and according to the navigation interface The real-time image position shows the virtual object about the robotic arm.

In some embodiments, the surgical space registration system further includes a camera, a spine positioning mark, and an arm positioning mark. The spine positioning mark is used to fix the patient's spine, the arm positioning mark is used to fix the robotic arm, and the camera is used to shoot images. The surgical space registration method further includes: recognizing the spine positioning mark and the arm positioning mark in the image to calculate the position of the spine positioning mark relative to the robotic arm in the surgical space.

In the above system and method, the use of the back frame can simplify Procedures for space registration.

100: back frame

110, 120: strap

130: Positioning mark

140: rigid structure

201~205: Steps

300: Surgical Space Registration System

310: Computer System

320: display

321: Virtual Object

330: Robotic Arm

331: Pedestal

332: tip

410: Image Space

420: Robotic arm space

430: Surgical Space

T _CR , T _RH , T _HC : matrix

510: Camera

520: arm positioning mark

530: spine positioning mark

In order to make the above-mentioned features and advantages of the present invention more comprehensible, the following specific embodiments are described in detail in conjunction with the accompanying drawings.

Fig. 1 is a schematic diagram showing a back frame according to an embodiment.

Fig. 2 is a flowchart illustrating a method for registering a surgical space according to an embodiment.

Fig. 3 is a schematic diagram illustrating a surgical space registration system according to an embodiment.

Fig. 4 is a schematic diagram depicting multiple spaces according to an embodiment.

Fig. 5 is a schematic diagram illustrating a surgical space registration system according to an embodiment.

In this disclosure, a surgical space registration system is proposed, which includes a computer system, a robotic arm, and a back frame. The back frame is used to be worn on the patient. The back frame includes multiple positioning marks. The following method can be used. Quickly calculate the conversion model between different spaces.

Fig. 1 is a schematic diagram showing a back frame according to an embodiment. Fig. 2 is a flowchart illustrating a method for registering a surgical space according to an embodiment. 1 and 2, in step 201, the patient wears the back frame 100. In the embodiment of FIG. 1, the back frame 100 includes a plurality of straps 110, 120, which are used to tie the patient's The body part is tied to the torso in this embodiment, but in other embodiments, it can also be tied to the patient's hands, feet and other parts. The straps 110 and 120 are provided with a plurality of positioning marks 130. The material of the positioning marks 130 includes metal, which can be used in computer tomography. In imaging. The location of the positioning mark 130 is, for example, at the four corners of the torso to avoid concealing the spine in the computed tomography image. However, the back frame 100 of FIG. 1 is only an example. In other embodiments, the back frame 100 may have any material, shape, and/or structure as required, and the present disclosure does not limit the number and positions of the positioning marks 130. After the patient puts on the back frame 100, the relative position between the positioning mark 130 and the patient is fixed. In some embodiments, the back frame 100 has a rigid frame that can help fix or support the patient's spine. In some embodiments, the back frame 100 is deformable, and when the posture of the patient changes, the positions of the positioning marks 130 will also change. In other words, the position of the spine can be calculated by detecting the position of the positioning mark 130.

In step 202, a computer tomography image is taken by the patient wearing a back frame. The computer system then recognizes the positioning marks 130 in the computed tomography image, and calculates multiple image coordinates of the positioning marks 130 in an image space. Hereinafter, X is used to represent these image coordinates, where X is a vector. Here, the computer system can recognize the positioning mark 130 in the CT image through any image processing or computer vision method. For example, the shape of the positioning mark 130 is known, so the computer system can use a preset pattern in the computer Find the positioning mark in the tomographic image, and record its position as the above-mentioned image coordinates.

In step 203, the back frame is connected to the robotic arm. Please refer to FIG. 2 and FIG. 3. FIG. 3 is a schematic diagram of a surgical space registration system according to an embodiment. The surgical space registration system 300 includes a computer system 310, a display 320, and a robotic arm 330. An instrument can be provided on the robotic arm 330, such as a drill, a file, a scraper, a saw, a screwdriver, or the like. It is usually used in surgical operations to repair or remove part of the tissues of the anatomical site by drilling, grinding, cutting or scraping. The robotic arm 330 may include, but is not limited to, a revolute joint, a sliding joint, a ball joint, a universal joint, a barrel joint, or any combination thereof to obtain the required degree of freedom. The doctor can control the robotic arm 330 to perform surgery through a suitable device (for example, a controller with 6 degrees of freedom). The robotic arm 330 includes a base 331 for connecting to the back frame in the operating space. For example, the back frame includes a rigid structure 140 that can be connected to the base 331. In this way, the relative position between the back frame and the robotic arm 330 will be fixed.

Next, in step 204, the robotic arm is controlled to contact the positioning mark in the surgical space. In this embodiment, the robotic arm 330 has a tip 332. The operator can control the robotic arm 330 so that the tip 332 touches each positioning mark 130. The computer system 310 will obtain the robotic arm in the robotic arm space when the robotic arm 330 touches the positioning mark 130. Coordinates, denoted as vector Y below .

In step 205, a conversion model between the image space, the operation space, and the robot arm space is generated according to the above-mentioned image coordinates and the robot arm coordinates. Please refer to FIG. 4, which is a schematic diagram illustrating multiple spaces according to an embodiment. In this embodiment, there are three spaces, which are the image space 410, the robotic arm space 420, and the operation space 430, respectively. The aforementioned image coordinate X is in the image space 410, and the aforementioned robotic arm coordinate Y is in the robotic arm space 420. The operating space 430 is also a real-world space, and the coordinates of the positioning mark 130 in the operating space 430 are expressed as a vector Z. First, by matching the image coordinate X and the robot arm coordinate Y , the conversion model between the image space 410 and the robot arm space 420 can be calculated, which is expressed as a matrix T _CR , which can also be expressed as the following mathematical formula 1.

[Math 1] X = T _CR Y

On the other hand, since the back frame 100 is connected to the robot arm 330 through the rigid structure 140, the position of the positioning mark 130 on the back frame 100 is fixed. Since the position of the positioning mark 130 on the back frame 100 is known, an appropriate reference point can be set as the origin of the real space 430, for example, but not limited to taking the centroid positions of the four positioning marks 130 as the origin In this way, the coordinate Z of the positioning mark 130 in the real space 430 can be calculated. When the robot arm 330 contacts the positioning mark 130, the conversion model between the robot arm space 420 and the operation space 430 can be calculated according to the following Mathematical Formula 2, which is expressed as a matrix T _RH .

[Math 2] Y = T _RH Z

Substituting Mathematical Formula 2 into Mathematical Formula 1, the conversion model between the image space 410 and the surgical space 430 can be obtained, expressed as a matrix T _HC , as shown in the following Mathematical Formula 3.

[Math 3] X = T _HC ZT _HC = T _CR T _RH

In this way, the coordinates between the image space 410, the robotic arm space 420, and the operation space 430 can be converted arbitrarily. 3 and 4, the computer system 310 can generate a navigation interface according to the above conversion model, where any virtual reality, augmented reality, alternative reality, or mixed reality technology can be used to generate the navigation interface. , This disclosure is not limited to this. In this embodiment, the navigation interface is displayed on the display 320, but in other embodiments, the navigation interface can also be displayed on any head-mounted device, tablet computer, or transparent display. For example, the computer system can segment the spine in the CT image according to the image processing technology to generate a virtual object about the spine, and display the virtual object in the navigation interface. The computer system 310 can convert the real-time arm position of the robotic arm 330 in the robotic arm space 420 to the real-time image location in the image space 410 according to the conversion model T _{CR, and then display virtual objects about the robotic arm 330 in the navigation interface according to the real-time image location} 321. In this way, when the instrument on the robotic arm enters the human body, the doctor can know the position of the instrument in the human body through the navigation interface on the display 320. In this embodiment, since the back frame 100 fixes the posture of the patient, and the back frame 100 is connected to the robotic arm 330 through the rigid structure 140, the patient's spine is not easy to move, and the position of the spine can be accurately displayed in the navigation interface.

Fig. 5 is a schematic diagram illustrating a surgical space registration system according to an embodiment. In the embodiment of FIG. 5, the system further includes a camera 510, an arm positioning mark 520, and a spine positioning mark 530. The spine positioning mark 530 is used to fix the spine of the patient. For example, the spine positioning mark 530 is set on the vertebrae to be operated on. The arm positioning mark 520 is fixed on the robotic arm 330. The camera 510 is used to shoot images. In some embodiments, the camera 510 may include an infrared transmitter, an infrared sensor, Dual cameras, structured light sensing device or any device that can sense the depth of the scene. The computer system 310 is used to identify the spine positioning mark 530 and the arm positioning mark 520 in the image to calculate the position of the spine positioning mark 530 relative to the robotic arm 330 in the operating space. In this way, the computer system 310 can also display a virtual object related to the spine positioning mark 530 on the display 320. In some embodiments, the back frame 100 (not shown in FIG. 5 for the sake of simplicity) is deformable. In the embodiment of FIG. 5, the camera 510 also captures the positioning marks 130 on the back frame 100. The computer system 310 can recognize the positioning mark 130 in the image to re-calibrate the position of the spine. For example, the computer system 310 can recognize the displacement of the positioning mark 130, so as to make the corresponding deformation of the CT image, so that the CT image can also be adjusted based on the posture of the patient.

In the above example, the surgical space registration system is used for spinal surgery, but in other embodiments it can also be used for thoracic surgery or other suitable first-stage operations. In other words, other steps can also be added between the steps in FIG. 2 described above.

In the above-mentioned system and method, the space to which the robotic arm, the computerized tomographic image and the patient’s body belong can be conveniently registered to provide a navigation system, and the registration procedure can be simplified through the use of the back frame.

Although the present invention has been disclosed in the above embodiments, it is not intended to limit the present invention. Anyone with ordinary knowledge in the technical field can make some changes and modifications without departing from the spirit and scope of the present invention. The scope of protection of the present invention shall be subject to those defined by the attached patent scope.

130: Positioning mark

140: rigid structure

300: Surgical Space Registration System

310: Computer System

320: display

321: Virtual Object

330: Robotic Arm

331: Pedestal

332: tip

Claims (6)

A surgical space registration system includes: a back frame including a plurality of positioning marks, the positioning marks can be imaged in a computer tomography image, the back frame is used for wearing a patient, wherein the back frame includes a plurality of straps And a rigid structure, the straps are used to tie a body part of the patient, and each of the positioning marks is set on one of the straps; a computer system is used to identify the computer tomographic image The positioning marks calculate multiple image coordinates of the positioning marks in an image space; and a robotic arm includes a base, the rigid structure is connected to the base, and the base is used to generate the computer After the tomographic image is connected to the back frame in an operating space, the robotic arm is used to contact the positioning marks in the operating space, and the computer system is used to obtain a robotic arm space when the robotic arm contacts the positioning marks The computer system is used to generate at least one conversion model between the image space, the operation space, and the robot arm space according to the image coordinates and the robot arm coordinates. The surgical space registration system according to claim 1, wherein the computer system is used to provide a navigation interface for converting a real-time arm position of the robotic arm in the robotic arm space into a real-time arm position in the image space according to the at least one conversion model A real-time image location, and a virtual object related to the robotic arm is displayed in the navigation interface according to the real-time image location. The surgical space registration system according to claim 1, further comprising a camera, a spine positioning mark and an arm positioning mark, the spine positioning mark is used to be fixed on a spine of the patient, and the arm positioning mark is used to be fixed on a spine of the patient On the robotic arm, the camera is used to shoot an image, and the computer system is used to identify the spine positioning mark and the arm positioning mark in the image to calculate the position of the spine positioning mark relative to the robotic arm in the operating space . An operation space registration method is suitable for an operation space registration system. The operation space registration system includes a computer system. The operation space registration method includes: obtaining a computerized tomographic image, identifying multiple positioning marks in the computerized tomographic image, and Calculate a plurality of image coordinates of the positioning marks in an image space, wherein the positioning marks are arranged on a back frame for wearing on a patient, wherein the back frame includes a plurality of straps and a rigid Structure, the straps are used to tie a body part of the patient, and each of the positioning marks is arranged on one of the straps; a robotic arm is controlled to contact the positioning marks in an operating space to obtain When the robotic arm is in contact with the positioning marks, the coordinates of the robotic arm in a robotic arm space, wherein the robotic arm includes a base, the rigid structure is connected to the base, and the computer tomographic image is generated and the The base is connected to the back frame in the operating space before the robotic arm touches the positioning marks; and According to the image coordinates and the robot arm coordinates, at least one conversion model between the image space, the operation space and the robot arm space is generated. The surgical space registration method according to claim 4, further comprising: providing a navigation interface; according to the at least one conversion model, converting a real-time arm position of the robotic arm in the robotic arm space into a real-time image in the image space Location; and displaying a virtual object about the robotic arm in the navigation interface according to the real-time image location. According to the surgical space registration method of claim 4, the surgical space registration system further includes a camera, a spine positioning mark and an arm positioning mark, the spine positioning mark is used to be fixed on a spine of the patient, and the arm positioning The mark is fixed on the robotic arm, the camera is used to shoot an image, and the surgical space registration method further includes: recognizing the spine positioning mark and the arm positioning mark in the image to calculate the spine positioning in the surgical space The position of the marker relative to the robot arm.

Images