CN114424975B - Surgical robot auxiliary system, method, medium, terminal and surgical robot - Google Patents

Surgical robot auxiliary system, method, medium, terminal and surgical robot Download PDF

Info

Publication number
CN114424975B
CN114424975B CN202011179121.9A CN202011179121A CN114424975B CN 114424975 B CN114424975 B CN 114424975B CN 202011179121 A CN202011179121 A CN 202011179121A CN 114424975 B CN114424975 B CN 114424975B
Authority
CN
China
Prior art keywords
information
doctor
handle
operator
module
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202011179121.9A
Other languages
Chinese (zh)
Other versions
CN114424975A (en
Inventor
范宏伟
齐金标
罗加松
何超
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Microport Medbot Group Co Ltd
Original Assignee
Shanghai Microport Medbot Group Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Microport Medbot Group Co Ltd filed Critical Shanghai Microport Medbot Group Co Ltd
Priority to CN202011179121.9A priority Critical patent/CN114424975B/en
Publication of CN114424975A publication Critical patent/CN114424975A/en
Application granted granted Critical
Publication of CN114424975B publication Critical patent/CN114424975B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/30Surgical robots
    • A61B34/37Master-slave robots
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/70Manipulators specially adapted for use in surgery
    • A61B34/74Manipulators with manual electric input means
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/70Manipulators specially adapted for use in surgery
    • A61B34/74Manipulators with manual electric input means
    • A61B2034/742Joysticks

Abstract

The invention provides a surgical robot auxiliary system, a method, a medium, a terminal and a surgical robot, wherein the system comprises: the handle operation end is used for an operator to control a doctor control end of the surgical robot; the handle operation information identification unit is used for acquiring the operation information of the handle operation end; an operator identifying unit for identifying whether an operator is in the process of surgery; the storage unit is used for storing the preset mapping relation between the operation information and the doctor control end; the control unit is respectively in communication connection with the handle operation information identification unit, the operator identification unit and the storage unit and is used for controlling at least one of the movement or the state of the doctor control end according to the information fed back by the operator identification unit, the received operation information and the preset mapping relation, and the handle operation end is used for directly carrying out operation and state adjustment on the doctor control end, so that the adjustment efficiency and the experience effect of the surgical robot are effectively improved.

Description

Surgical robot auxiliary system, method, medium, terminal and surgical robot
Technical Field
The invention relates to the technical field of medical instruments, in particular to an auxiliary system, an auxiliary method, a medium, a terminal and an operation robot of an operation robot.
Background
Surgical robots are a complex integrating a plurality of modern high-tech means and are applied to various surgical scenes. The surgeon can operate the machine away from the operating table, completely different from the traditional operating concept, and is a shapely revolutionary surgical tool in the world micro-trauma surgery field.
The use of robots for surgery has become increasingly popular, and it is known that robots have successfully completed 2 tens of thousands of surgical procedures, from prostatectomy to cardiac surgery, for example, in the united states for only one year in 2004. When the robot is used for operation, hands of a doctor do not need to touch a patient. Once the incision location is determined, the robotic arm with the camera and other surgical tools will perform the cutting, hemostasis, and suturing actions, and the surgeon need only sit at the console, typically an operating room, with the console viewing and directing the robotic arm work. Some surgeons also use robotic systems called "da vinci" for cardiac surgery, obstetrics and gynecology, and for birth control.
Aiming at the current surgical robots, a master-slave control mode is generally adopted, and the control end is used for controlling the surgical instruments at the operating end of the surgical trolley to finish the corresponding surgical operation. The conventional control terminal needs to be adjusted before performing the operation, including adjustment of the doctor display table and the armrest table, so as to be suitable for different doctors, however, the conventional adjustment mode is to perform the operation on the operation screen at the doctor display table, but the operation screen is smaller, and when the adjustment is performed, an operator (for example, a surgeon) needs to leave an eyepiece area for observing the operation condition to observe the operation screen each time, so that the adjustment mode is low in efficiency and is unfavorable for the adjustment and use of the operator.
Therefore, there is a need to provide a novel surgical robot assistance system, method, medium, terminal and surgical robot to solve the above-mentioned problems in the prior art.
Disclosure of Invention
The invention aims to provide an auxiliary system, a method, a medium, a terminal and a surgical robot for directly operating and regulating the state of a doctor control end through an operating handle, so that the regulating efficiency and the experience effect of the surgical robot are effectively improved.
To achieve the above object, the surgical robot assistance system of the present invention includes:
the handle operation end is used for an operator to control a doctor control end of the surgical robot;
the handle operation information identification unit is used for acquiring the operation information of the handle operation end;
an operator identifying unit for identifying whether an operator is in the process of surgery;
the storage unit is used for storing the preset mapping relation between the operation information and the doctor control end;
the control unit is respectively in communication connection with the handle operation information identification unit, the operator identification unit and the storage unit and is used for judging that the operator is in a non-operation process according to the information fed back by the operator identification unit and controlling at least one of the movement or the state of the doctor control end according to the received operation information and the preset mapping relation.
The invention has the beneficial effects that: the operator face information is collected through the operator identification unit, so that the control unit judges whether the operator is in the eyepiece area or not, after the operator confirms that the operator face is not in the eyepiece area, the operator can identify the operation information of the operator on the operation handle through the hand operation information identification unit after operating the handle operation end, and then the control unit combines the preset mapping relation stored in the storage unit according to the operation information to adjust the doctor control end of the surgical robot, so that the operator can intuitively observe and adjust the doctor control end when immersive operation is performed, and the operation efficiency is effectively improved.
Further, the handle operation information identification unit comprises a handle rotation identification module, a handle twist identification module and a handle push-pull identification module, wherein the handle rotation identification module is used for acquiring handle rotation information generated on the handle operation end, the handle twist identification module is used for acquiring handle twist information generated on the handle operation end, and the handle push-pull identification module is used for acquiring handle push-pull information generated on the handle operation end.
The beneficial effects are that: the handle operation information identification unit is used for identifying the operation information of an operator on the hand operation end so that the control unit can judge the operation which the operator wants to perform according to the operation information, wherein the handle rotation identification module, the handle twist identification module and the handle push-pull identification module are respectively used for identifying the handle rotation information, the handle twist information and the handle push-pull information which are generated on the handle operation end so as to accurately judge the adjustment operation of the operator according to the handle rotation information, the handle twist information and the handle push-pull information.
Further, the handle operation information recognition unit further comprises an operation gesture path recognition module, and the operation gesture path recognition module is used for recognizing gesture character path information generated by an operator operating on the handle operation end. The beneficial effects are that: when the operation is performed on the operation end of the handle, the generated gesture character path information is identified through the gesture path identification module, so that corresponding adjustment operation is performed according to the gesture character path information.
Further, the auxiliary system comprises a gesture operation module, when the control unit receives the gesture character path information and judges that the operator is in a non-operation process according to the information fed back by the operator identification unit, the control unit outputs gesture operation information through the gesture operation module to control the doctor control end to adjust, the storage unit comprises a gesture character mapping module, and the gesture character mapping module comprises a character operation mapping corresponding to the gesture character path information of the handle operation end. The beneficial effects are that: after the control unit receives the gesture character path information, the gesture operation module outputs gesture operation information to perform operation control on the doctor control end according to a preset mapping relation, so that the auxiliary system can perform operation adjustment on the doctor control end according to the gesture character path information of the handle operation end, and different use requirements are met.
Further, the operator recognition unit comprises a face recognition module and a distance recognition module, wherein the face recognition module is used for collecting face information positioned at the eyepiece area of the doctor control end, and the distance recognition module is used for collecting face distance information between the face of the operator and the eyepiece area of the doctor control end. The beneficial effects are that: face information in the eyepiece area is identified through the face identification module, face distance information between the face of the operator and the eyepiece area is identified through the distance identification module, and the face information and the face distance information are combined to accurately judge whether the face of the operator is in the eyepiece area of the doctor operation end or not, so that identification accuracy can be effectively improved.
Further, the doctor control end comprises a doctor display table and an armrest arm, the doctor display table is arranged on the armrest arm, the armrest table is arranged on the armrest arm, and the control unit controls the doctor display table and the armrest arm to perform state adjustment through the doctor control end.
Further, the auxiliary system further comprises at least one of an armrest platform inclination angle adjusting module and an armrest platform lifting module, wherein the armrest platform inclination angle adjusting module is used for adjusting the inclination angle of the armrest platform, the armrest platform lifting module is used for adjusting the height of the armrest platform, the storage unit comprises an armrest platform mapping module, and the armrest platform mapping module comprises an armrest motion mapping corresponding to the operation information of the handle operation end. The beneficial effects are that: the angle and the height of the armrest platform are respectively adjusted through the armrest platform inclination angle adjusting module and the armrest platform lifting module, so that the armrest platform is adjusted to a state more suitable for an operator to use.
Further, the auxiliary system comprises at least one of a doctor display table inclination angle adjusting module and a doctor display table lifting module, wherein the doctor display table inclination angle adjusting module is used for adjusting the inclination angle of the doctor display table, the doctor display table lifting module is used for adjusting the height of the doctor display table, the storage unit further comprises a doctor display table mapping module, and the doctor display table mapping module comprises a display table motion mapping corresponding to the operation information of the handle operation end. The beneficial effects are that: the angle and the height of the doctor display table are respectively adjusted through the doctor display table inclination angle adjusting module and the doctor display table lifting module, so that the doctor display table is adjusted to be in a state more suitable for an operator to use.
Further, an operation display module is further installed on the doctor control end, and the operation display module is used for displaying at least one of state information of the doctor display table, state information of the armrest table and gesture operation information. The beneficial effects are that: the operation display module displays the state information of the operator after the doctor control end is operated, wherein the state information comprises at least one of the state information of the doctor display table, the state information of the armrest table and gesture operation information, so that the operation condition of the doctor control end can be observed by external personnel intuitively.
Further, the gesture operation information includes at least one of login information, revocation information, start information, and end information.
Further, the storage unit includes an image mapping module including an image mapping between the operation display module and the eyepiece region, and the control unit projects information displayed in the operation display module to the eyepiece region according to the image mapping so that an operator observes the image displayed in the operation display module through the eyepiece region.
The invention provides an adjusting method of a surgical robot, which comprises the following steps:
providing a storage unit for storing a preset mapping relation between operation information and a doctor control end;
judging whether an operator is in the process of surgery or not through an operator identification unit;
and if the operator is determined to be in a non-operation process, acquiring the operation information on the handle operation end through the handle operation information identification unit, and outputting an instruction by the control unit according to the operation information and the preset mapping relation stored in the storage unit so as to control at least one of the movement or the state of the doctor control end.
The invention has the beneficial effects that: by the aid of the auxiliary method, when the face of the operator is not in the eyepiece area, the control unit correspondingly adjusts the doctor control end according to the operation information on the handle operation end through the preset mapping relation stored in the storage unit, so that the state adjustment of the doctor control end is directly completed through the handle operation end, and the adjustment efficiency is effectively improved.
Further, the operator recognition unit collects operator face information first, wherein the operator face information comprises at least one of face information and face distance information.
Further, the operation information includes at least one of handle rotation information, handle twist information, handle push-pull information, and gesture character path information.
Further, the preset mapping relationship includes at least one of an image map, a display table motion map, a handrail motion map, and a character operation map.
Further, the state adjustment of the doctor control end comprises at least one of state adjustment of a doctor display table, state adjustment of an armrest table and gesture operation of the doctor control end;
the state adjustment of the doctor display table comprises at least one of height adjustment and inclination angle adjustment of a doctor control end; the state adjustment of the armrest platform comprises at least one of height adjustment and inclination angle adjustment of the armrest platform; the gesture operation of the doctor control end comprises at least one of login operation, withdrawal operation, start operation and stop operation.
The invention also provides a storage medium having stored thereon a computer program which, when executed by a processor, implements the above-described method.
The invention further provides a terminal, which comprises a processor and a memory;
the processor is configured to execute the computer program stored in the memory, so that the terminal performs the above method.
The invention also provides a surgical robot which comprises a doctor control console, a surgical trolley and an auxiliary system, wherein the doctor control console is provided with a handle operation end of the auxiliary system, the surgical trolley is provided with a plurality of mechanical arms, and the handle operation end, the mechanical arms and surgical instruments form a master-slave control relationship.
Drawings
Fig. 1 is a schematic view of an actual operation scenario of a surgical robot according to an embodiment of the present invention;
fig. 2 is a schematic view of the overall structure of a doctor control end of the surgical robot according to the embodiment of the present invention;
FIG. 3 is a system block diagram of an auxiliary system according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of information displayed by the operation display module according to an embodiment of the present invention;
FIG. 5 is a schematic diagram showing the content of gesture operation information according to an embodiment of the present invention;
FIG. 6 is a flow chart of a surgical robot assistance method according to an embodiment of the present invention;
fig. 7 is a schematic top view of a doctor control end of a surgical robot according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention. Unless otherwise defined, technical or scientific terms used herein should be given the ordinary meaning as understood by one of ordinary skill in the art to which this invention belongs. As used herein, the word "comprising" and the like means that elements or items preceding the word are included in the element or item listed after the word and equivalents thereof without precluding other elements or items.
In view of the problems existing in the prior art, an embodiment of the present invention provides a surgical robot assistance system, specifically, a system for assisting in operating a surgical robot, as shown in fig. 1, which is a schematic view of an actual operation scene of the surgical robot according to an embodiment of the present invention, and shows a surgical application scene of the surgical robot, in an exemplary embodiment, the surgical robot includes a doctor console 100 and a surgical trolley 200, and a main manipulator (not labeled, hereinafter also referred to as a handle operation end) is provided on the doctor console 100. The surgical trolley 200 has a plurality of mechanical arms 201, and the surgical instrument 2 and the endoscope 8 can be respectively mounted on the mechanical arms 201. The primary procedure of the surgical robot is for an operator (e.g., surgeon) to perform minimally invasive surgery on a patient on a hospital bed through remote operation of the surgeon's console 100 and a master manipulator. Wherein the master manipulator, the mechanical arm 201 and the surgical instrument form a master-slave control relationship. Specifically, the robot arm 201 and the surgical instrument 2 move in accordance with the operation of the main manipulator during the operation, that is, the operation of the main manipulator in accordance with the operation of the operator's hand. Further, the main manipulator can be further arranged to receive force information of the human tissue and organ to the surgical instrument and feed back the force information to the hand of the operator, so that the operator can feel the surgical operation more intuitively. The physician console 100 has a display device communicatively coupled to the endoscope mounted on the robotic arm 201 of the surgical trolley 200 that is capable of receiving and displaying images acquired by the endoscope, such as images of lesions in the abdomen of a patient. The operator controls the movement of the mechanical arm and surgical instrument by the main manipulator based on the image displayed on the display device on the doctor console 100. The endoscope and surgical instrument are passed into the patient's site through an aperture made in the patient's body by the punch instrument, respectively.
Optionally, in some procedures, the surgical robot further includes auxiliary components such as an image trolley 300, an instrument table 400, a ventilator, and an anesthesia machine 500 for use in the procedure. Those skilled in the art can select and configure these auxiliary components according to the prior art, and will not be described here.
As shown in fig. 2 to 7, the auxiliary system of the surgical robot includes:
a handle operation end 1 including an operation left handle 101 and an operation right handle 102, which are respectively installed at left and right sides of a doctor console 100 of the surgical robot;
a handle operation information identification unit 3 communicatively connected to the handle operation terminal 1 and configured to collect operation information of an operator on at least one of the operation left handle 101 and the operation right handle 102;
an operator recognition unit 4 communicatively connected to the doctor console 100 of the surgical robot and configured to collect facial information of an operator on the doctor console 100;
a storage unit 6 for storing a preset mapping relationship between the operation information and the state of the doctor console 100;
the control unit 7 controls the doctor console 100 to perform state adjustment through the preset mapping relation, including corresponding movement or corresponding function realization; wherein:
The control unit 7 is respectively connected with the handle operation information identification unit 3, the operator identification unit 4 and the storage unit 6, more specifically, in a communication manner, and after the control unit 7 receives the operation information, the control unit 7 controls the doctor console 100 to perform movement or realize corresponding functions according to the preset mapping relationship.
Through the auxiliary system, when an operator operates the doctor control console 100 of the surgical robot, the operator identification unit 4 is used for collecting the operator face information of the operator in the eyepiece area 5 of the doctor control end, and the control unit 7 is used for receiving the operation information collected by the handle operation information identification unit 3 when the control unit 7 judges that the operator face is not in the eyepiece area 5, and the doctor control console 100 is controlled to perform state adjustment according to the preset mapping relation stored in the storage unit 6, so that the control adjustment of the doctor control console 100 is completed, in the whole process, the operator can directly perform state adjustment on the doctor control console 100 through the left handle 101 and the right handle 102 of the handle operation end 1, touch screen operation on a traditional operation screen is not needed, and meanwhile, the operator can intuitively observe the change of the doctor control console 100 in the state adjustment process through the corresponding action of the handle operation end 1 when operating, so that the overall adjustment efficiency is effectively improved, the use is more convenient, and the operation feeling of the operator is better.
In some embodiments, the handle operation information identifying unit 3 includes a handle rotation identifying module 301, a handle twist identifying module 302, and a handle push-pull identifying module 303, where when the control unit 7 determines that the face of the operator is not in the eyepiece area and an operation is generated on the handle operation end 1, the handle rotation identifying module 301 is used to collect handle rotation information generated on the handle operation end 1, the handle twist identifying module 302 is used to collect handle twist information generated on the handle operation end 1, and the handle push-pull identifying module 303 is used to collect handle push-pull information generated on the handle operation end 1.
The handle rotation recognition module 301, the handle twist recognition module 302 and the handle push-pull recognition module 303 are used for respectively collecting handle rotation information, handle twist information and handle push-pull information generated by an operator on the handle operation end 1, and the control unit 7 can accurately recognize operation information generated by the operator on the handle operation end 1 through the handle rotation information, the handle twist information and the handle push-pull information, and meanwhile, the control unit 7 can conveniently adjust the state of the doctor console 100 according to the operation information.
Further, the handle operation information recognition unit 3 further includes an operation gesture path recognition module 304, where the operation gesture path recognition module 304 is configured to collect gesture character path information generated by an operator operating on the handle operation end, for example, path information for generating corresponding characters.
The operator may not only rotate, twist and push the handle operation end 1, but also operate in a specific gesture path, such as a letter type, and the gesture path recognition module 304 recognizes the gesture path operation performed by the operator at the handle operation end 1, so that the control unit 7 can make the doctor console 100 respond correspondingly according to the gesture character path information of the operator.
In some embodiments, the operator recognition unit 4 includes a face recognition module 401 and a distance recognition module 402, the face recognition module 401 is configured to collect face information at an eyepiece area of the doctor control end, and the distance recognition module 402 is configured to collect face distance information between the face of the operator and the eyepiece area of the doctor console 100.
In order to accurately determine whether the face of the operator is in the eyepiece area of the doctor console 100, face information of the operator in the eyepiece area and face distance information between the operator and the eyepiece area are acquired through the face recognition module 401 and the distance recognition module 402, and the face information and the face distance information are combined with each other, so that the control unit 7 accurately determines whether the face of the operator is in the eyepiece area, that is, whether the operator performs operation in the eyepiece area or whether the surgical robot is in a state in the operation process, when the face of the operator leaves the eyepiece area, that is, the operator is not in the operation of a surgical instrument or in the operation process at the moment, and may be in a preparation state before the operation process starts, or may be in a state in which the operation process is suspended, and when the control unit 7 determines that the face of the operator is not in the eyepiece area, that is, the auxiliary function of the handle operation end 1 is started, that is, the doctor console 100 is adjusted through gesture operation of the handle operation end 1, so as to meet different use requirements.
Specifically, the face recognition module 401 is a face recognition device in the prior art, such as an infrared device, to recognize a face, while the distance recognition module 402 is mainly used for detecting distance information between the face of a person and the eyepiece area 5, and is generally detected by a distance sensor and a body sensor, and can be installed together with the infrared device to achieve the purpose of distance detection,
in the conventional surgical robot, when the doctor console 100 of the surgical robot is adjusted, the small-sized touch screen (such as the operation display module 204 shown in fig. 7) at the doctor console 100 is directly operated, but the touch screen on the doctor console 100 is smaller, so that the operation is inconvenient, after each operation through the touch screen, the operator needs to check whether the doctor control end is adjusted to a proper posture, such as whether the height/inclination of the eyepiece area of the handle is adjusted to a state suitable for the doctor to observe, and the operation is inconvenient.
In some embodiments, the doctor console 100 includes a doctor display table 205 and a armrest arm 202, the armrest arm 202 has an armrest table 203 mounted thereon, and the control unit 7 controls the doctor display table 205 and the armrest arm 202 to perform state adjustment through the above-described handle operation information identification unit 3, operator identification unit 4, and storage unit 6.
Specifically, the control unit 7 adjusts the states of the doctor display table 205 and the armrest arm 202 according to the operation information of the operator obtained by the handle operation information identifying unit 3 and according to the preset mapping relationship stored in the storage unit 6, so that the doctor display table 205 and the armrest arm 202 can more satisfy the use requirement of the operator.
Specifically, in this embodiment, the control unit 7 mainly adjusts the height/inclination of the doctor display stand 205 and the armrest stand 203, for example, the doctor display stand and the armrest stand, so that the operator can use the console more conveniently, and of course, in some embodiments, the control unit 7 may also adjust the doctor control stand 100 or other devices of the surgical robot through the handle operation end 1 to implement other auxiliary functions, and the present invention is not limited to the above embodiments.
In some embodiments, the auxiliary system includes a gesture operation module 701, after the control unit 7 receives the gesture character path information, according to a preset mapping relationship in the storage unit 6, the control unit 7 outputs gesture operation information through the gesture operation module 701 to control the doctor console 100 to make an adjustment action corresponding to the gesture character path information, the storage unit 6 includes a gesture character mapping module 604, and the gesture character mapping module 604 includes a character operation map corresponding to the gesture character path information of the handle operation terminal 1 to the doctor console 100.
Since during the non-surgical operation, after the operator generates the gesture path operation by operating the left handle 101 and operating the right handle 102, the gesture path identifying module 304 collects gesture character path information generated by the operator operating on the handle operation end 1, and after the collected gesture character path information is transmitted to the control unit 7, the control unit 7 identifies the gesture character path information, and combines the preset mapping relationship/character operation mapping relationship stored in the gesture character mapping module 604 in the storage unit 6, the control unit 7 performs the operation on the doctor console 100 through the gesture operation module 701, where the operation includes at least one of the login operation, the cancel operation, the start operation, and the stop operation on the doctor console 100.
Specifically, when the operator performs the handle operation through the handle operation end 1, after the operator uses the gesture character path information drawn by operating the left handle 101 or operating the right handle 10 to be a letter, the control unit 7 analyzes the letter or the character information in the gesture character path information, and then, in combination with the letter and the operation meaning represented by the letter, such as the letter C, indicates that the cancel operation, such as the letter D, needs to be performed, indicates that the delete operation needs to be performed, and by analyzing the letter or the character corresponding to the gesture character path information of the operator and in combination with the character operation mapping relationship in the gesture character mapping module 604, the gesture operation module 701 controls the doctor console 100 to complete different operations.
In some embodiments, the assistance system further comprises at least one of a handrail table tilt adjustment module 702, a handrail table lift module 703, the handrail table tilt adjustment module 702 for adjusting the tilt angle of the handrail table 203, the handrail table lift module 703 for adjusting the height of the handrail table 203, the storage unit 6 further comprises a handrail table mapping module 603, the handrail table mapping module 603 comprising a mapping between the operation information of the handle operation end 1 corresponding to the handrail movement of the handrail table 203.
The inclination angle of the armrest platform 203 is adjusted through the armrest platform inclination angle adjusting module 702, the height of the armrest platform 203 is adjusted through the armrest platform lifting module 703, and the height and the inclination angle of the armrest platform 203 are adjusted, so that the use requirement of an operator can be met by using the armrest platform 203, and the armrest platform 203 is convenient for the operator to operate and use.
When the armrest platform is specifically used, after the control unit 7 receives the operation information of the operator, the inclination angle and the height of the armrest platform 203 are respectively adjusted by the armrest platform inclination angle adjusting module 702 and the armrest platform lifting module 703 by combining the preset mapping relationship between the operation information stored in the armrest platform mapping module 603 and the armrest motion in the storage unit 6, that is, the armrest motion mapping.
In some embodiments, the assistance system includes at least one of a doctor display table inclination adjustment module 704 and a doctor display table elevation module 705, the doctor display table inclination adjustment module 704 is used for adjusting an inclination angle of the doctor display table 205, the doctor display table elevation module 705 is used for adjusting a height of the doctor display table 205, the storage unit 6 further includes a doctor display table mapping module 602, and the doctor display table mapping module 602 includes a mapping between operation information of the handle operation end 1 and display table movements of the doctor display table 205.
The inclination angle of the doctor display table 205 is adjusted through the doctor display table inclination angle adjusting module 704, and the height of the doctor display table 205 is adjusted through the doctor display table lifting module 705, so that the doctor display table 205 is adjusted to be suitable for the height and the inclination angle used by an operator, and the use requirement of the operator is met.
When the control unit 7 receives the operation information of the operator, the tilt angle and the height of the doctor display table 205 are adjusted by the doctor display table tilt angle adjusting module 704 and the doctor display table lifting module 705 respectively in combination with the preset mapping relationship, i.e. the display table motion map, stored in the doctor display table mapping module 602 of the storage unit 6.
Further, after the operator performs operation through the handle operation end 1 to generate operation information, the corresponding operation information is collected through the handle rotation identification module 301, the handle twist identification module 302, the handle push-pull identification module 303 and the operation gesture path identification module 304, the identification judgment is performed after the control unit 7 receives the operation information of the operator, meanwhile, the control unit 7 performs operation adjustment (including gesture operation corresponding to gesture character path information) on the doctor console 100 through the gesture operation module 701, adjusts the inclination angle and the height of the armrest platform 203 through the armrest platform inclination angle adjustment module 702 and the armrest platform lifting module 703, and adjusts the inclination angle and the height of the doctor display platform 205 through the doctor display platform inclination angle adjustment module 704 and the doctor display platform lifting module 705, so as to complete the process of performing adjustment operation on the doctor console 100 through the handle operation end 1.
In some embodiments, as shown in fig. 4, an operation display module 204 is further installed on the doctor console 100, and the operation display module 204 is configured to display at least one of status information of the doctor display console 205, status information of the armrest console 203, and gesture operation information.
Before or after the operator performs an operation on the doctor console 100 through the handle operation end 1, the operation display module 204 is mainly used for displaying various status information corresponding to the doctor console 100 after the operation to the operator, including status information of the doctor display console 205, status information of the armrest console 203 and gesture operation information, so that other operators can observe/query/learn corresponding adjustments caused by the operations performed on the doctor console 100 by the operator.
In some embodiments, as illustrated in fig. 5, the gesture operation information includes at least one of login information, revocation information, start information, and end information.
Preferably, the operation display module 204 adopts a touch display screen, is arranged on the armrest platform 203 of the doctor console 100, faces the front of the operator when the operator sits on the operation seat, integrates the operation adjusting function of the doctor console 100, and can also complete the operation processing process of the doctor console 100 by directly operating the operation buttons on the touch display screen.
Through the touch display screen, not only can the operation information of an operator be directly observed, but also the operation processing can be directly carried out through the touch display screen.
It is further preferable that when the operation adjusting function of the doctor console 100 is reserved on the touch display screen, that is, when the function of operating the doctor console 100 by the operation buttons on the touch display screen is reserved, the priority of executing the adjustment of the doctor console 100 by the handle operation terminal 1 is higher than the priority of executing the adjustment of the doctor console 100 by the operation buttons on the touch display screen, generally the operation of the handle operation terminal 1 and the operation buttons on the touch display screen are not operated at the same time, and when the operation buttons on the handle operation terminal 1 and the touch display screen are operated at the same time, the operation information of the handle operation terminal 1 is preferentially executed so as to avoid the collision or the misoperation of the two operations.
In some embodiments, the storage unit 6 includes an image mapping module 601, the image mapping module 601 includes an image mapping between the operation display module 204 and the eyepiece region, and the control unit 7 projects information displayed in the operation display module 204 to the eyepiece region according to the image mapping so that an operator observes the image displayed in the operation display module 204 through the eyepiece region.
Specifically, when the operator operates the handle operation end 1, the control unit 7 operates the doctor console 100 in combination with the preset mapping information stored in the storage unit 6 for operating the doctor console 100, and meanwhile, the control unit 7 can project the operation information of the doctor console 100 displayed on the operation display module 204 to the eyepiece area of the doctor display 205 according to the image mapping stored in the image mapping module 601, so that the operator can directly observe the operation information, and can amplify the content displayed by the operation display module 204, thereby facilitating the observation.
The projection mapping between the operation display module 204 and the eyepiece of the doctor display table 205 is controlled by the control unit 7, and the operator can select to perform the projection mapping at an arbitrary timing as needed.
The control unit 7 performs motion adjustment on the doctor display table 205 through the display table motion map in the doctor display table mapping module 602, performs motion adjustment on the armrest table 203 through the armrest motion map in the armrest table mapping module 603, and simultaneously performs operation adjustment on the doctor control table 100 through the character operation map in the gesture character mapping module 604, thereby completing the whole adjustment process.
It should be noted that, the operations of the respective modules in the control unit 7 are in one-to-one correspondence with the mappings in the doctor display table mapping module 602, the armrest table mapping module 603, and the gesture character mapping module 604, so that the respective modules in the control unit 7 perform the operation processing on the doctor console 100 according to the operation information of the handle operation end 1.
Now, to describe the specific case, the operator identifying unit 4 specifically includes an infrared receiving end and an infrared transmitting end, so as to determine whether the operator is in the eyepiece area, that is, identify whether the face of the operator is in the immersive operation area, and when the control unit determines that the infrared receiving end fails to receive the information transmitted by the infrared transmitting end, determine that the operator is in the eyepiece area and is in operation, otherwise, when it is determined that the face of the operator leaves the eyepiece area and is not in the immersive operation area, the handle operating end 1 controls the function of the doctor console 100 to be started, that is, adjusts the function of the doctor console to be started. The handle rotation recognition module 301, the handle twist recognition module 302, the handle push-pull recognition module 303 and the gesture path recognition module 304 respectively collect the operation information of the handle operation end 1, when the handle twist recognition module 302 collects the operation left handle 101 and the operation right handle 102 which are twisted twice at the same time, the operation display module 204 displays a moving cursor, then the position of the moving cursor is changed by the movement of the operation right handle 102 to adjust, after the moving cursor is moved to the doctor display table 205, the operator confirms the selection of the doctor display table 205 by the operation left handle 101 twist representation, namely, the doctor display table 205 is adjusted; operating the right handle 102 to push forward to control the doctor display table 205 to rise, and operating the right handle 102 to pull backward to control the doctor display table 205 to descend; operating the left handle 101 pushes forward so that the doctor display table 205 is tilted upward, i.e., raised, operating the left handle 101 pushes backward so that the doctor display table 205 is tilted downward, i.e., depressed, operating the left handle 101 and operating the right handle 102 three times twist, and then withdrawing the operation.
When the handle twist recognition module 302 collects that the left handle 101 and the right handle 102 are twisted twice simultaneously, the object selection function is indicated to be entered, a moving cursor is displayed on the operation display module 204, then the position of the moving cursor is changed to adjust through the movement of the right handle 102, after the moving cursor is moved to the armrest platform 203, the armrest platform 203 is confirmed to be selected through the operation of the left handle 101 twist representation, namely, the armrest platform 203 is adjusted, the right handle 102 is operated to rotate clockwise to adjust the armrest platform 203 to be lifted, and the right handle 102 is operated to rotate anticlockwise to adjust the armrest platform 203 to be lowered; after the left handle 101 and the right handle 102 are operated to twist three times, the operation is exited.
The invention also provides an auxiliary method of the surgical robot based on the operation handle, as shown in fig. 6, comprising the following steps:
s1, judging whether an operator is in the operation process or not through an operator identification unit;
s2, if the operator is determined to be in a non-operation process, acquiring operation information on the handle operation end through the handle operation information identification unit, and outputting an instruction by the control unit according to a preset mapping relation stored in the storage unit so as to control at least one of movement or state of the doctor control end.
In the process of the method, during specific operation, the operator face information is collected through the operator identification unit, wherein the operator face information comprises at least one of face information and face distance information, and the operator identification unit is enabled to collect the face information of an operator more accurately through the at least one of the face information and the face distance information, so that information errors are reduced.
And the control unit is used for identifying and judging whether the face of the operator is positioned in the eyepiece area according to the collected face information of the operator, and when the face of the operator is confirmed not to be positioned in the eyepiece area, the operation information on the handle operation end can be obtained through the handle operation information identification unit, wherein the operation information comprises at least one of handle rotation information, handle twist information, handle push-pull information and gesture character path information.
And accurately judging the operation information generated on the handle operation end through one or more of the handle rotation information, the handle twisting information, the handle push-pull information and the gesture character path information in the operation information, so that the control unit can perform corresponding operation processing according to the operation information.
And then the operation information is transmitted to the control unit, the control unit respectively outputs instructions to control the doctor control end to perform state adjustment according to the preset mapping relation stored in the storage unit, and the adjustment work of the doctor control end is completed, wherein the preset mapping relation comprises at least one of image mapping, display table movement mapping, armrest movement mapping and character operation mapping.
Since the specific adjustment process in the method is basically the same as the process principle of the auxiliary system, the detailed description is omitted here.
In some embodiments, the state adjustment of the doctor control includes at least one of a state adjustment of a doctor display table, a state adjustment of an armrest table, and a gesture operation of the doctor control.
In some embodiments, the status adjustment of the doctor display table includes at least one of a height adjustment and a tilt angle adjustment of the doctor display table.
In some embodiments, the condition adjustment of the armrest platform includes at least one of a height adjustment and an inclination angle adjustment of the armrest platform.
In some embodiments, the gesture operation of the doctor control includes at least one of a login operation, a revocation operation, a start operation, and a termination operation.
The invention also provides a storage medium having stored thereon a computer program which, when executed by a processor, implements the above-described method.
The invention further provides a terminal, which comprises a processor and a memory;
the processor is configured to execute the computer program stored in the memory, so that the terminal performs the above method.
The memory is used for storing a computer program.
Preferably, the memory includes: various media capable of storing program codes, such as ROM, RAM, magnetic disk, U-disk, memory card, or optical disk.
The processor is connected with the memory and is used for executing the computer program stored in the memory so as to enable the terminal to execute the method.
Preferably, the processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, abbreviated as CPU), a network processor (Network Processor, abbreviated as NP), etc.; but also digital signal processors (Digital Signal Processor, DSP for short), application specific integrated circuits (Application Specific Integrated Circuit, ASIC for short), field programmable gate arrays (Field Programmable Gate Array, FPGA for short) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
The invention also provides a surgical robot which comprises a doctor control end, a handle operation end with a master-slave operation relation with the doctor control end and an auxiliary system as described above.
While embodiments of the present invention have been described in detail hereinabove, it will be apparent to those skilled in the art that various modifications and variations can be made to these embodiments. It is to be understood that such modifications and variations are within the scope and spirit of the present invention as set forth in the following claims. Moreover, the invention described herein is capable of other embodiments and of being practiced or of being carried out in various ways.

Claims (19)

1. A surgical robotic assistance system, comprising:
the handle operation end is used for an operator to control a doctor control end of the surgical robot;
the handle operation information identification unit is used for acquiring the operation information of the handle operation end;
an operator identifying unit for identifying whether an operator is in the process of surgery;
the storage unit is used for storing the preset mapping relation between the operation information and the doctor control end;
the control unit is respectively in communication connection with the handle operation information identification unit, the operator identification unit and the storage unit and is used for judging that the operator is in a non-operation process according to the information fed back by the operator identification unit and controlling at least one of the movement or the state of the doctor control end according to the received operation information and the preset mapping relation.
2. The surgical robot assistance system according to claim 1, wherein the handle operation information recognition unit includes a handle rotation recognition module for collecting handle rotation information generated on the handle operation end, a handle twist recognition module for collecting handle twist information generated on the handle operation end, and a handle push-pull recognition module for collecting handle push-pull information generated on the handle operation end.
3. The surgical robot assistance system according to claim 1 or 2, wherein the handle operation information recognition unit further includes an operation gesture path recognition module for recognizing gesture character path information generated by an operator operating on the handle operation end.
4. A surgical robot assistance system according to claim 3, wherein the assistance system includes a gesture operation module, when the control unit receives the gesture character path information and determines that the operator is in a non-surgical process according to the information fed back by the operator recognition unit, the control unit outputs gesture operation information through the gesture operation module to control the doctor control terminal to perform adjustment, and the storage unit includes a gesture character mapping module, where the gesture character mapping module includes a character operation map of the doctor control terminal corresponding to the gesture character path information of the handle operation terminal.
5. The surgical robot assistance system of claim 1, wherein the operator recognition unit includes a face recognition module for collecting face information at an eyepiece region of the doctor control side and a distance recognition module for collecting face distance information between an operator face and the eyepiece region of the doctor control side.
6. The surgical robotic assistance system of claim 1, wherein the doctor control end includes a doctor display table and a armrest arm having the armrest table mounted thereon, the control unit controlling at least one of the doctor display table and the armrest arm via the handle manipulation end to adjust at least one of movement and state.
7. The surgical robot assistance system of claim 6, further comprising at least one of a handrail table tilt adjustment module for adjusting a tilt angle of the handrail table, a handrail table lift module for adjusting a height of the handrail table, the storage unit comprising a handrail table mapping module comprising a handrail movement map of the handle manipulation end corresponding to the handrail table.
8. The surgical robot assistance system according to claim 6, wherein the assistance system includes at least one of a doctor display table inclination adjustment module for adjusting an inclination angle of the doctor display table and a doctor display table elevation module for adjusting a height of the doctor display table, the storage unit further includes a doctor display table mapping module including therein an operation information of the handle operation end corresponding to a display table movement map of the doctor display table.
9. The surgical robot assistance system according to claim 7, wherein the doctor control terminal is further provided with an operation display module for displaying at least one of status information of the doctor display table, status information of the armrest table, and gesture operation information.
10. The surgical robot assistance system of claim 9, wherein the gesture operation information includes at least one of login information, revocation information, start information, and end information.
11. A surgical robot assistance system as claimed in claim 9, wherein the storage unit comprises an image mapping module comprising an image mapping between the operation display module and an eyepiece region, the control unit projecting information displayed in the operation display module to the eyepiece region according to the image mapping such that an operator observes the image displayed in the operation display module through the eyepiece region.
12. A method of adjusting a surgical robot, comprising:
providing a storage unit for storing a preset mapping relation between operation information and a doctor control end;
Judging whether an operator is in the process of surgery or not through an operator identification unit;
and if the operator is determined to be in a non-operation process, acquiring the operation information on the handle operation end through the handle operation information identification unit, and outputting an instruction by the control unit according to the operation information and the preset mapping relation stored in the storage unit so as to control at least one of the movement or the state of the doctor control end.
13. The method of claim 12, wherein the operator recognition unit first collects operator face information, the operator face information including at least one of face information and face distance information.
14. The method of claim 12, wherein the operation information includes at least one of handle rotation information, handle twist information, handle push-pull information, and gesture character path information.
15. The method of claim 12, wherein the predetermined mapping relationship comprises at least one of an image map, a display table motion map, a handrail motion map, and a character operation map.
16. The method of claim 12, wherein the state adjustment of the doctor control includes at least one of a state adjustment of a doctor display table, a state adjustment of an armrest table, and a gesture operation of the doctor control;
wherein the state adjustment of the doctor control end comprises at least one of height adjustment and inclination angle adjustment of the doctor control end; the state adjustment of the armrest platform comprises at least one of height adjustment and inclination angle adjustment of the armrest platform; the gesture operation of the doctor control end comprises at least one of login operation, withdrawal operation, start operation and stop operation.
17. A storage medium having stored thereon a computer program which, when executed by a processor, implements the method of any of claims 12 to 16.
18. A terminal, comprising a processor and a memory;
the processor is configured to execute the computer program stored in the memory, to cause the terminal to perform the method according to any one of claims 12 to 16.
19. A surgical robot, comprising a doctor console, a surgical trolley and an auxiliary system according to any one of claims 1 to 11, wherein the doctor console is provided with a handle operation end of the auxiliary system, the surgical trolley is provided with a plurality of mechanical arms, and the handle operation end, the mechanical arms and surgical instruments form a master-slave control relationship.
CN202011179121.9A 2020-10-29 2020-10-29 Surgical robot auxiliary system, method, medium, terminal and surgical robot Active CN114424975B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011179121.9A CN114424975B (en) 2020-10-29 2020-10-29 Surgical robot auxiliary system, method, medium, terminal and surgical robot

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011179121.9A CN114424975B (en) 2020-10-29 2020-10-29 Surgical robot auxiliary system, method, medium, terminal and surgical robot

Publications (2)

Publication Number Publication Date
CN114424975A CN114424975A (en) 2022-05-03
CN114424975B true CN114424975B (en) 2024-02-09

Family

ID=81310194

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011179121.9A Active CN114424975B (en) 2020-10-29 2020-10-29 Surgical robot auxiliary system, method, medium, terminal and surgical robot

Country Status (1)

Country Link
CN (1) CN114424975B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115969524A (en) * 2022-12-27 2023-04-18 哈尔滨思哲睿智能医疗设备股份有限公司 Operation control system, control method and electronic equipment

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106456148A (en) * 2014-03-19 2017-02-22 直观外科手术操作公司 Medical devices, systems, and methods using eye gaze tracking
CN107028662A (en) * 2015-12-11 2017-08-11 美好罗伯特有限公司 Long-distance operating device and telesurgery systems
CN107320186A (en) * 2017-07-31 2017-11-07 成都中科博恩思医学机器人有限公司 The mobile handrail of operating robot surgeon console
CN107874834A (en) * 2017-11-23 2018-04-06 苏州康多机器人有限公司 A kind of open surgeon consoles of 3D applied to laparoscope robotic surgical system
CN108472097A (en) * 2016-09-21 2018-08-31 威博外科公司 user console system for robotic surgery
CN109009447A (en) * 2017-06-08 2018-12-18 美好罗伯特有限公司 The long-distance operating device of medical apparatus
CN208989131U (en) * 2017-11-23 2019-06-18 苏州康多机器人有限公司 A kind of open surgeon console of 3D applied to laparoscope robotic surgical system
JP2020049300A (en) * 2019-12-25 2020-04-02 株式会社メディカロイド Remote handling equipment and remote surgery system

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106456148A (en) * 2014-03-19 2017-02-22 直观外科手术操作公司 Medical devices, systems, and methods using eye gaze tracking
CN107028662A (en) * 2015-12-11 2017-08-11 美好罗伯特有限公司 Long-distance operating device and telesurgery systems
CN108472097A (en) * 2016-09-21 2018-08-31 威博外科公司 user console system for robotic surgery
CN109009447A (en) * 2017-06-08 2018-12-18 美好罗伯特有限公司 The long-distance operating device of medical apparatus
CN107320186A (en) * 2017-07-31 2017-11-07 成都中科博恩思医学机器人有限公司 The mobile handrail of operating robot surgeon console
CN107874834A (en) * 2017-11-23 2018-04-06 苏州康多机器人有限公司 A kind of open surgeon consoles of 3D applied to laparoscope robotic surgical system
CN208989131U (en) * 2017-11-23 2019-06-18 苏州康多机器人有限公司 A kind of open surgeon console of 3D applied to laparoscope robotic surgical system
JP2020049300A (en) * 2019-12-25 2020-04-02 株式会社メディカロイド Remote handling equipment and remote surgery system

Also Published As

Publication number Publication date
CN114424975A (en) 2022-05-03

Similar Documents

Publication Publication Date Title
US20210157403A1 (en) Operating room and surgical site awareness
US20230157776A1 (en) Systems and methods for constraining a virtual reality surgical system
US11612446B2 (en) Systems, methods, and computer-readable program products for controlling a robotically delivered manipulator
CN111134849A (en) Surgical robot system
JP2021151519A (en) Surgical system and display method
US11234779B2 (en) Handheld user interface device for a surgical robot
JP6856594B2 (en) Surgical system and display method
KR101798590B1 (en) Master robot, surgical robot system using thereof, control method thereof, and recording medium thereof
US11633247B2 (en) Graphical user guidance for a robotic surgical system
JP2021019949A (en) Surgery system
CN115334992A (en) Virtual console for controlling surgical robot
CN114424975B (en) Surgical robot auxiliary system, method, medium, terminal and surgical robot
JP6902639B2 (en) Surgical system
KR20120045734A (en) Master iput device of surgical robitc system and method of controlling surgical robitc system using the same
CN116423547A (en) Surgical robot pedal control system, method, readable medium and surgical robot
US20220096197A1 (en) Augmented reality headset for a surgical robot
JP2021062216A (en) Surgical system and display method
JP7128326B2 (en) surgery system
EP4344668A1 (en) Remote surgery support system
US20230395250A1 (en) Customization, troubleshooting, and wireless pairing techniques for surgical instruments
US20210038329A1 (en) Augmented reality using eye tracking in a robot assisted srugical system
WO2023202291A1 (en) Surgical robot system and control device apparatus thereof
KR101501955B1 (en) Remote Control System of Surgical Operating Room Equippments and Control Method therefor
EP4356396A1 (en) Customization, troubleshooting, and wireless pairing techniques for surgical instruments
CN113384291A (en) Medical ultrasonic detection method and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant