CN115281587B

CN115281587B - Method and device for controlling flexible endoscope robot

Info

Publication number: CN115281587B
Application number: CN202210773070.5A
Authority: CN
Inventors: 陈健; 刘宏斌
Original assignee: Institute of Automation of Chinese Academy of Science
Current assignee: Institute of Automation of Chinese Academy of Science
Priority date: 2022-06-30
Filing date: 2022-06-30
Publication date: 2023-08-04
Anticipated expiration: 2042-06-30
Also published as: CN115281587A

Abstract

The invention provides a method and a device for controlling a flexible endoscope robot, which relate to the technical field of medical appliances, and the method comprises the following steps: receiving a first input of a user; the first input is an operation that a user desires to insert the endoscope and the sheath assembly integrally into the endoscope; in response to a first input, sending a first instruction to the drive mechanism, the first instruction for instructing the drive mechanism to drive the insertion portion endoscope and the sheath assembly integrally into the endoscope; wherein the flexible endoscope robot comprises a flexible endoscope and a driving mechanism; the flexible endoscope includes an insertion portion endoscope and an outer sheath assembly. The invention designs a flexible endoscope structure which is inserted in a grading manner, based on the method for controlling the flexible endoscope robot, the flexible endoscope can be enabled to penetrate into a cavity with a finer branch, so that micro-wound biopsy sampling in the cavity with the finer branch is realized, and the efficiency and the accuracy of operation are improved.

Description

Method and device for controlling flexible endoscope robot

Technical Field

The invention relates to the technical field of medical equipment, in particular to a method and a device for controlling a flexible endoscope robot.

Background

The flexible endoscope robot has the characteristics of accurate positioning, accurate control, high stability, low operation requirement, short training time of doctors and the like, and can assist the doctors to safely and reliably perform operations such as examination, biopsy taking, lavage, medicine delivery and the like in the bronchi with finer branches.

At present, some flexible endoscope robots in the market have thicker outer diameter of the tail end endoscope body, cannot reach the cavity of the far-end finer branch, and cannot realize micro-wound biopsy sampling and treatment in the cavity of the finer branch.

Disclosure of Invention

The invention provides a method and a device for controlling a flexible endoscope robot, which are used for solving the defect that a cavity channel with a far-end finer branch cannot be reached in the prior art and realizing micro-wound biopsy and treatment in the cavity channel with the finer branch.

The invention provides a method for controlling a flexible endoscope robot, comprising the following steps:

receiving a first input of a user; the first input is an operation of the user to insert the endoscope and the sheath assembly integrally;

responsive to the first input, sending a first instruction to a drive mechanism, the first instruction for instructing the drive mechanism to drive the insertion portion endoscope and the sheath assembly into the endoscope as a whole;

wherein the flexible endoscope robot comprises a flexible endoscope and the driving mechanism; the flexible endoscope includes the insertion portion endoscope and the outer sheath assembly.

Optionally, the method further comprises:

receiving a second input from the user; the second input is an operation that the user desires the insertion portion endoscope and the outer sheath assembly to be bent integrally;

In response to the second input, a second instruction is sent to the drive mechanism, the second instruction for instructing the drive mechanism to drive the insertion portion endoscope and the outer sheath assembly to bend integrally.

Optionally, the method further comprises:

receiving a third input from the user; the third input is an operation that the user desires the insertion portion endoscope to enter;

and in response to the third input, sending a third instruction to the driving mechanism, wherein the third instruction is used for instructing the driving mechanism to drive the insertion portion endoscope to enter the lens.

Optionally, the method further comprises:

receiving a fourth input from the user; the fourth input is an operation that the user desires the insertion portion endoscope to bend;

in response to the fourth input, a fourth instruction is sent to the drive mechanism, the fourth instruction being for instructing the drive mechanism to drive the insertion portion endoscope to bend.

Optionally, the method further comprises:

receiving a fifth input from a user if a length of the distal end of the insertion portion endoscope extending beyond the distal end of the outer sheath assembly is greater than a first threshold; the fifth input is an operation that the user desires the insertion portion endoscope to be retracted;

and in response to the fifth input, sending a fifth instruction to the driving mechanism, wherein the fifth instruction is used for instructing the driving mechanism to drive the insertion portion endoscope to withdraw.

Optionally, the method further comprises:

in the event that it is determined that the distal end of the insertion portion endoscope is retracted to a length of the distal end of the outer sheath assembly equal to a first threshold, a sixth instruction is sent to the drive mechanism, the sixth instruction being for instructing the drive mechanism to stop driving the insertion portion endoscope.

The present invention also provides an apparatus for controlling a flexible endoscope robot, comprising:

the receiving module is used for receiving a first input of a user; the first input is an operation of the user to insert the endoscope and the sheath assembly integrally;

the response module is used for responding to the first input and sending a first instruction to the driving mechanism, and the first instruction is used for instructing the driving mechanism to drive the insertion part endoscope and the outer sheath assembly to integrally enter the endoscope;

The invention also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing a method of controlling a flexible endoscope robot as described in any of the above when executing the program.

The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a method of controlling a flexible endoscopic robot as described in any of the above.

The invention also provides a computer program product comprising a computer program which, when executed by a processor, implements a method of controlling a flexible endoscopic robot as described in any of the above.

The invention designs a flexible endoscope structure which is inserted in a grading manner, based on the method for controlling the flexible endoscope robot, the flexible endoscope can be enabled to penetrate into a cavity with a finer branch, so that micro-wound biopsy sampling in the cavity with the finer branch is realized, and the efficiency and the accuracy of operation are improved.

Drawings

In order to more clearly illustrate the invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.

FIG. 1 is one of the block diagrams of a flexible endoscope provided by the present invention;

FIG. 2 is a block diagram of an insertion portion endoscope provided by the present invention;

FIG. 3 is a second block diagram of the insertion portion endoscope provided by the present invention;

FIG. 4 is a block diagram of an insertion portion flexible controllable instrument provided by the present invention;

FIG. 5 is a cross-sectional view of an insertion portion flexible controllable instrument provided by the present invention;

FIG. 6 is a cross-sectional view of an insertion portion endoscope provided by the present invention;

FIG. 7 is a cross-sectional view of an insert mirror body provided by the present invention;

FIG. 8 is a block diagram of a camera assembly provided by the present invention;

FIG. 9 is a block diagram of the distal end of an insertion portion flexible controllable instrument provided by the present invention;

FIG. 10 is a block diagram of one of the sheath assemblies provided by the present invention;

FIG. 11 is a second block diagram of the sheath assembly provided by the present invention;

FIG. 12 is a cross-sectional view of a sheath flexible controllable instrument provided by the present invention;

FIG. 13 is a cross-sectional view of a sheath body provided by the present invention;

FIG. 14 is a block diagram of a sheath tip provided by the present invention;

FIG. 15 is a block diagram of the distal end of the sheath assembly provided by the present invention;

FIG. 16 is a second block diagram of a flexible endoscope provided by the present invention;

FIG. 17 is one of the cross-sectional views of the flexible endoscope provided by the present invention;

FIG. 18 is a second cross-sectional view of the flexible endoscope provided by the present invention;

FIG. 19 is a third block diagram of the flexible endoscope provided by the present invention;

FIG. 20 is a fourth block diagram of a flexible endoscope provided by the present invention;

FIG. 21 is a fifth block diagram of a flexible endoscope provided by the present invention;

FIG. 22 is a block diagram of a flexible endoscope robot provided by the present invention;

FIG. 23 is an overall flow of flexible endoscopic robotic system manipulation provided by the present invention;

FIG. 24 is a sixth block diagram of a flexible endoscope provided by the present invention;

FIG. 25 is a schematic view of a method of controlling a flexible endoscope robot provided by the present invention;

FIG. 26 is a schematic diagram of a mirror-in operation flow provided by the present invention;

FIG. 27 is a schematic illustration of a steering operation flow provided by the present invention;

FIG. 28 is a schematic diagram of mirror-back control logic provided by the present invention;

FIG. 29 is a schematic diagram of a push rod map ratio control logic provided by the present invention;

FIG. 30 is a schematic diagram of steering control logic provided by the present invention;

FIG. 31 is a schematic diagram of master hand mapping ratio control logic provided by the present invention;

FIG. 32 is a schematic illustration of a bronchial surgical target point and navigation path provided by the present invention;

FIG. 33 is a schematic view of a procedure for insertion of a flexible endoscope robot into a target site provided by the present invention;

FIG. 34 is a schematic view of an apparatus for controlling a flexible endoscope robot provided by the present invention;

fig. 35 is a schematic structural diagram of an electronic device according to the present invention.

Detailed Description

The invention has a bronchoscope as a typical application of flexible endoscopes. Traditional bronchoscopy is not ideal for the reliability of peripheral pulmonary nodule diagnosis, especially those that are small, lack bronchogenic, benign. Compared with a bronchoscope controlled by a non-robot, the bronchoscope operation robot has the characteristics of accurate positioning, accurate control, high stability, low operation requirement, short doctor training time and the like technically; functionally, the bronchosurgical robot can assist a doctor to perform operations such as examination, biopsy, lavage, drug delivery and the like safely and reliably in a bronchus with a finer branch.

Some bronchoscope robots existing in the current market are thicker in outer diameter of the tail end of the bronchoscope robot, cannot reach a far-end cavity, can only finish functions of sputum suction, alveolar lavage and the like at present, and cannot realize micro-wound biopsy sampling.

At present, two schemes are mainly adopted for an endoscope part of a bronchoscope robot. The first is to simply combine the existing mature electronic bronchoscope with a driving mechanism, finish preliminary endoscope entering and steering operation by means of a mechanical arm, and ensure that the endoscope is large in size and cannot enter into a deep bronchus. The second type is a bronchoscope equipped with an independent camera assembly, which is taken out after reaching the target position and inserted into a surgical tool for surgical operation. The bronchoscope can not provide real-time image feedback during operation diagnosis such as biopsy sampling, cutting, puncture and the like, and has safety risks. The two types of endoscopes are commonly used in a bronchoscope system which is repeatedly used for a plurality of times, risks of complex operation, high maintenance cost, cross infection and the like caused by repeated disinfection are faced, and the manufacturing cost and the operation cost are high.

Secondly, the existing bronchus mainly adopts the traditional polymer plastic tube embedded with a metal net as the endoscope body of the endoscope, and the tube internally comprises more channel tubes for isolating traction guide wires, tool channels and other various wire harnesses, so that the assembly difficulty of the endoscope body is high, the cost is high, and friction exists between the pipelines.

Finally, the bending flexible controllable instruments of the bronchoendoscopes commonly used at present mainly adopt a riveting structure or a laser cutting forming process. The riveting flexible controllable instrument is characterized in that a plurality of flexible controllable instrument units are connected in series into a whole through riveting, and the flexible controllable instrument is driven by a traction guide wire. The flexible controllable laser cutting instrument cuts a complete conduit through a laser cutting processing technology to form a hollowed-out structure at one time. Both types of flexible controllable instruments require additional structural parts to mount the traction guide wire for driving, which increases the outer diameter size of the endoscope, limits the scope of bronchial intervention, and increases the assembly complexity and cost of the endoscope.

As shown in fig. 1, an embodiment of the present invention provides a bronchoscope, which mainly includes two parts: an insertion portion endoscope 1 and an outer sheath assembly 2. The sheath assembly is provided with a through cavity, and the diameter of the through cavity is larger than any section of the endoscope of the insertion part, so that the endoscope of the insertion part can pass through the cavity of the sheath assembly. As shown in fig. 2 and 3, the insertion endoscope 1 includes an insertion flexible controllable instrument 11, an insertion connector 12, an insertion scope body 13, a camera assembly 14, a tool channel 15, and an insertion flexible controllable instrument protective sheath 16. The insertion flexible steerable instrument 11 is tubular with an axially hollow lumen that may be used to pull a guide wire 112, camera harness, tool channel 15, etc. through the insertion and aligned with the lumen of the insertion scope 13 and through the insertion scope 13. The tool passage 15 is provided so as to extend through the entire insertion portion scope 13, and surgical tools such as biopsy forceps and cytobrushes can be passed through the tool passage 15. The tool channel 15 is produced using a disposable precision extrusion process using materials including PE (polyethylene), PEBAX (polyether amide block copolymer), TPU (polyurethane rubber) and the like.

The insertion portion flexible controllable instrument protective sheath 16 has a certain elasticity and hydrophilicity, can protect the internal insertion portion flexible controllable instrument 11, enhance the resilience of the insertion portion flexible controllable instrument 11, and provide a smooth surface. The insertion portion flexible controllable instrument protective sheath 16 is mounted as follows:

the insertion flexible controllable instrument protective sheath 16 is passed through a thin-walled hollow metal tube of slightly larger inside diameter and slightly shorter length, and the insertion flexible controllable instrument protective sheath 16 is turned over at both ends to be sleeved on both ends of the metal tube. At this time, the outer wall of the insertion portion flexible controllable instrument protection sheath 16 and the inner wall of the metal tube form a closed space. One side of the metal tube is provided with a micro small hole, and a hollow metal thin tube is connected (welded, bonded and the like) at the small hole, and the middle cavity of the metal thin tube is aligned with the micro small hole. The metal tubule is connected with a negative pressure device (such as a vacuum pump, a getter and the like), so that air in a closed space formed between the outer wall of the insertion part flexible controllable instrument protective sleeve 16 and the inner wall of the metal tube can be pumped away, the inner diameter of the insertion part flexible controllable instrument protective sleeve 16 is enlarged under the action of atmospheric pressure, the outer wall of the insertion part flexible controllable instrument protective sleeve 16 is tightly attached to the inner wall of the metal tube, at the moment, the insertion part flexible controllable instrument 11 passes through the insertion part flexible controllable instrument protective sleeve 16, the negative pressure device is removed, and the protective sleeve is tightly attached to the insertion part flexible controllable instrument 11.

As shown in fig. 4 and 5, the insertion portion flexible controllable device 11 has a regularly arranged hollow structure, or the width and the gap size of the hollow structure along the axis linearly or nonlinearly decrease/increase along a certain direction, so that the bending stiffness of the insertion portion flexible controllable device 11 gradually changes, and the tail end or the root portion of the insertion portion flexible controllable device 11 is softer. And a plurality of groups of insertion portion guide wire channels 111 are distributed along the axial direction, each group of insertion portion guide wire channels 111 is concave towards the inner wall of the insertion portion flexible controllable appliance 11, guide wires 112 are pulled through the insertion portion in the insertion portion guide wire channels 111, and the insertion portion guide wires 112 are welded or adhered in the insertion portion guide wire channels at the distal end of the insertion portion flexible controllable appliance 11. The design can uniformly distribute 3 groups or 4 groups of insertion part guide wire channels along the circumferential direction. The insertion portion flexible controllable device 11 is manufactured by adopting an additive manufacturing integrated molding process, and multiple groups of insertion portion guide wire channels 111 and hollow structures can be integrally manufactured. Because no additional insertion guide wire channel parts are needed, the outer diameter of the insertion flexible controllable instrument 11 can be reduced, so that the insertion endoscope can easily enter into a higher bronchus branch to perform operation, and single part is used for reducing the process complexity and the assembly complexity and reducing the cost.

Bending of the insertion portion flexible controllable device 11 may be controlled by the insertion portion traction guide wire 112. The insertion portion pulls the guide wire 112 to control the stretched/relaxed state of the guide wire by the driving mechanism. When one side of the guide wire of the insertion portion flexible controllable device 11 is tensioned and all the guide wires of the other side are relaxed, the insertion portion flexible controllable device 11 will bend to that side. Bending of the insertion portion flexible controllable appliance 11 in any direction can be achieved by applying different tensions or displacements to the different guidewires.

As shown in fig. 6, the insertion portion body 13 and the insertion portion flexible controllable device 11 are attached by an insertion portion connector 12. The proximal end of the connecting piece is connected with the insertion part mirror body 13 and is connected with the inner step, the distal end of the connecting piece is connected with the insertion part flexible controllable appliance 11, and the insertion part flexible controllable appliance 11 is inserted into the connecting piece and is connected with the insertion part mirror body 13. The outer diameter of the connector, when installed, is consistent with the insert flexible controllable instrument protective sheath 16.

The insertion portion traction guide wire, the camera harness, the tool channel and other structures passing through the insertion portion flexible controllable appliance 11 can pass through the cavity corresponding to the endoscope body of the insertion portion. Fig. 7 shows an example of a cross-section of an insertion section mirror body, including three insertion section traction guide wire channels 131, a camera harness channel 132, a magnetic positioning sensor channel 133, two symmetrical fiber channels 134, and a tool channel 135 distributed along 120 ° in the circumferential direction. The insertion mirror body 13 is manufactured by a precision extrusion process, and the material used is not limited to PE (polyethylene), PEBAX (polyether amide block copolymer), TPU (polyurethane rubber) and the like. The different channels are independent channels which are parallel to each other, and the wall thickness among the channels can be selected to be 0.05mm, 0.1mm, 0.15mm, 0.2mm and the like. In order to ensure that the mirror body is not easy to bend when the catheter is inserted, a material with the Shore hardness reaching more than 50D can be selected. The multi-channel mirror body extruded precisely can effectively isolate different wire harnesses and prevent the wire harnesses from winding; the friction between the wire harnesses and the outer wall of the wire harnesses is reduced; the processing precision is high, and the outer diameter size can be limited while a plurality of cavities are extruded; the insert body 13 has high processing efficiency and low cost, and is very suitable for the body of a disposable bronchoscope.

As shown in fig. 8, the camera assembly 14 includes a camera 141, (two) LED lights 142, a transparent camera tip 143. The camera and the LED are respectively arranged at the front end of the camera, and the fixing mode is not limited by bonding. The LED can emit light through the front end of the camera to illuminate the field of view of the camera. A tool passage 15 is mounted on the camera head and aligned with the passage axis of the camera head from which the tool will exit after passing through the tool passage.

To secure the camera tip and orient the tip, the tip is inserted from the head into the lumen of the flexible steerable instrument 11 and the U-shaped slot is used to limit its circumferential rotation, as shown in fig. 9. The front end of the camera can be manufactured by using 3D printing, injection molding or machining and the like.

As shown in fig. 10, the outer sheath assembly 2 includes a sheath flexible controllable instrument 21, a sheath connector 22, a sheath shaft 23, a sheath tip 24, and a sheath flexible controllable instrument protective sheath 25. The sheath flexible controllable instrument protection sheath 25 is mounted in the same way as the insertion section flexible controllable instrument protection sheath 16. The matching relationship between the sheath connector 22 and the sheath flexible controllable instrument 21 and the sheath endoscope body 23 is similar to that of an endoscope at the insertion part, the outer diameter of the sheath connector is consistent with that of the assembled sheath flexible controllable instrument protective sleeve, and the installation method is not repeated here.

As shown in fig. 11, the sheath flexible controllable device 21 includes a hollow structure, a sheath traction guide wire channel 211 and a sheath traction guide wire 212, wherein the sheath traction guide wire channels are distributed in 3, 4 or 6 groups along the circumferential direction, and at least two groups along the axial direction. And each group of channels is internally penetrated by a guide wire, the guide wire is fixed in the channel at the forefront end of the flexible controllable instrument, and the fixing mode is not limited by welding and bonding. In order to strengthen the rigidity of the sheath flexible controllable instrument, the outer diameter and the wall thickness of the sheath flexible controllable instrument are larger than those of the insertion part flexible controllable instrument, the length of the sheath flexible controllable instrument is shorter, and the overall rigidity is high.

As shown in FIG. 12, the maximum allowable passage cylinder boundary 213 inside the sheath flexible controllable instrument is larger than any cross-sectional diameter of the insertion portion endoscope in order for the insertion portion endoscope to pass smoothly through the sheath flexible controllable instrument. And, the inner space still allows the insertion portion endoscope to pass through when the sheath flexible controllable instrument is bent.

Similar to the insertion section endoscope, the outer sheath assembly also uses a precisely extruded multichannel catheter as the sheath shaft 21. The material used is not limited to a polymer material such as PE (polyethylene), PEBAX (polyether amide block copolymer), or TPU (polyurethane rubber).

Fig. 13 is an example thereof, including 6 guide wire lumens 231 and 1 insertion portion lumen 232 circumferentially distributed, and the guide wire lumens may also be used as optical fiber lumens when fewer than 6 guide wire lumens are actually used. The inner diameter of the outer sheath assembly 2 is larger than any cross-sectional diameter of the insertion section endoscope 1. In order to improve the endoscope advancing efficiency and provide support for the insertion portion endoscope 1, the hardness of the sheath body 21 is generally greater than that of the insertion portion endoscope body, and a hardness material such as 60D, 70D, 80D, 90D or the like can be selected for extrusion.

To prevent exposure of the sharp edges of the sheath flexible controllable instrument, which could cause injury to the human body and the insertion portion endoscope, the distal end of the outer sheath assembly 2 is provided with a sheath tip 24. As shown in fig. 14 and 15, the sheath tip 24 is inserted into the sheath flexible controllable device 21 from the distal end, and is abutted against the head, and fixed by means of gluing or the like. The sheath tip 24 has an outer diameter that is consistent with the sheath flexible controllable instrument sheath 25. Both the surface and the inner edge of the sheath tip 24 are rounded.

The combined design of the insertion portion endoscope 1 and the outer sheath assembly 2 adopted by the invention has the great advantage that the combined movement of the two groups of flexible instruments which can be bent and controlled omnidirectionally enables the bronchoscope to enter deeper branches in the bronchus of the lung.

When the system is initialized, both the insertion portion endoscope 1 and the traction guide wire of the outer sheath assembly 2 are in a relaxed state, and neither is bent. Typically, in the first stage of bronchoscopy, most of the structure of the insertion endoscope is retracted within the sheath assembly, and only the camera tip extends from the anterior segment of the sheath assembly. At this time, the insertion portion endoscope and the outer sheath assembly can be regarded as a single integral endoscope, and the driving mechanism drives the robot system to perform integral endoscope feeding and steering movements. Fig. 16 illustrates the bronchoscope robotic system in an initial state, with both the insertion portion endoscope and the sheath assembly guidewire in a free state. Figures 17 and 18 show the insertion portion endoscope and the sheath assembly guidewire both in a free state, with both sides of the bronchoscope. Fig. 19 illustrates the bending state of the bronchoscope when the sheath flexible controllable instrument is bent.

When the bronchoscope robotic system is climbing to near the target location, for example, to the superior branch of the target location or to a branch where the sheath assembly cannot access, the drive mechanism cannot drive the entire advancing mirror. Because the rigidity of the outer sheath component is much larger than that of the insertion part flexible controllable instrument and the insertion part endoscope body, the driving mechanism maintains the position and the bending angle of the outer sheath component, and independently pushes out the insertion part endoscope to drive the insertion part endoscope to enter the endoscope and steer. The outer sheath assembly provides support for the insertion portion endoscope. Fig. 20 shows the operation of pushing out the insertion section endoscope while the outer sheath assembly is kept in shape. Figure 21 shows the working state of the insertion section endoscope in bending and steering, while the outer sheath assembly is in shape.

When the endoscope of the insertion part of the bronchoscope robot system continues to climb to the target position, the surgical tool stretches out of the tool channel to perform surgical operations such as sampling, puncture and the like.

The bronchoscope provided by the invention can be used once or repeatedly.

According to the bronchoscope provided by the invention, a two-stage combination scheme of the insertion part endoscope and the outer sheath assembly is adopted, and the combined movement of the outer sheath assembly and the insertion part endoscope can realize that the bronchoscope can enter into deep branches of bronchi. The outer sheath assembly is capable of being bent in all directions, and the bending angle can meet the conventional requirement of branching and bending of bronchi. The insertion portion endoscope is provided with a high-definition camera and a standard tool channel, can enter branches with more than 7 stages of bronchus, can be bent in all directions, and the bending angle meets the requirements of most bronchus branches. The outer sheath component has high rigidity, and can provide support for the insertion part endoscope and the insertion part endoscope body, so that the insertion part endoscope further penetrates into bronchus.

A precision extrusion multichannel catheter was used as the scope for the insert endoscope and sheath assembly. All the needed channels can be formed by the multichannel insertion tube at one time, and the multichannel insertion tube has the advantages of high precision, quick processing and multiple material selectivity. The multichannel insertion tube is used as the mirror body, various wire harnesses can be effectively combed, redundant through pipes are not needed, the number of parts in the mirror body is greatly reduced, the assembly complexity is reduced, the cost is greatly reduced, and the multichannel insertion tube is very suitable for disposable bronchoscopes. The precision extrusion processing precision is high, and only extremely thin wall thickness is needed during design, so that the outer diameter of the mirror body can be miniaturized.

The insertion part flexible controllable instrument and the sheath flexible controllable instrument are both single parts, and the insertion part traction guide wire channel, the camera front end mounting structure, the hollow structure and the like are included, so that the number of parts and the assembly difficulty of the flexible controllable instrument unit can be greatly reduced by using the additive manufacturing technology to manufacture the flexible controllable instrument, the manufacturing cost of the flexible controllable instrument unit is reduced, and the flexible controllable instrument is suitable for a disposable bronchus. The additive manufacturing machine has high precision, and the wall thickness of the flexible controllable instrument units is 0.05 mm-0.5 mm.

The master-slave control mode is a common operation mode of a medical robot system, and a doctor generally controls a slave end of the robot according to some feedback information (such as images, forces and the like) through a master manipulator and other input devices. The operation assembly of the existing endoscope is generally a handle, operations which can be completed include bending of the tail end of the endoscope, torsion of the whole endoscope and the like, the control precision of the movement such as insertion, steering precision and the like of the endoscope is required to be improved, and the existing operation mode is not applicable to a more complex endoscope robot system. In part of endoscope robot systems, the operation mode of the traditional endoscope handle is still adopted, so that the master-slave operation advantage of the robot system, namely, simple operation, is not maximized, physical burden of doctors is reduced, and doctors are assisted to finish accurate operation.

The prior master-slave control methods of the surgical robot system for minimally invasive surgery mainly comprise two types, wherein the first type adopts an improved design based on a common endoscope handle, so that the improved design can meet the basic control of the surgical robot, such as bending (also called steering) of a flexible controllable instrument, and auxiliary functions of pumping, flushing, sputum suction and the like. Or an operator uses a command input device similar to a keyboard to adjust the gesture and the azimuth of the robot, so as to realize the control of insertion, steering and the like of the endoscope. Such master-slave control methods may not provide haptic feedback to the physician or provide less or inaccurate haptic feedback. The second kind of master-slave control method is to send the expected motion track to the flexible controllable instrument at the slave end through the equipped master manipulator, and control the flexible controllable instrument to complete the corresponding operation. And the main operator can provide a certain tactile feedback information for doctors. However, when the doctor does not directly contact the endoscope, but the movement of the endoscope is controlled by the corresponding driving mechanism, the force information fed back by the master hand is still insufficient to safely and efficiently complete the operation. For example, primary manual force feedback in only a single direction can make it difficult for a physician to determine the contact or collision of the endoscope with the inner wall of the natural lumen at this time. The feedback image can only provide information in the front view of the camera, and the current position of the endoscope and the front cavity wall distance cannot be estimated. In addition, the model of the flexible controllable instrument in the endoscope robot system is nonlinear, and when modeling errors and external disturbance exist, the open loop control method can generate larger accumulated errors, so that the operation cannot be completed accurately.

Based on the technical problems, the embodiment of the invention also provides a method for controlling the bronchoscope robot, and based on the method for controlling the bronchoscope robot, the bronchoscope can be enabled to penetrate into a cavity with a finer branch, so that micro-wound biopsy sampling in the cavity with the finer branch is realized, and the efficiency and accuracy of operation are improved.

As shown in fig. 22, the bronchoscope robot system provided by the embodiment of the invention includes a force feedback master arm 207 (abbreviated as "master arm") at the operation end of a doctor, a lens feeding push rod 209 (abbreviated as "push rod"), a pedal 208, and a display 206 capable of displaying various sensor and estimation algorithm information, and the aspect of an operation table includes an operation table 205, a patient 204, a cart 210, a mechanical arm 201, a driving mechanism 203, and a bronchoscope 202. The doctor sits at the operation end, the left hand and the right hand respectively operate the push rod and the main hand (the left hand and the right hand can also respectively operate the main hand and the push rod), feedback information on the display is observed, the bronchoscope is driven to enter the patient body through the nasal cavity or the oral cavity, and the bronchoscope is driven to reach the focus position in the bronchus through a series of operation mode switching and control instructions.

The input devices of the bronchoscope robot are all in accordance with the ergonomic arrangement, a doctor only needs to sit on a chair, the left hand and the right hand respectively control the push rod and the main hand, the pedal is controlled by the feet, eyes look at the information of the display, and the bronchoscope robot is controlled. The three functions of eyes, hands and feet can effectively relieve the fatigue of doctors in long-term operation. And the operation table is far away from the surgical instrument, so that risks of cross infection, radiation risk, body fluid splashing and the like are reduced.

Fig. 23 is a general flow chart of the bronchoscope robot system manipulation provided by the present invention, as shown in fig. 23, a doctor inputs desired control signals such as control mode switching, flexible controllable instrument movement, or operation of surgical tools, etc., to equipped input devices including pedals, push rods, force feedback master hands, etc., according to the real-time status of the bronchoscope robot shown on the display. Signal acquisition devices, such as acquisition cards, analog-to-digital converters (ADCs), communication devices, etc., acquire the input signals into the memory of the computer. After being processed by a control algorithm in a computer, the control system sends signals to a driving mechanism, drives the bronchoscope to complete corresponding actions, and various sensing devices (such as a camera, a force sensor and the like) and algorithms (such as a depth estimation algorithm, a collision detection algorithm, a contact force state algorithm and the like) update feedback information in real time, and feed back the responsive sensing force to a force feedback master hand and a push rod, update various sensing information and early warning signals in real time through images or characters, and assist a doctor to continuously perform operation.

As shown in fig. 24, the bronchoscope comprises two parts of an insertion part endoscope 1 and an outer sheath assembly 2, wherein the insertion part endoscope 1 comprises a camera assembly 14, an insertion part flexible controllable instrument protective sleeve 16, an insertion part connecting piece 12 and an insertion part endoscope body 13. The outer sheath assembly 2 includes a sheath tip 24, a sheath flexible controllable instrument protective sheath 25, a sheath connector 22, and a sheath barrel 23. The insertion part flexible controllable instrument and the sheath flexible controllable instrument can be driven by the driving mechanism to realize omnibearing large-angle bending and steering. The camera component of the insertion portion endoscope can transmit back the image in real time and display the image on the display, and transmit the image to the buffer memory for use by algorithms such as depth estimation and the like. The insertion portion scope and the sheath scope can provide channels for various internal harnesses, surgical tools and the like on one hand, and the optical fiber-based force sensor (the insertion portion scope force sensor 17 and the sheath scope force sensor 26) with spiral structures on the other hand can collect extrusion contact force of the inner cavity of the bronchus.

In the embodiment of the invention, the insertion part endoscope and the outer sheath assembly can be respectively and independently driven (comprising the endoscope advancing, the endoscope retracting and the endoscope bending), and the insertion part endoscope can freely pass through the outer sheath assembly.

In an embodiment of the present invention, a bronchoscope includes: the three working modes comprise an integral lens feeding mode, an integral bending mode, an inserting portion endoscope lens feeding mode, an inserting portion endoscope bending mode, an integral lens withdrawing mode and an inserting portion endoscope lens withdrawing mode.

For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

Fig. 25 is a schematic view of a method for controlling a bronchoscope robot according to the present invention, and as shown in fig. 25, the present invention provides a method for controlling a bronchoscope robot, an apparatus, e.g., a controller, a computer, etc., of which an execution body can control the bronchoscope robot, the method comprising:

Step 2501, receiving a first input of a user; the first input is an operation of the user to insert the endoscope and sheath assembly together.

Step 2502, in response to the first input, sending a first instruction to a drive mechanism, the first instruction for instructing the drive mechanism to drive the insertion portion endoscope and the sheath assembly into an endoscope;

wherein the bronchoscope robot comprises a bronchoscope and the driving mechanism; the bronchoscope includes the insertion portion endoscope and the outer sheath assembly.

Specifically, as shown in fig. 16, generally, before the operation starts or at an early stage of the operation, the traction guide wires of the insertion portion endoscope and the outer sheath assembly are in a loose state, only the camera assembly is exposed to the insertion portion endoscope, and the insertion portion endoscope and the outer sheath assembly are integrally advanced as a whole (equivalent to a bronchoscope).

For example, at least two independent pushrods (a first pushrod and a second pushrod) are arranged on the left hand side of a doctor, wherein one of the pushrods controls the whole endoscope feeding and the whole endoscope withdrawing, and the other one controls the insertion part endoscope feeding and the insertion part endoscope withdrawing. Each push rod has a force feedback function, and feedback is not limited to the functions of pushing resistance change, pushing soft limit and the like. Each push rod has a zero position and does not have an automatic zero return function.

The push rod has two directions of movement, and the movement direction determines the operations of advancing or retracting the endoscope or the outer sheath assembly of the bronchoscope insertion part. In order to prevent the damage to the inner wall of the trachea caused by the false operation of the propulsion of a doctor, the bronchoscope robot system provided by the embodiment of the invention can activate the function of the push rod only when the pedal is stepped on.

As shown in fig. 26, when a user (doctor or operator) desires to advance the insertion portion endoscope and sheath assembly in its entirety, the user depresses the pedal, operates the first push rod (e.g., left push rod or right push rod) in a first direction (e.g., pushing forward or pulling backward), controls the device of the bronchoscope robot to receive such input from the user, switches the control mode to the in-entirety endoscope mode, and in response to such input from the user, sends an instruction to the drive mechanism instructing the drive mechanism to drive the insertion portion endoscope and sheath assembly in its entirety.

Accordingly, when the user desires to withdraw the entire insertion portion endoscope and sheath assembly, the user depresses the pedal, operates the first push rod in a second direction (e.g., in a direction opposite to the first direction (pulling back or pushing forward)), after the device controlling the bronchoscope robot receives this input from the user, the control mode switches to the entire withdrawal mode, and in response to this user input, sends an instruction to the drive mechanism instructing the drive mechanism to drive the insertion portion endoscope and sheath assembly to withdraw the entire endoscope.

When the pedal is released, the mirror-in/off operation cannot be performed. There is a certain mapping ratio between the motion of the first push rod and the bronchoscope robot lens advancing/retreating speed, and the ratio may be changed according to a preset rule in the whole operation process so as to adapt to the contact condition between the current bronchoscope and the bronchus.

In some embodiments, the method of controlling a bronchoscope robot further comprises:

Specifically, as shown in fig. 19, after the bronchoscope is inserted into the trachea of a patient to a certain depth, the bronchoscope needs to be bent to be well inserted into the bent trachea, at this time, the insertion portion endoscope is only exposed from the camera assembly, and the insertion portion endoscope and the outer sheath assembly are integrally bent as a whole (equivalent to a bronchoscope).

Since the outer sheath assembly is substantially stiffer than the insertion portion flexible controllable instrument and the insertion portion scope, only the drive mechanism is required to drive the sheath pull wire 212 to control bending of the sheath flexible controllable instrument 21. The sheath-pull guidewire 212 is configured to control the tension/relaxation state of the guidewire by the drive mechanism. When one side of the guide wire of the sheath flexible controllable device 21 is tensioned and all the guide wires of the other side are relaxed, the sheath flexible controllable device 21 will bend towards that side. Bending of the sheath flexible controllable device 21 in any direction can be achieved by applying different tensions or displacements to the different guidewires. The traction guide wires of the insertion portion endoscope are in a loose state, the outer sheath component drives the insertion portion endoscope to bend together, and the outer sheath component provides support for the insertion portion endoscope so as to achieve integral bending of the bronchoscope.

For example, the right hand side of the physician sets the force feedback master hand. The holding part of the master hand can move freely in a certain three-dimensional space, and a fixedly connected monitoring point exists on the holding part, the point is not necessarily a physical key or any visible point, the position and the pose change of the point are stored in the buffer memory in real time at a certain sampling frequency and are sent to the slave end equipment (comprising a driving mechanism and a bronchoscope) at a certain frequency, and the movement of the slave end equipment is driven. A physical key is present on the grip portion for switching the control mode of the force feedback master hand.

When the user desires to steer the insertion portion endoscope and sheath assembly in its entirety, as shown in fig. 27, the user presses a button to operate the master hand in the desired direction, the monitoring point of the master hand will be mapped to the distal end of the sheath flexible controllable instrument, the means for controlling the bronchoscope robot will switch to the entirety steering mode upon receipt of this user input, and in response to this user input, an instruction will be sent to the drive mechanism instructing the drive mechanism to steer the insertion portion endoscope and sheath assembly in its entirety in the direction desired by the user.

There is a certain mapping ratio between the movement of the master hand and the overall steering speed of the bronchoscope, which may be changed according to a preset rule during the whole operation process to adapt to the contact condition between the current bronchoscope and the bronchus.

Specifically, as shown in fig. 20, when the bronchoscope robotic system is climbing to near the target location, for example, to the superior branch of the target location or to a branch where the sheath assembly cannot access, the drive mechanism cannot drive the entire advancing mirror. Because the rigidity of the outer sheath component is much larger than that of the flexible controllable instrument of the insertion part and the endoscope body of the insertion part, the driving mechanism maintains the position and the bending angle of the outer sheath component, and independently pushes out the endoscope of the insertion part, so that the endoscope of the insertion part is driven to enter the advancing mirror, and the endoscope of the insertion part penetrates into a finer branch of the trachea.

As shown in fig. 26, when the user desires to insert a scope, the user depresses the pedal, operates the second push rod in the first direction, the means for controlling the bronchoscope robot receives this input from the user, switches the control mode to the insert scope mode, and in response to this input from the user, sends an instruction to the drive mechanism instructing the drive mechanism to drive the insert scope.

Accordingly, when the user desires to withdraw the insertion portion endoscope, the user depresses the pedal, operates the second push rod in the second direction, the device controlling the bronchoscope robot receives the input of the user, switches the control mode to the insertion portion endoscope withdraw mode, and responds to the input of the user, sends an instruction to the driving mechanism to instruct the driving mechanism to drive the insertion portion endoscope withdraw.

When the pedal is released, the mirror-in/off operation cannot be performed. There is a certain mapping ratio between the motion of the second push rod and the bronchoscope robot lens advancing/retreating speed, and the ratio may be changed according to a preset rule in the whole operation process so as to adapt to the contact condition between the current bronchoscope and the bronchus.

Specifically, as shown in fig. 21, after the insertion portion endoscope is inserted into the trachea of the finer branch of the patient to a certain depth, the insertion portion endoscope needs to be bent to be well inserted into the bent trachea of the finer branch, and at this time, the driving mechanism maintains the position and the bending angle of the outer sheath assembly and bends the extended insertion portion endoscope so that the insertion portion endoscope penetrates into the trachea of the finer branch.

When the user desires the insertion endoscope to be turned, as shown in fig. 27, the user releases the key, operates the master hand in the desired direction, the monitoring point of the master hand will be mapped to the distal end of the insertion flexible controllable instrument, the device controlling the bronchoscope robot receives this input from the user, the control mode switches to the insertion endoscope turning mode, and in response to this user input, sends an instruction to the driving mechanism instructing the driving mechanism to drive the insertion endoscope to turn in the direction desired by the user.

There is a certain mapping ratio between the movement of the master hand and the steering speed of the insertion portion endoscope, which may be changed according to a preset rule during the whole operation process to adapt to the contact condition between the current bronchoscope and the bronchus.

Specifically, when the user desires to withdraw the insertion portion endoscope, the user depresses the pedal, operates the second push rod in the second direction, and after the device controlling the bronchoscope robot receives the input of the user, the control mode is switched to the insertion portion endoscope withdraw mode, and in response to the input of the user, an instruction is sent to the driving mechanism to instruct the driving mechanism to drive the insertion portion endoscope withdraw.

The first threshold may be set according to practical situations, and is not suitable to be too long or too short.

Specifically, as shown in fig. 28, when the surgical operation is completed, the surgical tool retrieval is confirmed and the mirror retracting operation is confirmed, or in other cases, when it is determined that the mirror retracting operation is required, the mirror retracting and steering operation of the endoscope of the insertion portion is performed first, and the position, shape and movement state of the outer sheath assembly are still unchanged. To ensure the field of view of the bronchoscope, the insertion portion endoscope cannot be fully retracted into the outer sheath assembly. And to prevent the insertion portion endoscope from collapsing when the sheath assembly is bent, resulting in limited field of view, the system sets a threshold for the distance of the insertion portion endoscope front end (also referred to as the "distal end") from the sheath assembly front end. And stopping the lens withdrawal of the endoscope at the insertion part when the distance between the lens withdrawal and the endoscope reaches the threshold value, and switching the control mode to the whole lens withdrawal and the whole steering. And when the bronchoscope completely exits the human body, finishing the mirror retracting operation of master-slave control.

For safety reasons, bronchoscope robots are equipped with complete sensors and estimation algorithms, including fiber-based force sensors wrapped around the scope of the insertion portion endoscope and sheath assembly, image-based depth estimation algorithms, and collision detection and contact force estimation algorithms, which will display the measurements or estimates of these sensors and estimation algorithms in real time on a display for providing reference information for the physician to advance/retract the scope, in order to avoid injury to the patient due to excessive speed of advancement or retraction or excessive contact force or direct collision with the lumen wall.

Fig. 29 is a schematic diagram of a push rod mapping ratio control logic provided in the present invention, as shown in fig. 29, according to a depth estimation algorithm, the distance between the forefront end (typically a camera) of a bronchoscope and the front cavity wall can be obtained in real time. When the bronchi are at the bifurcation, collision with the front bifurcation should be avoided. Therefore, the system sets a threshold value of the distance between the front end of the bronchoscope and the wall of the cavity, when the distance is too close, the push rod held by the left hand of the doctor feeds back larger resistance, and the system automatically increases the mapping proportion and reduces the advancing/retreating speed of the bronchoscope in the cavity. Similarly, the bronchoscope will acquire the contact of the body and flexible controllable instrument with the inner wall of the bronchi in real time and display it on the display. When the contact force exceeds the set threshold during lens advance/retreat, the push rod also increases the feedback resistance and increases the mapping proportion, and reduces the lens advance/retreat speed.

FIG. 30 is a schematic view of steering control logic provided by the present invention, as shown in FIG. 30, wherein a physician operates a force feedback master according to image information fed back by a display. The calculation formula of the difference between the flexible controllable instrument end position measured by the current position sensor and the predicted value obtained by the forward kinematics model is as follows:

ε＝Δp-Δp ^m

Δq is the control increment of the flexible controllable instrument, i.e. the displacement or tension change applied to all the traction wires. Δp is the amount of change in position of the distal end of the flexible controllable instrument. In the steering control of the bronchoscope robot, the acquisition system converts the position change of the monitoring point on the force feedback main hand into the reference displacement deltap applied to the tail end of the flexible controllable instrument ^* . The error amount epsilon is defined as the difference between the flexible controllable instrument end position measured by the current position sensor and the predicted value obtained by the forward kinematics model, and comprises modeling errors in the whole steering control and errors caused by other interferences.

The control algorithm adopted is a sliding mode controller, and the optimization goal of the controller is to reduce the expected position delta p ^d And model predicted position Δp ^m Is a function of the error of (a). Therefore, the desired position Δp can be obtained from the control block diagram in fig. 30 ^d The calculation formula of (2) is as follows:

Δp ^d ＝Δp ^* -ε＝Δp ^* -(Δp-Δp ^m )

namely:

Δp ^d -Δp ^m ＝Δp ^* -Δp

thus, the flexible controllable instrument tip displacement Δp versus the reference tip position Δp ^* Is equivalent to the tracking of the desired position deltap ^d Predicting the position deltap for the model ^m Can effectively eliminate possible uncertain disturbance and model error, and improve the flexible controllable instrument in the endoscope and sheath assembly of the insertion part of the bronchoscope robotSteering control accuracy of (a).

For safety reasons, to avoid excessive extrusion or impact on the bronchial lumen wall during steering of the flexible steerable instrument of the insertion endoscope or sheath assembly, and thus injury to the patient. The doctor will automatically adjust the mapping ratio of the master hand when manipulating the force feedback master hand.

FIG. 31 is a schematic diagram of the master hand mapping ratio control logic provided by the present invention, as shown in FIG. 31, wherein when a physician issues a steering control command for a flexible controllable instrument via a force feedback master hand, the system will evaluate the current contact force conditions, including contact force distribution and magnitude. If the contact force is too large, the feedback force and vibration effect of the force feedback main hand are increased, the mapping proportion of the main hand is increased, and the steering speed and amplitude are reduced.

The following describes the above method further by the procedure of bronchoscope robot insertion into the target site:

Fig. 32 is a schematic view of a bronchial surgical target point and navigation path provided by the present invention, as shown in fig. 32, the target point location 321, three-dimensionally reconstructed bronchial model 322 and real-time path planning 323 obtained by preoperative CT scan will be displayed on a display, providing navigation information for a doctor.

Fig. 33 is a schematic diagram of a process of inserting the bronchoscope robot into a target point, as shown in fig. 33, after the power-on self-test is passed, the system will be initialized, and generally, the initial working mode of the bronchoscope will be in the integral endoscope-entering mode.

Considering that the outer diameter of the outer sheath assembly is larger than the insertion endoscope, the outer sheath assembly is generally unable to reach deep branches of the bronchi, if the planned target point is within a branch that the outer sheath assembly cannot reach, such as a bronchiole of approximately 3mm diameter, the preoperative planning will additionally plan the outer sheath target point 324. In the preoperative stage, the main bronchus has larger diameter, the outer sheath assembly can move in the cavity, and a doctor operates the left push rod and the force feedback main hand at the moment to perform integral endoscope feeding and integral steering control on the bronchoscope along the navigation path. The doctor needs to judge whether the target point 321 is reached or not through the navigation information on the display and the returned image of the camera component in real time. If the target is not reached, a further determination is made as to whether the sheath target 324 is reached. If the sheath target point is reached, the bronchoscope robot manual mode is switched to an insertion part endoscope entering mode and an insertion part steering mode, and the shape and the movement state of the sheath assembly are kept at the moment, and the insertion part endoscope is independently driven. Based on the real-time navigation information and the camera image, it is determined whether the insertion portion endoscope reaches the target point 321. And after the target point is reached, finishing the lens feeding operation of the bronchoscope, and inserting the tool for operation until the operation is finished.

The method for controlling the bronchoscope robot provided by the embodiment of the invention combines the configuration of the bronchoscope robot system and the achievable bending combination, and the six operation mode switching methods and the specific implementation examples in the embodiment of the invention provide a foundation for controlling the bronchoscope to safely, efficiently and noninvasively reach deep bronchi.

The various components of the bronchoscope robot are ergonomic. The doctor only needs to sit on the chair, and the left hand and the right hand respectively control the push rod and the main hand, and the pedal is controlled by the foot, and eyes look at the information of the display to control the bronchoscope robot. The three functions of eyes, hands and feet can effectively relieve the fatigue of doctors in long-term operation. And the operation table is far away from the surgical instrument, so that risks of cross infection, radiation risk, body fluid splashing and the like are reduced.

The control of the bronchoscopic robot is safe, and the bronchoscopic robot system is provided with various sensors and estimation algorithms, including force sensors, depth estimation algorithms, contact force estimation, collision detection and the like. In addition to the real-time display of these information on the display, in addition to providing auxiliary information for the doctor, the information will also be used to automatically adjust the mapping relationship between the push rod and the force feedback master hand, assisting the doctor to complete minimally invasive surgery more safely.

The control precision of the bronchoscope robot is high, and the bronchoscope robot system is provided with a position sensor, so that real-time accurate position information feedback can be provided for the steering of the flexible controllable instrument. Based on the designed internal model control frame, errors caused by model and uncertain interference are reduced, the sliding mode controller is suitable for a nonlinear system, effective and stable flexible controllable instrument control quantity can be obtained, and steering control precision of the bronchoscope robot system is improved.

Fig. 34 is a schematic view of an apparatus for controlling a bronchoscope robot according to the present invention, and as shown in fig. 34, an embodiment of the present invention provides an apparatus for controlling a bronchoscope robot, including a receiving module 3401 and a response module 3402, where:

In some embodiments, the receiving module is further for receiving a second input from the user; the second input is an operation that the user desires the insertion portion endoscope and the outer sheath assembly to be bent integrally;

the response module is further configured to send a second instruction to the drive mechanism in response to the second input, the second instruction being configured to instruct the drive mechanism to drive the insertion portion endoscope and the outer sheath assembly to bend as a whole.

In some embodiments, the receiving module is further configured to receive a third input from the user; the third input is an operation that the user desires the insertion portion endoscope to enter;

the response module is further configured to send a third instruction to the drive mechanism in response to the third input, the third instruction being configured to instruct the drive mechanism to drive the insertion portion endoscope.

In some embodiments, the receiving module is further for receiving a fourth input by the user; the fourth input is an operation that the user desires the insertion portion endoscope to bend;

the response module is further configured to send a fourth instruction to the drive mechanism in response to the fourth input, the fourth instruction being configured to instruct the drive mechanism to drive the insertion portion endoscope to bend.

In some embodiments, the receiving module is further configured to receive a fifth input from the user if the distal end of the insertion portion endoscope extends beyond the distal end of the outer sheath assembly by a length greater than a first threshold; the fifth input is an operation that the user desires the insertion portion endoscope to be retracted;

the response module is further configured to send a fifth instruction to the driving mechanism in response to the fifth input, the fifth instruction being configured to instruct the driving mechanism to drive the insertion portion endoscope to retract.

In some embodiments, the response module is further configured to send a sixth instruction to the drive mechanism to instruct the drive mechanism to stop driving the insertion portion endoscope to retract upon determining that the length of the distal end of the insertion portion endoscope retracted to the distal end of the outer sheath assembly is equal to a first threshold.

Specifically, the device for controlling a bronchoscope robot provided by the embodiment of the present invention can implement all the method steps implemented by the method embodiment for controlling a bronchoscope robot, and can achieve the same technical effects, and the same parts and beneficial effects as those of the method embodiment in the embodiment are not described in detail herein.

Fig. 35 is a schematic structural diagram of an electronic device according to the present invention, and as shown in fig. 35, the electronic device may include: a processor (processor) 3510, a communication interface (Communications Interface) 3520, a memory (memory) 3530, and a communication bus 3540, wherein the processor 3510, the communication interface 3520, and the memory 3530 communicate with each other via the communication bus 3540. The processor 3510 can invoke logic instructions in the memory 3530 to perform a method of controlling a bronchoscope robot, the method comprising:

Further, the logic instructions in the memory 3530 described above can be implemented in the form of software functional units and can be stored in a computer readable storage medium when sold or used as a stand alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

In another aspect, the present invention also provides a computer program product comprising a computer program storable on a non-transitory computer readable storage medium, the computer program, when executed by a processor, being capable of performing the method of controlling a bronchoscope robot provided by the methods described above, the method comprising:

In yet another aspect, the present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, is implemented to perform the method of controlling a bronchoscope robot provided by the above methods, the method comprising:

The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.

From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.

Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method of controlling a bronchoscopic robot, comprising:

wherein the bronchoscope robot comprises a bronchoscope and the driving mechanism; the bronchoscope includes the insertion portion endoscope and the outer sheath assembly; force sensors are respectively wound on the insertion part endoscope and the outer sheath component; the force sensor is used for detecting the contact force between the bronchoscope and the inner wall of the bronchus;

The method further comprises the steps of:

acquiring the contact force between the bronchoendoscope and the inner wall of the bronchus, which are acquired by the force sensor;

based on the contact force between the bronchoscope and the inner wall of the bronchus, the control resistance on the force feedback main hand and the push rod is adjusted.

2. The method of controlling a bronchoscopic robot according to claim 1, wherein the method further comprises:

receiving a second input from the user; the second input is that the user controls the operation of integrally bending the insertion part endoscope and the outer sheath assembly through a force feedback main hand;

in response to the second input, obtaining tip displacement of the flexible controllable instrument in the insertion portion endoscope and the outer sheath assembly through a position sensor, determining a reference tip position of the flexible controllable instrument in the insertion portion endoscope and the outer sheath assembly based on a position change of a monitoring point on the force feedback main hand, obtaining a predicted position of the flexible controllable instrument in the insertion portion endoscope and the outer sheath assembly using a forward kinematic model based on the position and pose change of the monitoring point on the force feedback main hand, determining a desired position of the flexible controllable instrument in the insertion portion endoscope and the outer sheath assembly based on the tip displacement, the reference tip position and the predicted position, and sending a second instruction to the driving mechanism based on the desired position, the second instruction being for instructing the driving mechanism to drive the insertion portion endoscope and the outer sheath assembly to bend integrally to the desired position.

3. The method of controlling a bronchoscope robot according to claim 1 or 2, further comprising:

4. A method of controlling a bronchoendoscope robot according to claim 3 and also comprising:

5. The method of controlling a bronchoscopic robot according to claim 1, wherein the method further comprises:

Transmitting a fifth instruction to the driving mechanism in response to the fifth input, the fifth instruction being for instructing the driving mechanism to drive the insertion portion endoscope to retract;

the method further comprises the steps of:

acquiring the distance between the front end of the bronchoscope and the wall of the broncholumen;

and adjusting the control resistance on the push rod based on the distance, and automatically adjusting the mapping proportion between the movement of the push rod and the endoscope advancing or retracting speed of the bronchoscope, so as to adjust the endoscope advancing or retracting speed of the bronchoscope in the cavity.

6. The method of controlling a bronchoscopic robot according to claim 5, wherein the method further comprises:

7. An apparatus for controlling a bronchoscope robot, comprising:

wherein the bronchoscope robot comprises a bronchoscope, the driving mechanism and a controller; the bronchoscope includes the insertion portion endoscope and the outer sheath assembly; force sensors are respectively wound on the insertion part endoscope and the outer sheath component; the force sensor is used for detecting the contact force between the bronchoscope and the inner wall of the bronchus;

the apparatus further comprises:

the acquisition module is used for acquiring the contact force between the bronchoendoscope acquired by the force sensor and the inner wall of the bronchus;

and the adjusting module is used for adjusting the control resistance on the force feedback main hand and the push rod based on the contact force between the bronchoscope and the inner wall of the bronchus.

8. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of controlling a bronchoscopic robot according to any one of claims 1 to 6 when the program is executed.

9. A non-transitory computer readable storage medium, having stored thereon a computer program, which when executed by a processor, implements a method of controlling a bronchoscopic robot according to any one of claims 1 to 6.

10. A computer program product comprising a computer program, which, when executed by a processor, implements a method of controlling a bronchoscope robot according to any one of claims 1 to 6.