WO2020220890A1 - Dispositif d'affichage et robot chirurgical - Google Patents

Dispositif d'affichage et robot chirurgical Download PDF

Info

Publication number
WO2020220890A1
WO2020220890A1 PCT/CN2020/081811 CN2020081811W WO2020220890A1 WO 2020220890 A1 WO2020220890 A1 WO 2020220890A1 CN 2020081811 W CN2020081811 W CN 2020081811W WO 2020220890 A1 WO2020220890 A1 WO 2020220890A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
component
display
display device
virtual
Prior art date
Application number
PCT/CN2020/081811
Other languages
English (en)
Chinese (zh)
Inventor
孙颖浩
陈功
毛昊阳
袁帅
罗中宝
何超
Original Assignee
微创（上海）医疗机器人有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 微创（上海）医疗机器人有限公司 filed Critical 微创（上海）医疗机器人有限公司
Publication of WO2020220890A1 publication Critical patent/WO2020220890A1/fr

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/25—User interfaces for surgical systems
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/30—Surgical robots
    • A61B34/37—Master-slave robots
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/70—Manipulators specially adapted for use in surgery
    • A61B34/76—Manipulators having means for providing feel, e.g. force or tactile feedback
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • A61B2034/2046—Tracking techniques
    • A61B2034/2065—Tracking using image or pattern recognition
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/30—Surgical robots
    • A61B2034/301—Surgical robots for introducing or steering flexible instruments inserted into the body, e.g. catheters or endoscopes

Definitions

  • the invention relates to the field of surgical instruments, in particular to a display device and a surgical robot.
  • Minimally invasive surgery is accepted by more and more patients due to its advantages of small trauma and quick recovery. More and more surgical modes have slowly developed from traditional open surgery to minimally invasive surgery, and the corresponding surgical tools have also slowly developed from the early surgical forceps and scalpels to the current surgical robotic arms, Surgical robots, etc.
  • the display device can provide doctors with intuitive images, which is a very important part.
  • the display device of the existing surgical robot system can provide the doctor with high-fidelity three-dimensional (3D) vision, and can provide the doctor with an accurate spatial distance.
  • the existing naked-eye 3D image is realized by providing images with different parallaxes for the left and right eyes respectively. Therefore, the doctor's console of the current surgical robot is mostly immersive.
  • the doctor can observe three-dimensional images through the observation window on the doctor's console, and the observation position is restricted to an extremely limited position.
  • the complicated surgical environment and operating tasks require doctors to manipulate robots for a long time to perform surgical operations. Observing three-dimensional images for a long time will make the doctors visually fatigued.
  • the three-dimensional image can observe the depth of the position, when the endoscope is closer to the surgical instrument or tissue, the two-dimensional planar image is easier to observe.
  • the purpose of the present invention is to provide a display device and a surgical robot to solve the problem that the existing display device cannot switch between two-dimensional and three-dimensional images.
  • a display device including:
  • the display includes two display areas, a first display area and a second display area, the first display area and the second display area are respectively used to receive image information, and respectively display the first image and the second image;
  • the switching component includes a first state and a second state, and the switching component is configured to:
  • Two light path components respectively a first light path component and a second light path component
  • the first light path component is used to receive a first image and form a first virtual image for the observer's perception of an eye
  • the second light path component For receiving the second image and forming a second virtual image for perception by another eye of the observer, and the first virtual image and the second virtual image are arranged in the same direction.
  • the display device further includes an observation surface and a symmetry surface
  • the observation plane is arranged perpendicular to the symmetry plane, and the observation plane is used for the observer to perceive the first virtual image and the second virtual image;
  • the first display area and the second display area are symmetrically arranged with respect to the symmetry plane;
  • the first optical path component and the second optical path component are symmetrically arranged about the symmetry plane;
  • the first optical path component and the second optical path component are configured to make the first virtual image and the second virtual image parallel to the observation surface.
  • the first optical path component and the second optical path component are configured to at least partially overlap the first virtual image and the second virtual image in the lateral direction.
  • the first optical path component and the second optical path component are configured such that the image planes of the first virtual image and the second virtual image are both located on the same virtual plane.
  • the first virtual image and the second virtual image are both arranged in a vertical direction and located on the same horizontal height.
  • each of the light path components is a reflective component
  • the reflective component is used to reflect the image displayed by the display to form the first virtual image or the second virtual image.
  • each of the reflective components includes:
  • the first reflecting mirror is arranged at an angle to the corresponding one of the display areas.
  • the second reflecting mirror is arranged at an angle to the first reflecting mirror, one extending direction of the second reflecting mirror is the same as one extending direction of the observation surface, and the other extending direction of the second reflecting mirror is the same as The other extension direction of the observation surface is arranged at an angle;
  • the reflective component is configured such that after the first image or the second image is displayed by the corresponding display area, it passes through the reflection of the corresponding first mirror and the second mirror in sequence , To form the first virtual image or the second virtual image.
  • one extension direction of the second reflector and one extension direction of the observation surface both extend in a vertical direction.
  • the display device further includes an offset adjustment device for adjusting the position or angle of the optical path component to adjust the lateral direction between the first virtual image and the second virtual image. distance.
  • the display device further includes an image segmentation component, the image segmentation component is communicatively connected with the switching component, and the image segmentation component is configured to receive two-dimensional image information and perform processing to obtain a second The first disparity image information with the horizontal disparity in the three-dimensional image information;
  • the switching component When the switching component is in the first state, the switching component receives the two-dimensional image information and the first parallax image information output by the image segmentation component, and provides them to the first display area and the second display area respectively.
  • a display area so that the first image and the second image displayed in the two display areas have lateral parallax;
  • the switching component When the switching component is in the second state, the switching component receives the two-dimensional image information, and provides the same two-dimensional image information to the two display areas, so that the second display area is displayed in the two display areas.
  • One image is the same as the second image.
  • the display device further includes an image merging component, the image merging component is respectively communicatively connected with the switching component, and the image merging component is configured to receive and process the second parallax image information to obtain the combined image information ,
  • the second parallax image information includes information of two two-dimensional images with lateral parallax;
  • the switching component When the switching component is configured in the first state, the switching component receives the second parallax image information, and provides information of two two-dimensional images in the second parallax image information to the The first display area and the second display area, so that the first image and the second image displayed in the two display areas have lateral parallax;
  • the switching component When the switching component is configured in the second state, the switching component receives the merged image information output by the image merging component and provides it to the two display areas, so that the two display areas The first image and the second image displayed are the same.
  • the switching component when the switching component is configured in the second state, the switching component receives second parallax image information, wherein the second parallax image information includes two two-dimensional Image information, the switching component provides information of a two-dimensional image in the second parallax image information to the two display areas, so that the first image and the second image displayed in the two display areas the same.
  • a surgical robot includes the display device as described above, and also includes an image acquisition device, which is communicatively connected with the display device, To obtain image information of the surgical environment and provide it to the display device.
  • the surgical robot further includes:
  • the side cart includes at least one image arm and a tool arm, the image acquisition device is mounted on the image arm, and the surgical instrument is mounted on the tool arm;
  • the doctor's console includes a main operator, and the display device is arranged on the doctor's console;
  • the master operating hand, the tool arm and the surgical instrument constitute a master-slave control relationship.
  • the display device includes a display, a switching component, and two optical path components.
  • the two images displayed in the two display areas of the display are respectively formed by the two optical path components.
  • Two virtual images respectively perceived by the observer’s two eyes are switched between the first state and the second state through the switching component.
  • the switching component is in the first state, the two display areas are made to receive the horizontal parallax respectively.
  • the virtual image formed by the optical path component has lateral parallax; when the switching component is in the second state, the two display areas are made to receive the same image information, and correspondingly, the virtual image formed by the optical path component is the same ; So that the display mode of the display device can be switched between the three-dimensional mode and the two-dimensional mode, the doctor can easily select the appropriate image display mode according to the needs, which not only helps the doctor relieve the eye fatigue caused by long-term observation of three-dimensional images, but also It is good for observing objects in different situations, especially when the object is closer to the observation.
  • FIG. 1 is a schematic diagram of a surgical robot provided by Embodiment 1 of the present invention.
  • FIG. 2 is a schematic diagram of a doctor console provided by Embodiment 1 of the present invention.
  • FIG. 3 is a three-dimensional schematic diagram of a display device provided by Embodiment 1 of the present invention.
  • FIG. 4 is a front view of the display device shown in FIG. 3;
  • FIG. 5 is a light path diagram of the display device shown in FIG. 3;
  • FIG. 6 is a schematic diagram of an imaging plane provided by Embodiment 1 of the present invention.
  • FIG. 7 is a flowchart of image segmentation provided by Embodiment 1 of the present invention.
  • Fig. 8 is a flowchart of image merging provided by the second embodiment of the present invention.
  • 211-display 211L-first display area; 211R-second display area;
  • 210-optical path component 210L-first optical path component; 210R-second optical path component;
  • the singular forms “a”, “an” and “the” include plural referents unless the content clearly dictates otherwise.
  • the term “or” is generally used to include the meaning of “and/or”, unless the content clearly indicates otherwise, the term “proximal” usually refers to The end of the observer/observer, the term “distal” is usually the end close to the patient's lesion.
  • the present invention provides a display device, which includes a display, a switching component, and two light path components.
  • the display includes two display areas, a first display area and a second display area, respectively.
  • the two display areas are respectively used to receive image information and respectively display the first image and the second image;
  • the switching component includes a first state and a second state, and the switching component is configured to: when the switching component is in the first state In the state, the two display areas are made to respectively receive two different image information with lateral parallax; when the switching component is in the second state, the two display areas are made to receive the same image information;
  • the light path components are respectively a first light path component and a second light path component.
  • the first light path component is used for receiving a first image and forming a first virtual image for the observer's perception of an eye
  • the second light path component is used for The second image is received and a second virtual image for perception by another eye of the observer is formed, and the first virtual image and the second virtual image are arranged in the same direction.
  • the virtual images formed by the optical path component have lateral parallax, so that the observer can perceive Two virtual images of lateral parallax, and a three-dimensional image is formed in the brain; when the switching component is in the second state, the two display areas receive the same image information, and correspondingly, the virtual images formed by the optical path components are the same to make the observer perceive the same Two virtual images of, and form a two-dimensional image on the brain; thus the display mode of the display device can be switched between three-dimensional mode and two-dimensional mode, and observers (such as doctors or assistants) can easily select the appropriate image according to their needs
  • the display mode not only helps the observer alleviate eye fatigue caused by observing three-dimensional images for a long time, but also helps to observe the object in different situations, especially when it is close to the object to be observed.
  • Figure 1 is a schematic diagram of a surgical robot provided in Embodiment 1 of the present invention
  • Figure 2 is a schematic diagram of a doctor console provided in Embodiment 1 of the present invention
  • Figure 3 is a schematic diagram of a doctor console provided in Embodiment 1 of the present invention.
  • Fig. 4 is a front view of the display device shown in Fig. 3
  • Fig. 5 is a light path diagram of the display device shown in Fig. 3
  • Fig. 6 is a schematic diagram of an imaging plane provided by Embodiment 1 of the present invention
  • FIG. 7 is a flowchart of image segmentation provided by Embodiment 1 of the present invention.
  • the first embodiment of the present invention provides a surgical robot, which includes: an image cart 10, a side cart 11, an operating trolley 12, a tool cart 13, and a doctor console 20.
  • the doctor can remotely use the doctor console 20. Operation to achieve minimally invasive surgical treatment of patients on the operating trolley 12.
  • the image trolley 10 can provide the doctor with auxiliary image data, etc.
  • the tool trolley 13 can provide the doctor with the tools needed in the operation.
  • the side trolley 11 includes at least one image arm 110 and a tool arm 112.
  • An image acquisition device is mounted on the image arm 110, and the image acquisition device is communicatively connected with the display device, and is used to acquire image information of the surgical environment and provide it to the display device.
  • the image acquisition device is used to acquire and provide image information of the surgical environment including human tissues and organs, surgical instruments, blood vessels, and body fluids to the display device.
  • a surgical instrument 113 is mounted on the tool arm 112.
  • the image acquisition device may be an endoscope 111 or a laparoscope, for example.
  • the endoscope 111 and the surgical instrument 113 respectively enter the patient's position through the wound on the patient's body to achieve minimally invasive surgical treatment.
  • the doctor console 20 includes a main operating hand 22.
  • the master operator 22, the tool arm 112 of the side cart 11 and the surgical instrument 113 constitute a master-slave control relationship.
  • the tool arm 112 moves according to the movement of the main operating hand 22 during the operation, that is, moves according to the operation of the doctor's hand. Further, the main operating hand 22 also receives the force information of the human tissues and organs on the surgical instrument 113 and feeds it back to the doctor's hand, so that the doctor can more intuitively feel the surgical operation.
  • the display device 21 includes a switching component (not shown), a display 211 and two optical path components 210.
  • the display device 21 further includes a housing 24 and an observation window 23.
  • the display 211 and the two light path components 210 are arranged inside the housing 24, and the observation window 23 is arranged on the housing 24 for the observer to observe images of the surgical environment.
  • the display device 21 is arranged on the doctor console 20.
  • the doctor console 20 further includes a control component (not shown) for connecting with the main operating hand 22.
  • the original image information acquired by the endoscope 111 or the processed image information obtained by processing the original image information can be transmitted to the control component, and the control component is used to process the original image information or the original image information through processing.
  • the display device 21 further includes an observation plane and a symmetry plane, and the observation plane is arranged perpendicular to the symmetry plane.
  • the “symmetry plane” can be a physical object or a virtual object, and the observation plane The surface can coincide with the plane where the observation window 23 is located for observation by the observer.
  • the display 211 includes two display areas, a first display area 211L and a second display area 211R, respectively.
  • the first display area 211L and the second display area 211R are arranged symmetrically about the symmetry plane.
  • the two display areas are respectively used for receiving image information and respectively displaying the first image and the second image.
  • the first display area 211L is not limited to be located on the left side of the second display area 211R as shown in FIG. 3, but it means that the first display area 211L and the second display area 211R are two oppositely arranged display areas.
  • the positions of the one display area 211L and the second display area 211R are interchangeable.
  • the two display areas may be composed of two different displays respectively, or may be composed of different display areas in the same display, which is not limited in the present invention.
  • the two optical path components 210 are respectively a first optical path component 210L and a second optical path component 210R.
  • the first optical path component 210L and the second optical path component 210R are arranged symmetrically about the plane of symmetry, and the first optical path component 210L is used for Receive the first image and form a first virtual image for one eye of the observer, and the second optical path component 210R is used to receive the second image and form the other eye for the observer
  • the second virtual image is perceived, and the first virtual image and the second virtual image are arranged in the same direction.
  • the three-dimensional image can be formed in the brain when the observer's left and right eyes respectively observe a virtual image with the same arrangement direction and lateral parallax.
  • the arrangement of the first virtual image and the second virtual image in the same direction means that the postures of the same object described in the first virtual image and the second virtual image in each virtual image are the same or similar.
  • the first virtual image is an image with an arrow pointing up
  • the second virtual image must also be an image with an arrow pointing up.
  • the first optical path component 210L and the second optical path component 210R are configured to make the image planes of the first virtual image and the second virtual image parallel to the observation plane.
  • the first light path component 210L and the second light path component 210R are configured to make the first virtual image and the second virtual image in The horizontal direction (ie, the lateral direction) overlaps at least partially.
  • the "overlap” here can mean overlapping on the first virtual image and the second virtual image on the same virtual plane, or it can be understood that the projection planes of the first virtual image and the second virtual image respectively projected onto a parallel plane overlap each other.
  • the first optical path component 210L and the second optical path component 210R are configured such that the image planes of the first virtual image and the second virtual image are both located on the same virtual plane 215.
  • the first optical path component 210L is located on the left side of the second optical path component 210R, but refers to the first optical path component 210L and the second optical path component 210R as two opposing optical path components.
  • the first light path component 210L corresponds to the first display area 211L and is used to form a first virtual image
  • the second light path component 210R corresponds to the second display area 211R and is used to form a second virtual image.
  • the image plane position of the first virtual image can be changed by changing the structure of the first optical path component 210R.
  • the image plane position of the second virtual image can be changed by changing the structure of the second optical path component 210R.
  • the switching component includes a first state and a second state, and the switching component is configured to: when the switching component is in the first state, the two display areas respectively receive two different image information with lateral parallax ; When the switching component is in the second state, the two display areas receive the same image information. Specifically, please refer to FIG. 6 in conjunction with FIG. 2.
  • the switching component is configured in the first state, the display mode of the display device 21 is a three-dimensional mode, and the two display areas respectively receive different values with lateral parallax. The two image information, and form the first image and the second image.
  • the first image forms a first virtual image 215L
  • the second image forms a second virtual image 215R
  • the first virtual image 215L and the second virtual image 215R have lateral parallax.
  • the observer’s eyes are located on the observation surface.
  • the observation window 23 human eye observation point LR
  • the left eye is at the human eye observation point L
  • the first virtual image 215L is observed by the left eye
  • the right eye is at the human eye observation point R.
  • the second virtual image 215R is observed by the right eye
  • the first virtual image 215L and the second virtual image 215R with lateral parallax enter the left and right eyes of the observer respectively.
  • the synthesized three-dimensional image can be observed.
  • the display mode of the display device 21 is a two-dimensional mode
  • the first image and the second image are the same image
  • the first The first virtual image 215L formed by the image and the second virtual image 215R formed by the second image are also the same image.
  • the left and right eyes of the observer see the same image, and thus the two-dimensional image is observed.
  • the switching component is used to switch between the first state and the second state.
  • the two display areas are made to receive two different images with lateral parallax.
  • the two display areas receive the same image information; so that the display mode of the display device 21 can be switched between the three-dimensional mode and the two-dimensional mode, and the doctor can easily select the appropriate one according to needs
  • the image display mode not only helps doctors alleviate eye fatigue caused by long-term observation of three-dimensional images, but also helps to observe objects in different situations, especially when the endoscope 111 is close to the surgical instrument 113 or human tissue.
  • the two-dimensional plane image is easier to be recognized and observed by the observer, so it is especially suitable for the situation where the object to be observed is closer.
  • the display device 21 further includes a switch, the switch is in communication connection with the switch component, and the observer inputs an instruction to make the switch component between the first state and the second state by operating the switch Switching, in turn, controls the display device 21 to switch between the three-dimensional mode and the two-dimensional mode, so that the doctor can switch when visual fatigue or select a more appropriate image display mode according to factors such as the position of the displayed object, and improve the safety of the operation.
  • the present invention has no particular restrictions on the specific shape and position of the switch, which can be a virtual button on the display device 21, can be a physical component on the doctor's console, such as the main operating hand 22, or Physical parts located near the feet of the observer, etc.
  • the first virtual image and the second virtual image are arranged along the vertical direction, and are located on the same horizontal height, that is, the first virtual image and the second virtual image are respectively vertical to the horizontal plane, and both are along the horizontal Direction arrangement.
  • the observer is generally sitting or standing, and the binocular connection is generally horizontal. Therefore, the first virtual image 215L and the second virtual image 215R are arranged along the vertical direction and are located at the same horizontal height, which is beneficial for the observer to observe.
  • the optical path component 210 is preferably a reflective component, and the reflective component is used to reflect the image displayed by the display 211.
  • the following takes the first optical path assembly 210L as an example for specific introduction.
  • the reflecting assembly includes a first left reflecting mirror 212L and a second left reflecting mirror 213L.
  • the first left mirror 212L is arranged at an angle to the corresponding first display area 211L;
  • the second left mirror 213L is arranged at an angle to the first left mirror 212L, and one of the second left mirrors 213L extends
  • the direction is the same as one extension direction of the observation surface, and the other extension direction of the second mirror 213L is arranged at an angle to the other extension direction of the observation surface.
  • a plane can be considered to have two extension directions (that is, the directions of two adjacent sides, such as the X and Y directions perpendicular to each other), and the mirror and the observation surface can be abstracted into a plane, so both It has two extension directions, namely the directions of two adjacent sides of the mirror and the observation surface respectively.
  • an extension direction of the second left mirror 213L is the same as an extension direction of the observation surface, which can ensure that the arrangement direction of the first virtual image 215L observed from the observation surface will not be changed by the second left mirror 213L .
  • one extension direction of the second left mirror 213L and one extension direction of the observation surface both extend in the vertical direction, so that the arrangement direction of the first virtual image 215L can also be the vertical direction.
  • each optical path assembly preferably includes two reflective devices. If the number of reflective devices in the optical path assembly is too large, the display device 21 will be bulky and increase the difficulty of installation; and if the number of reflective devices is too small (such as Only one piece) will cause the size of the reflective device to be too large, which in turn will also cause the display device 21 to be bulky.
  • the first display area 211L and the second display area 211R are both separate displays, and the two light path components and the two display areas are all arranged symmetrically about the symmetry plane.
  • the first display area 211L is located above the first left mirror 212L and the second left mirror 213L, and the second left mirror 213L is located between the symmetry plane and the first left mirror 212L.
  • the first left mirror 212L is used to reflect the first image displayed in the first display area 211L to form a left virtual image 214L; the second left mirror 213L is used to perform a first reflection on the left virtual image 214L.
  • the second reflection forms a first virtual image 215L.
  • the left virtual image 214L and the first image displayed in the first display area 211L are symmetric about the first left mirror 212L, and the left virtual image 214L and the first virtual image 215L are symmetric about the second left mirror 213L. Further, by adjusting the arrangement angles of the first display area 211L, the first left mirror 212L, and the second left mirror 213L, the first image in the first display area 211L passes through the first left mirror 212L and the second left mirror 213L. The reflection forms a first virtual image 215L, which is parallel to the observation surface.
  • the angle of the second left mirror 213L may be determined according to the angle of the first virtual image 215L and the angle of the left virtual image 214L.
  • the image plane of the left virtual image 214L is parallel to the symmetry plane, and the angle of the first left mirror 212L can be determined according to the angle of the first display area 211L and the left virtual image 214L.
  • the two extension directions of the first left mirror 212L and the two extension directions of the second left mirror 213L are respectively arranged at an angle.
  • the first left mirror 212L is arranged perpendicular to the image plane of the first virtual image 215L, which is beneficial to reduce the volume of the entire optical path assembly.
  • the second optical path component 210R has the same configuration as the first optical path component 210L and is symmetrical to the first optical path component 210L about the symmetry plane.
  • the second optical path component 210R includes a first right mirror 212R and a second right mirror 213R,
  • the first right mirror 212R is used to reflect the second image displayed in the second display area 211R to form a right virtual image 214R
  • the second right mirror 213R is used to reflect the right virtual image 214R a second time to form a second virtual image 215R.
  • the specific structure and principle of the second optical path assembly 210R are the same as those of the first optical path assembly 210L, and will not be repeated here.
  • the display device 21 further includes an offset adjustment device (not shown), the offset adjustment device is used to adjust the optical path assembly 210 to adjust the gap between the first virtual image 215L and the second virtual image 215R To adjust the horizontal overlap range of the two virtual images.
  • the offset adjustment device can control the arrangement angle of each reflector in the optical path assembly 210 in linkage, so that different observers can adjust the displacement of the first virtual image 215L and the second virtual image 215R as needed to adjust the first virtual image 215L and the second virtual image 215L.
  • the horizontal distance between the two virtual images 215R is the horizontal overlap range, which is more in line with the different pupil distances of the observer and ensures the clarity of the observed image.
  • the endoscope 111 is a two-dimensional endoscope, and the two-dimensional endoscope is used to obtain a two-dimensional original image and output first original image information.
  • the display device 21 further includes an image segmentation component, and the image segmentation component is in communication connection with the switching component.
  • the image segmentation component is used to receive and process two-dimensional image information to obtain first parallax image information having a lateral parallax with respect to the received two-dimensional image information.
  • the two-dimensional image information received by the image segmentation component may be the first original image information output by the two-dimensional endoscope, or the first original image information processed by other processing components using existing technology, such as Image denoising, image contrast adjustment, image distortion correction, image size cropping, image grayscale processing, etc. In this regard, this embodiment does not impose restrictions.
  • the switching component is in the first state 810 according to the external instruction 801 obtained by the input device (for example, the above-mentioned switching switch), and notifies the processing function of the image segmentation component to start 811.
  • the image segmentation component obtains the first disparity image information 813 according to the received two-dimensional image information 803, and the switching component 814 receives the two-dimensional image information 812 (this two-dimensional image information 812 may be the same as the two-dimensional image information received by the image segmentation component).
  • the switching component is in the second state 820 according to the external instruction 801 and notifies the processing function of the image segmentation component to close 821.
  • the switching component 824 receives the two-dimensional image information 822, and provides the two display areas (ie, the first display area 825 and the second display area 826) of the display with the same two-dimensional image information 822.
  • the two-dimensional image information 822 may be the same as the two-dimensional image information 803 received by the switching component), so that the first image and the second image displayed in the two display areas are the same, even if the display mode of the display device 21 is Two-dimensional model 827.
  • the image segmentation component may perform image segmentation on the two-dimensional original image corresponding to the first original image information, thereby generating a depth map, and by adjusting the displacement of the segmentation block, the original image relative to the two-dimensional original image can be obtained.
  • the image has a two-dimensional parallax image with lateral parallax, and outputs first parallax image information.
  • the image segmentation component receives the first original image information output by the two-dimensional endoscope, performs image segmentation on the two-dimensional original image, and then generates a depth map. By adjusting the displacement of the segmentation block, the The two-dimensional original image has a two-dimensional parallax image with lateral parallax, and outputs first parallax image information.
  • the switching component transmits the first original image information and the first parallax image information to the two display areas of the display, so that the two display areas respectively receive the first original image information and the first parallax image information, and display the first original image information and the first parallax image information.
  • the first original image information is transmitted to the display area corresponding to the right eye, such as the second display area 211R. Since the first image and the second image displayed on the display 211 have lateral parallax, the first virtual image and the second virtual image formed through the optical path component also have lateral parallax. In addition, the first virtual image and the second virtual image are respectively observed by the left eye and the right eye of the observer. Therefore, a corresponding three-dimensional image is formed in the brain of the observer.
  • the display mode of the display device 21 is a two-dimensional mode.
  • the switching component controls the processing function of the image segmentation component to turn off, and controls the first original image information output by the two-dimensional endoscope to be transmitted to two display areas, the first image displayed in the two display areas It is the same image as the second image, and the first virtual image and the second virtual image formed by the optical path component are also the same image. Therefore, what the observer observes is a two-dimensional image.
  • the display device and the surgical robot of the second embodiment of the present invention are basically the same as those of the first embodiment, and the same parts will not be described, and only the different points will be described below.
  • FIG. 8 is a flowchart of image merging provided in the second embodiment of the present invention.
  • the endoscope 111 is a three-dimensional endoscope, and the three-dimensional endoscope is used to obtain two images with The two-dimensional original image with lateral parallax is output, and the second original image information including the information of the two two-dimensional original images with lateral parallax is output.
  • the display device 21 further includes an image merging component, and the image merging component is respectively communicatively connected with the switching component.
  • the image merging component is used to receive the second parallax image information and can process the second parallax image information to obtain a combined image information, wherein the second parallax image information includes two two-dimensional images with lateral parallax information.
  • the second parallax image information received by the image merging component here may be the second original image information output by the three-dimensional endoscope, or it may be the second original image information mentioned above through other processing components using existing technology
  • the processed image information such as image denoising, image contrast adjustment, image distortion correction, image size cropping, image gray-scale processing, etc. (It should be understood that the processed image information also includes two two with lateral parallax Image information). In this regard, this embodiment does not impose restrictions.
  • the switching component 902 when the switching component is configured in the first state 910 according to an external instruction 901 obtained by an input device (for example, the above-mentioned switch), the switching component 902 notifies the image merging component
  • the processing function is turned off 911, the switching component 914 receives the second parallax image information 903 (which includes the information 912 of the first two-dimensional image and the information 913 of the second two-dimensional image), and combines the information of the first two-dimensional image 912 and the information 913 of the second two-dimensional image are respectively transmitted to the two display areas (that is, the first display area 915 and the second display area 916), so that the first image and the second image displayed in the two display areas
  • the image has lateral parallax, so that the display mode of the display device 21 is the three-dimensional mode 917.
  • the switching component When the switching component is configured in the second state 920, the switching component notifies the processing function of the image merging component to start 921. At this time, the image merging component processes the second disparity image information 903 and obtains the merged image information 922, and the switching component 924 receives the merged image information 922 and provides it to the two display areas (ie, the first display area). 925 and the second display area 926), so that the first image and the second image displayed in the two display areas are the same, so that the display mode of the display device 21 is the two-dimensional mode 927.
  • the image merging component processes the second disparity image information 903 and obtains the merged image information 922
  • the switching component 924 receives the merged image information 922 and provides it to the two display areas (ie, the first display area). 925 and the second display area 926), so that the first image and the second image displayed in the two display areas are the same, so that the display mode of the display device 21 is the two-dimensional mode 927.
  • This embodiment does not specifically limit the specific implementation method for the image merging component to obtain the merged image information according to the received second disparity image information.
  • the display mode of the display device is a two-dimensional mode.
  • the switching component controls the processing function of the image merging component to be turned on.
  • the image merging component receives the second original image information output by the three-dimensional endoscope, and performs image segmentation on two two-dimensional original images in the second original image information, and corrects the segmented blocks to obtain a merge Image, and output the combined image information.
  • the switching component transmits the combined image information to the two display areas.
  • the first image and the second image displayed in the two display areas are the same image, and the first virtual image and the second virtual image formed by the optical path component are also The same image, therefore, what the observer observes is a two-dimensional image.
  • the switching component When the switching component is switched to the first state by an external command, the display mode of the display device is the three-dimensional mode, the switching component controls the processing function of the image merging component to turn off, and controls the second output of the three-dimensional endoscope.
  • the information of the two two-dimensional original images in the original image information is respectively transmitted to the two display areas. In this way, the first image and the second image displayed on the display 211 have lateral parallax.
  • first virtual image and the second virtual image formed by the first image and the second image through the optical path component also have lateral parallax, and the first virtual image and the second virtual image are respectively observed by the left eye and the right eye of the observer, therefore, A corresponding three-dimensional image is formed in the observer's brain.
  • the switching component can also be used to directly switch the display mode of the display device. Specifically, when the switching component is configured in the second state, the display device is no longer equipped with an image merging module, but receives the second parallax image information through the switching component, and combines the second disparity image information contained therein. The information of the two-dimensional image is provided to the two display areas, and the information of the other two-dimensional image is discarded. This setting can also achieve a similar effect.
  • the observer can select any one of the two-dimensional original image information in the second parallax image information for output through the input device.
  • one of the two original two-dimensional images acquired by the three-dimensional endoscope may include the target position to be observed, while the other cannot be observed due to angle or parallax. , At this time, you can output the required two-dimensional image through the observer's choice.

Landscapes

  • Health & Medical Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Surgery (AREA)
  • Robotics (AREA)
  • Medical Informatics (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Molecular Biology (AREA)
  • Animal Behavior & Ethology (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Human Computer Interaction (AREA)
  • Endoscopes (AREA)

Abstract

La présente invention concerne un dispositif d'affichage et un robot chirurgical. Le dispositif d'affichage comprend un affichage, un élément de commutation et deux éléments de trajet optique. Deux images affichées dans deux zones d'affichage de l'affichage forment respectivement deux images virtuelles au moyen de deux éléments de trajet optique. Lorsque l'élément de commutation est dans un premier état, les deux zones d'affichage reçoivent respectivement deux informations d'image différentes ayant une parallaxe horizontale et, de manière correspondante, il y a une parallaxe horizontale entre les images virtuelles formées par les éléments de trajet optique. Lorsque l'élément de commutation est dans le second état, les deux zones d'affichage reçoivent les mêmes informations d'image et, de manière correspondante, les images virtuelles formées par les éléments de trajet optique sont identiques. Ainsi, le mode d'affichage du dispositif d'affichage peut être commuté entre un mode tridimensionnel et un mode bidimensionnel, de telle sorte que les médecins peuvent commodément sélectionner des modes d'affichage d'image appropriés selon les besoins, ce qui facilite non seulement l'atténuation de la fatigue oculaire de médecins provoquée par une observation à long terme d'images tridimensionnelles, mais également l'observation d'un objet dans différentes situations, en particulier dans des situations où l'objet à observer est proche.
PCT/CN2020/081811 2019-04-30 2020-03-27 Dispositif d'affichage et robot chirurgical WO2020220890A1 (fr)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CN201910363826.7 2019-04-30
CN201910363826 2019-04-30
CN201910371727.3A CN110101455B (zh) 2019-04-30 2019-05-06 显示装置及手术机器人
CN201910371727.3 2019-05-06

Publications (1)

Publication Number Publication Date
WO2020220890A1 true WO2020220890A1 (fr) 2020-11-05

Family

ID=67488319

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/081811 WO2020220890A1 (fr) 2019-04-30 2020-03-27 Dispositif d'affichage et robot chirurgical

Country Status (2)

Country Link
CN (1) CN110101455B (fr)
WO (1) WO2020220890A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116531110A (zh) * 2023-06-06 2023-08-04 上海睿触科技有限公司 一种腹腔镜手术机器人主控制台成像系统及瞳距调节方法

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110101455B (zh) * 2019-04-30 2021-01-01 微创（上海）医疗机器人有限公司 显示装置及手术机器人
CN111035452B (zh) * 2019-12-27 2021-07-02 苏州微创畅行机器人有限公司 定位工具、机械臂系统、手术系统以及注册配准方法
CN112022357B (zh) * 2020-09-16 2022-08-05 上海微创医疗机器人（集团）股份有限公司 医生控制台、手术机器人系统及医生控制台的控制方法

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1136750A (zh) * 1995-05-24 1996-11-27 王希峰 立体电视方案
CN102025942A (zh) * 2009-09-15 2011-04-20 三星电子株式会社 视频处理系统和视频处理方法
US20120127288A1 (en) * 2010-11-24 2012-05-24 Himax Media Solutions, Inc. 2D-to-3D DELAY COMPENSATION SYSTEM AND METHOD THEREOF
CN104135962A (zh) * 2012-03-21 2014-11-05 奥林巴斯株式会社 手术用图像系统以及图像显示方法
CN205003365U (zh) * 2015-09-07 2016-01-27 易志根 一种虚拟显示装置
CN109288591A (zh) * 2018-12-07 2019-02-01 微创（上海）医疗机器人有限公司 手术机器人系统
CN110101455A (zh) * 2019-04-30 2019-08-09 微创（上海）医疗机器人有限公司 显示装置及手术机器人

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8934000B2 (en) * 2008-01-29 2015-01-13 Eastman Kodak Company Switchable 2-D/3-D display system
KR20100075068A (ko) * 2008-12-24 2010-07-02 삼성전자주식회사 입체영상 표시장치 및 그 제어방법
CN101917642B (zh) * 2010-09-08 2014-08-27 利亚德光电股份有限公司 Led立体显示器及显示系统
CN102469321A (zh) * 2010-11-16 2012-05-23 承景科技股份有限公司 可自动切换操作模式的二维/三维兼用显示系统
EP4206870A1 (fr) * 2014-06-14 2023-07-05 Magic Leap, Inc. Procédés de mise à jour d'un monde virtuel
CN105629641A (zh) * 2014-10-29 2016-06-01 北京云豆科技有限公司 可穿戴式投影显示光学系统
CN105411681B (zh) * 2015-12-22 2018-07-03 哈尔滨工业大学 分体式微创手术机器人的手眼协调控制系统及方法
TWI609199B (zh) * 2016-06-30 2017-12-21 葉天守 反射式虛像顯示裝置
CN108984075B (zh) * 2017-05-31 2021-09-07 华为技术有限公司 显示模式切换方法、装置及终端

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1136750A (zh) * 1995-05-24 1996-11-27 王希峰 立体电视方案
CN102025942A (zh) * 2009-09-15 2011-04-20 三星电子株式会社 视频处理系统和视频处理方法
US20120127288A1 (en) * 2010-11-24 2012-05-24 Himax Media Solutions, Inc. 2D-to-3D DELAY COMPENSATION SYSTEM AND METHOD THEREOF
CN104135962A (zh) * 2012-03-21 2014-11-05 奥林巴斯株式会社 手术用图像系统以及图像显示方法
CN205003365U (zh) * 2015-09-07 2016-01-27 易志根 一种虚拟显示装置
CN109288591A (zh) * 2018-12-07 2019-02-01 微创（上海）医疗机器人有限公司 手术机器人系统
CN110101455A (zh) * 2019-04-30 2019-08-09 微创（上海）医疗机器人有限公司 显示装置及手术机器人

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116531110A (zh) * 2023-06-06 2023-08-04 上海睿触科技有限公司 一种腹腔镜手术机器人主控制台成像系统及瞳距调节方法
CN116531110B (zh) * 2023-06-06 2024-01-26 上海睿触科技有限公司 一种腹腔镜手术机器人主控制台成像系统及瞳距调节方法

Also Published As

Publication number Publication date
CN110101455A (zh) 2019-08-09
CN110101455B (zh) 2021-01-01

Similar Documents

Publication Publication Date Title
WO2020220890A1 (fr) Dispositif d'affichage et robot chirurgical
US11336804B2 (en) Stereoscopic visualization camera and integrated robotics platform
JP7379373B2 (ja) 立体視覚化カメラ及び統合ロボットプラットフォーム
US20220417492A1 (en) Medical devices, systems, and methods using eye gaze tracking for stereo viewer
CA3059237C (fr) Camera de visualisation stereoscopique et plate-forme
EP3146715B1 (fr) Systèmes et procédés de visualisation chirurgicale par réalité induite
CA2967470C (fr) Agrandissement au cours d'interventions ophtalmiques ainsi que dispositifs, systemes et procedes associes
JP5525521B2 (ja) ユーザによる３次元環境の検査のための内視鏡システム
WO2020147691A1 (fr) Système d'imagerie pour robot chirurgical et robot chirurgical
KR100998182B1 (ko) 수술용 로봇의 3차원 디스플레이 시스템 및 그 제어방법
CN109288591A (zh) 手术机器人系统
JP2016158911A (ja) 画像表示装置を使った外科手術方法及び、その外科手術に用いる装置
WO2022036302A1 (fr) Systèmes et procédés de superposition d'image virtuelle sur une image en temps réel
US20230222740A1 (en) Medical image processing system, surgical image control device, and surgical image control method
EP1275258A1 (fr) Conception, fonctionnement et exploitation d'un equipement de capture d'images tridimensionnelles
EP4192330A1 (fr) Gabarit assemblé sur un microscope chirurgical stéréoscopique pour l'application de techniques de réalité augmentée à des procédures chirurgicales
WO2001060076A1 (fr) Conception, fonctionnement et exploitation d'un equipement de capture d'images tridimensionnelles
RU2785887C1 (ru) Система визуализации для хирургического робота и хирургический робот
CN116531110B (zh) 一种腹腔镜手术机器人主控制台成像系统及瞳距调节方法
CN109498162B (zh) 提升沉浸感的主操作台及手术机器人
CN109498163B (zh) 主操作台及手术机器人

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20799058

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20799058

Country of ref document: EP

Kind code of ref document: A1