WO2020220890A1

WO2020220890A1 - Display device and surgical robot

Info

Publication number: WO2020220890A1
Application number: PCT/CN2020/081811
Authority: WO
Inventors: 孙颖浩; 陈功; 毛昊阳; 袁帅; 罗中宝; 何超
Original assignee: 微创（上海）医疗机器人有限公司
Priority date: 2019-04-30
Filing date: 2020-03-27
Publication date: 2020-11-05
Also published as: CN110101455B; CN110101455A

Abstract

The present invention provides a display device and a surgical robot. The display device comprises a display, a switching component, and two optical path components. Two images displayed in two display areas of the display respectively form two virtual images by means of two optical path components. When the switching component is in a first state, the two display areas respectively receive two different image information having horizontal parallax, and correspondingly, there is horizontal parallax between the virtual images formed by the optical path components. When the switching component is in the second state, the two display areas receive the same image information, and correspondingly, the virtual images formed by the optical path components are the same. Thus, the display mode of the display device can be switched between a three-dimensional mode and a two-dimensional mode, so that doctors can select appropriate image display modes conveniently according to needs, thereby facilitating not only alleviating of eye fatigue of doctors caused by long-term observation of three-dimensional images, but also observing of an object in different situations, especially in situations that the object to be observed is close.

Description

Display device and surgical robot

Technical field

The invention relates to the field of surgical instruments, in particular to a display device and a surgical robot.

Background technique

Minimally invasive surgery is accepted by more and more patients due to its advantages of small trauma and quick recovery. More and more surgical modes have slowly developed from traditional open surgery to minimally invasive surgery, and the corresponding surgical tools have also slowly developed from the early surgical forceps and scalpels to the current surgical robotic arms, Surgical robots, etc. In minimally invasive surgery, the display device can provide doctors with intuitive images, which is a very important part.

The display device of the existing surgical robot system can provide the doctor with high-fidelity three-dimensional (3D) vision, and can provide the doctor with an accurate spatial distance. The existing naked-eye 3D image is realized by providing images with different parallaxes for the left and right eyes respectively. Therefore, the doctor's console of the current surgical robot is mostly immersive. The doctor can observe three-dimensional images through the observation window on the doctor's console, and the observation position is restricted to an extremely limited position. In the process of minimally invasive surgery, the complicated surgical environment and operating tasks require doctors to manipulate robots for a long time to perform surgical operations. Observing three-dimensional images for a long time will make the doctors visually fatigued.

In addition, although the three-dimensional image can observe the depth of the position, when the endoscope is closer to the surgical instrument or tissue, the two-dimensional planar image is easier to observe.

Summary of the invention

The purpose of the present invention is to provide a display device and a surgical robot to solve the problem that the existing display device cannot switch between two-dimensional and three-dimensional images.

Therefore, according to an aspect of the present invention, there is provided a display device including:

The display includes two display areas, a first display area and a second display area, the first display area and the second display area are respectively used to receive image information, and respectively display the first image and the second image;

The switching component includes a first state and a second state, and the switching component is configured to:

When the switching component is in the first state, enabling the two display areas to respectively receive two different image information with lateral parallax;

When the switching component is in the second state, enabling the two display areas to receive the same image information; and

Two light path components, respectively a first light path component and a second light path component, the first light path component is used to receive a first image and form a first virtual image for the observer's perception of an eye, the second light path component For receiving the second image and forming a second virtual image for perception by another eye of the observer, and the first virtual image and the second virtual image are arranged in the same direction.

Optionally, the display device further includes an observation surface and a symmetry surface;

The observation plane is arranged perpendicular to the symmetry plane, and the observation plane is used for the observer to perceive the first virtual image and the second virtual image;

The first display area and the second display area are symmetrically arranged with respect to the symmetry plane;

The first optical path component and the second optical path component are symmetrically arranged about the symmetry plane;

The first optical path component and the second optical path component are configured to make the first virtual image and the second virtual image parallel to the observation surface.

Optionally, in the display device, the first optical path component and the second optical path component are configured to at least partially overlap the first virtual image and the second virtual image in the lateral direction.

Optionally, in the display device, the first optical path component and the second optical path component are configured such that the image planes of the first virtual image and the second virtual image are both located on the same virtual plane.

Optionally, in the display device, the first virtual image and the second virtual image are both arranged in a vertical direction and located on the same horizontal height.

Optionally, in the display device, each of the light path components is a reflective component, and the reflective component is used to reflect the image displayed by the display to form the first virtual image or the second virtual image.

Optionally, in the display device, each of the reflective components includes:

The first reflecting mirror is arranged at an angle to the corresponding one of the display areas; and

The second reflecting mirror is arranged at an angle to the first reflecting mirror, one extending direction of the second reflecting mirror is the same as one extending direction of the observation surface, and the other extending direction of the second reflecting mirror is the same as The other extension direction of the observation surface is arranged at an angle;

Wherein, the reflective component is configured such that after the first image or the second image is displayed by the corresponding display area, it passes through the reflection of the corresponding first mirror and the second mirror in sequence , To form the first virtual image or the second virtual image.

Optionally, in the display device, one extension direction of the second reflector and one extension direction of the observation surface both extend in a vertical direction.

Optionally, the display device further includes an offset adjustment device for adjusting the position or angle of the optical path component to adjust the lateral direction between the first virtual image and the second virtual image. distance.

Optionally, the display device further includes an image segmentation component, the image segmentation component is communicatively connected with the switching component, and the image segmentation component is configured to receive two-dimensional image information and perform processing to obtain a second The first disparity image information with the horizontal disparity in the three-dimensional image information;

When the switching component is in the first state, the switching component receives the two-dimensional image information and the first parallax image information output by the image segmentation component, and provides them to the first display area and the second display area respectively. A display area, so that the first image and the second image displayed in the two display areas have lateral parallax;

When the switching component is in the second state, the switching component receives the two-dimensional image information, and provides the same two-dimensional image information to the two display areas, so that the second display area is displayed in the two display areas. One image is the same as the second image.

Optionally, the display device further includes an image merging component, the image merging component is respectively communicatively connected with the switching component, and the image merging component is configured to receive and process the second parallax image information to obtain the combined image information , Wherein the second parallax image information includes information of two two-dimensional images with lateral parallax;

When the switching component is configured in the first state, the switching component receives the second parallax image information, and provides information of two two-dimensional images in the second parallax image information to the The first display area and the second display area, so that the first image and the second image displayed in the two display areas have lateral parallax;

When the switching component is configured in the second state, the switching component receives the merged image information output by the image merging component and provides it to the two display areas, so that the two display areas The first image and the second image displayed are the same.

Optionally, in the display device, when the switching component is configured in the second state, the switching component receives second parallax image information, wherein the second parallax image information includes two two-dimensional Image information, the switching component provides information of a two-dimensional image in the second parallax image information to the two display areas, so that the first image and the second image displayed in the two display areas the same.

In addition, according to another aspect of the present invention, a surgical robot is also provided. The surgical robot includes the display device as described above, and also includes an image acquisition device, which is communicatively connected with the display device, To obtain image information of the surgical environment and provide it to the display device.

Optionally, the surgical robot further includes:

The side cart includes at least one image arm and a tool arm, the image acquisition device is mounted on the image arm, and the surgical instrument is mounted on the tool arm; and

The doctor's console includes a main operator, and the display device is arranged on the doctor's console;

The master operating hand, the tool arm and the surgical instrument constitute a master-slave control relationship.

In summary, in the display device and the surgical robot provided by the present invention, the display device includes a display, a switching component, and two optical path components. The two images displayed in the two display areas of the display are respectively formed by the two optical path components. Two virtual images respectively perceived by the observer’s two eyes are switched between the first state and the second state through the switching component. When the switching component is in the first state, the two display areas are made to receive the horizontal parallax respectively. For two different pieces of image information, correspondingly, the virtual image formed by the optical path component has lateral parallax; when the switching component is in the second state, the two display areas are made to receive the same image information, and correspondingly, the virtual image formed by the optical path component is the same ; So that the display mode of the display device can be switched between the three-dimensional mode and the two-dimensional mode, the doctor can easily select the appropriate image display mode according to the needs, which not only helps the doctor relieve the eye fatigue caused by long-term observation of three-dimensional images, but also It is good for observing objects in different situations, especially when the object is closer to the observation.

Description of the drawings

Those of ordinary skill in the art will understand that the accompanying drawings are provided for a better understanding of the present invention, and do not constitute any limitation on the scope of the present invention. among them:

FIG. 1 is a schematic diagram of a surgical robot provided by Embodiment 1 of the present invention;

FIG. 2 is a schematic diagram of a doctor console provided by Embodiment 1 of the present invention;

FIG. 3 is a three-dimensional schematic diagram of a display device provided by Embodiment 1 of the present invention;

FIG. 4 is a front view of the display device shown in FIG. 3;

FIG. 5 is a light path diagram of the display device shown in FIG. 3;

FIG. 6 is a schematic diagram of an imaging plane provided by Embodiment 1 of the present invention;

FIG. 7 is a flowchart of image segmentation provided by Embodiment 1 of the present invention;

Fig. 8 is a flowchart of image merging provided by the second embodiment of the present invention.

In the attached picture:

10-image cart; 11-side trolley; 110-image arm; 111-endoscope; 112-tool arm; 113-surgical instrument; 12-surgical trolley; 13-tool cart; 20-doctor console; 21- Display device; 22-main operator; 23-observation window; 24-case;

211-display; 211L-first display area; 211R-second display area;

210-optical path component; 210L-first optical path component; 210R-second optical path component;

212L-first left mirror; 212R-first right mirror; 213L-second left mirror; 213R-second right mirror; 214L-left virtual image; 214R-right virtual image; 215-virtual plane; 215L-th One virtual image; 215R-second virtual image; 216-symmetry plane.

Detailed ways

In order to make the purpose, advantages and features of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the drawings are in a very simplified form and are not drawn to scale, and are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention. In addition, the structure shown in the drawings is often part of the actual structure. In particular, the focus of each drawing is different, and different scales are sometimes used.

As used in this specification and the appended claims, the singular forms "a", "an" and "the" include plural referents unless the content clearly dictates otherwise. As used in this specification and the appended claims, the term "or" is generally used to include the meaning of "and/or", unless the content clearly indicates otherwise, the term "proximal" usually refers to The end of the observer/observer, the term "distal" is usually the end close to the patient's lesion.

The present invention provides a display device, which includes a display, a switching component, and two light path components. The display includes two display areas, a first display area and a second display area, respectively. The two display areas are respectively used to receive image information and respectively display the first image and the second image; the switching component includes a first state and a second state, and the switching component is configured to: when the switching component is in the first state In the state, the two display areas are made to respectively receive two different image information with lateral parallax; when the switching component is in the second state, the two display areas are made to receive the same image information; The light path components are respectively a first light path component and a second light path component. The first light path component is used for receiving a first image and forming a first virtual image for the observer's perception of an eye, and the second light path component is used for The second image is received and a second virtual image for perception by another eye of the observer is formed, and the first virtual image and the second virtual image are arranged in the same direction. With this configuration, when the switching component is in the first state, the two display areas are made to receive two different image information with lateral parallax. Correspondingly, the virtual images formed by the optical path component have lateral parallax, so that the observer can perceive Two virtual images of lateral parallax, and a three-dimensional image is formed in the brain; when the switching component is in the second state, the two display areas receive the same image information, and correspondingly, the virtual images formed by the optical path components are the same to make the observer perceive the same Two virtual images of, and form a two-dimensional image on the brain; thus the display mode of the display device can be switched between three-dimensional mode and two-dimensional mode, and observers (such as doctors or assistants) can easily select the appropriate image according to their needs The display mode not only helps the observer alleviate eye fatigue caused by observing three-dimensional images for a long time, but also helps to observe the object in different situations, especially when it is close to the object to be observed.

The following describes with reference to the drawings.

[Example One]

Please refer to Figures 1 to 7, where Figure 1 is a schematic diagram of a surgical robot provided in Embodiment 1 of the present invention, Figure 2 is a schematic diagram of a doctor console provided in Embodiment 1 of the present invention, and Figure 3 is a schematic diagram of a doctor console provided in Embodiment 1 of the present invention. Fig. 4 is a front view of the display device shown in Fig. 3, Fig. 5 is a light path diagram of the display device shown in Fig. 3, and Fig. 6 is a schematic diagram of an imaging plane provided by Embodiment 1 of the present invention, FIG. 7 is a flowchart of image segmentation provided by Embodiment 1 of the present invention.

As shown in FIG. 1, the first embodiment of the present invention provides a surgical robot, which includes: an image cart 10, a side cart 11, an operating trolley 12, a tool cart 13, and a doctor console 20. The doctor can remotely use the doctor console 20. Operation to achieve minimally invasive surgical treatment of patients on the operating trolley 12. The image trolley 10 can provide the doctor with auxiliary image data, etc. The tool trolley 13 can provide the doctor with the tools needed in the operation. The side trolley 11 includes at least one image arm 110 and a tool arm 112. An image acquisition device is mounted on the image arm 110, and the image acquisition device is communicatively connected with the display device, and is used to acquire image information of the surgical environment and provide it to the display device. The image acquisition device is used to acquire and provide image information of the surgical environment including human tissues and organs, surgical instruments, blood vessels, and body fluids to the display device. A surgical instrument 113 is mounted on the tool arm 112. The image acquisition device may be an endoscope 111 or a laparoscope, for example. The endoscope 111 and the surgical instrument 113 respectively enter the patient's position through the wound on the patient's body to achieve minimally invasive surgical treatment. Preferably, as shown in FIGS. 2 to 4, the doctor console 20 includes a main operating hand 22. The master operator 22, the tool arm 112 of the side cart 11 and the surgical instrument 113 constitute a master-slave control relationship. Specifically, the tool arm 112 moves according to the movement of the main operating hand 22 during the operation, that is, moves according to the operation of the doctor's hand. Further, the main operating hand 22 also receives the force information of the human tissues and organs on the surgical instrument 113 and feeds it back to the doctor's hand, so that the doctor can more intuitively feel the surgical operation.

As shown in FIGS. 3 to 5, the display device 21 includes a switching component (not shown), a display 211 and two optical path components 210. Preferably, the display device 21 further includes a housing 24 and an observation window 23. The display 211 and the two light path components 210 are arranged inside the housing 24, and the observation window 23 is arranged on the housing 24 for the observer to observe images of the surgical environment. Preferably, the display device 21 is arranged on the doctor console 20. Furthermore, the doctor console 20 further includes a control component (not shown) for connecting with the main operating hand 22. The original image information acquired by the endoscope 111 or the processed image information obtained by processing the original image information can be transmitted to the control component, and the control component is used to process the original image information or the original image information through processing. The image information is transmitted to the display device 21. Those skilled in the art should understand that the switching component in the display device 21 and the control component in the doctor console 20 may be a whole, that is, the function of the switching component can be implemented in the control component. This is also within the protection scope of the present invention. Preferably, the display device 21 further includes an observation plane and a symmetry plane, and the observation plane is arranged perpendicular to the symmetry plane. Here, the “symmetry plane” can be a physical object or a virtual object, and the observation plane The surface can coincide with the plane where the observation window 23 is located for observation by the observer. The display 211 includes two display areas, a first display area 211L and a second display area 211R, respectively. Preferably, the first display area 211L and the second display area 211R are arranged symmetrically about the symmetry plane. The two display areas are respectively used for receiving image information and respectively displaying the first image and the second image. It should be understood that the first display area 211L is not limited to be located on the left side of the second display area 211R as shown in FIG. 3, but it means that the first display area 211L and the second display area 211R are two oppositely arranged display areas. The positions of the one display area 211L and the second display area 211R are interchangeable. In particular, the two display areas may be composed of two different displays respectively, or may be composed of different display areas in the same display, which is not limited in the present invention.

The two optical path components 210 are respectively a first optical path component 210L and a second optical path component 210R. Preferably, the first optical path component 210L and the second optical path component 210R are arranged symmetrically about the plane of symmetry, and the first optical path component 210L is used for Receive the first image and form a first virtual image for one eye of the observer, and the second optical path component 210R is used to receive the second image and form the other eye for the observer The second virtual image is perceived, and the first virtual image and the second virtual image are arranged in the same direction. The three-dimensional image can be formed in the brain when the observer's left and right eyes respectively observe a virtual image with the same arrangement direction and lateral parallax. The arrangement of the first virtual image and the second virtual image in the same direction means that the postures of the same object described in the first virtual image and the second virtual image in each virtual image are the same or similar. For example, the first virtual image is an image with an arrow pointing up, and the second virtual image must also be an image with an arrow pointing up. Preferably, the first optical path component 210L and the second optical path component 210R are configured to make the image planes of the first virtual image and the second virtual image parallel to the observation plane. More preferably, according to the overlapping features of the left and right eyes of the human eye, and in order to make the display device more compact, the first light path component 210L and the second light path component 210R are configured to make the first virtual image and the second virtual image in The horizontal direction (ie, the lateral direction) overlaps at least partially. The "overlap" here can mean overlapping on the first virtual image and the second virtual image on the same virtual plane, or it can be understood that the projection planes of the first virtual image and the second virtual image respectively projected onto a parallel plane overlap each other. Preferably, the first optical path component 210L and the second optical path component 210R are configured such that the image planes of the first virtual image and the second virtual image are both located on the same virtual plane 215. Similarly, it is not limited here that the first optical path component 210L is located on the left side of the second optical path component 210R, but refers to the first optical path component 210L and the second optical path component 210R as two opposing optical path components. Further, the first light path component 210L corresponds to the first display area 211L and is used to form a first virtual image, and the second light path component 210R corresponds to the second display area 211R and is used to form a second virtual image. The image plane position of the first virtual image can be changed by changing the structure of the first optical path component 210R. Similarly, the image plane position of the second virtual image can be changed by changing the structure of the second optical path component 210R.

The switching component includes a first state and a second state, and the switching component is configured to: when the switching component is in the first state, the two display areas respectively receive two different image information with lateral parallax ; When the switching component is in the second state, the two display areas receive the same image information. Specifically, please refer to FIG. 6 in conjunction with FIG. 2. When the switching component is configured in the first state, the display mode of the display device 21 is a three-dimensional mode, and the two display areas respectively receive different values with lateral parallax. The two image information, and form the first image and the second image. After passing through the optical path component 210, the first image forms a first virtual image 215L, the second image forms a second virtual image 215R, and the first virtual image 215L and the second virtual image 215R have lateral parallax. The observer’s eyes are located on the observation surface. For example, when the observation window 23 (human eye observation point LR) observes, the left eye is at the human eye observation point L, the first virtual image 215L is observed by the left eye, and the right eye is at the human eye observation point R. The second virtual image 215R is observed by the right eye, and the first virtual image 215L and the second virtual image 215R with lateral parallax enter the left and right eyes of the observer respectively. With the recognition and matching ability of the brain, the synthesized three-dimensional image can be observed. When the switching component is configured in the second state, the display mode of the display device 21 is a two-dimensional mode, the first image and the second image are the same image, and after passing through the optical path component 210, the first The first virtual image 215L formed by the image and the second virtual image 215R formed by the second image are also the same image. At this time, the left and right eyes of the observer see the same image, and thus the two-dimensional image is observed. In the display device provided by this embodiment, the switching component is used to switch between the first state and the second state. When the switching component is in the first state, the two display areas are made to receive two different images with lateral parallax. Information; when the switching component is in the second state, the two display areas receive the same image information; so that the display mode of the display device 21 can be switched between the three-dimensional mode and the two-dimensional mode, and the doctor can easily select the appropriate one according to needs The image display mode not only helps doctors alleviate eye fatigue caused by long-term observation of three-dimensional images, but also helps to observe objects in different situations, especially when the endoscope 111 is close to the surgical instrument 113 or human tissue. The two-dimensional plane image is easier to be recognized and observed by the observer, so it is especially suitable for the situation where the object to be observed is closer.

Preferably, the display device 21 further includes a switch, the switch is in communication connection with the switch component, and the observer inputs an instruction to make the switch component between the first state and the second state by operating the switch Switching, in turn, controls the display device 21 to switch between the three-dimensional mode and the two-dimensional mode, so that the doctor can switch when visual fatigue or select a more appropriate image display mode according to factors such as the position of the displayed object, and improve the safety of the operation. It should be noted that the present invention has no particular restrictions on the specific shape and position of the switch, which can be a virtual button on the display device 21, can be a physical component on the doctor's console, such as the main operating hand 22, or Physical parts located near the feet of the observer, etc.

Further, please refer to Figures 3 to 6, the first virtual image and the second virtual image are arranged along the vertical direction, and are located on the same horizontal height, that is, the first virtual image and the second virtual image are respectively vertical to the horizontal plane, and both are along the horizontal Direction arrangement. In the operation, the observer is generally sitting or standing, and the binocular connection is generally horizontal. Therefore, the first virtual image 215L and the second virtual image 215R are arranged along the vertical direction and are located at the same horizontal height, which is beneficial for the observer to observe.

The optical path component 210 is preferably a reflective component, and the reflective component is used to reflect the image displayed by the display 211. The following takes the first optical path assembly 210L as an example for specific introduction. The reflecting assembly includes a first left reflecting mirror 212L and a second left reflecting mirror 213L. The first left mirror 212L is arranged at an angle to the corresponding first display area 211L; the second left mirror 213L is arranged at an angle to the first left mirror 212L, and one of the second left mirrors 213L extends The direction is the same as one extension direction of the observation surface, and the other extension direction of the second mirror 213L is arranged at an angle to the other extension direction of the observation surface. Generally, a plane can be considered to have two extension directions (that is, the directions of two adjacent sides, such as the X and Y directions perpendicular to each other), and the mirror and the observation surface can be abstracted into a plane, so both It has two extension directions, namely the directions of two adjacent sides of the mirror and the observation surface respectively. Here, an extension direction of the second left mirror 213L is the same as an extension direction of the observation surface, which can ensure that the arrangement direction of the first virtual image 215L observed from the observation surface will not be changed by the second left mirror 213L . For example, one extension direction of the second left mirror 213L and one extension direction of the observation surface both extend in the vertical direction, so that the arrangement direction of the first virtual image 215L can also be the vertical direction. Wherein, the reflection component is configured such that, after the first image is displayed by the first display area 211L, it is reflected by the first left mirror 212L and the second left mirror 213L in order to form a first virtual image 215L. In this embodiment, each optical path assembly preferably includes two reflective devices. If the number of reflective devices in the optical path assembly is too large, the display device 21 will be bulky and increase the difficulty of installation; and if the number of reflective devices is too small (such as Only one piece) will cause the size of the reflective device to be too large, which in turn will also cause the display device 21 to be bulky.

As shown in FIG. 4, in an exemplary embodiment, the first display area 211L and the second display area 211R are both separate displays, and the two light path components and the two display areas are all arranged symmetrically about the symmetry plane. The first display area 211L is located above the first left mirror 212L and the second left mirror 213L, and the second left mirror 213L is located between the symmetry plane and the first left mirror 212L. As shown in FIG. 5, the first left mirror 212L is used to reflect the first image displayed in the first display area 211L to form a left virtual image 214L; the second left mirror 213L is used to perform a first reflection on the left virtual image 214L. The second reflection forms a first virtual image 215L. According to the imaging principle, the left virtual image 214L and the first image displayed in the first display area 211L are symmetric about the first left mirror 212L, and the left virtual image 214L and the first virtual image 215L are symmetric about the second left mirror 213L. Further, by adjusting the arrangement angles of the first display area 211L, the first left mirror 212L, and the second left mirror 213L, the first image in the first display area 211L passes through the first left mirror 212L and the second left mirror 213L. The reflection forms a first virtual image 215L, which is parallel to the observation surface. The angle of the second left mirror 213L may be determined according to the angle of the first virtual image 215L and the angle of the left virtual image 214L. Furthermore, the image plane of the left virtual image 214L is parallel to the symmetry plane, and the angle of the first left mirror 212L can be determined according to the angle of the first display area 211L and the left virtual image 214L. Preferably, the two extension directions of the first left mirror 212L and the two extension directions of the second left mirror 213L are respectively arranged at an angle. More preferably, the first left mirror 212L is arranged perpendicular to the image plane of the first virtual image 215L, which is beneficial to reduce the volume of the entire optical path assembly. The second optical path component 210R has the same configuration as the first optical path component 210L and is symmetrical to the first optical path component 210L about the symmetry plane. The second optical path component 210R includes a first right mirror 212R and a second right mirror 213R, The first right mirror 212R is used to reflect the second image displayed in the second display area 211R to form a right virtual image 214R, and the second right mirror 213R is used to reflect the right virtual image 214R a second time to form a second virtual image 215R. The specific structure and principle of the second optical path assembly 210R are the same as those of the first optical path assembly 210L, and will not be repeated here.

Preferably, the display device 21 further includes an offset adjustment device (not shown), the offset adjustment device is used to adjust the optical path assembly 210 to adjust the gap between the first virtual image 215L and the second virtual image 215R To adjust the horizontal overlap range of the two virtual images. The offset adjustment device can control the arrangement angle of each reflector in the optical path assembly 210 in linkage, so that different observers can adjust the displacement of the first virtual image 215L and the second virtual image 215R as needed to adjust the first virtual image 215L and the second virtual image 215L. The horizontal distance between the two virtual images 215R is the horizontal overlap range, which is more in line with the different pupil distances of the observer and ensures the clarity of the observed image.

In this embodiment, the endoscope 111 is a two-dimensional endoscope, and the two-dimensional endoscope is used to obtain a two-dimensional original image and output first original image information. The display device 21 further includes an image segmentation component, and the image segmentation component is in communication connection with the switching component. The image segmentation component is used to receive and process two-dimensional image information to obtain first parallax image information having a lateral parallax with respect to the received two-dimensional image information. Here, the two-dimensional image information received by the image segmentation component may be the first original image information output by the two-dimensional endoscope, or the first original image information processed by other processing components using existing technology, such as Image denoising, image contrast adjustment, image distortion correction, image size cropping, image grayscale processing, etc. In this regard, this embodiment does not impose restrictions.

As shown in FIG. 7, the switching component is in the first state 810 according to the external instruction 801 obtained by the input device (for example, the above-mentioned switching switch), and notifies the processing function of the image segmentation component to start 811. The image segmentation component obtains the first disparity image information 813 according to the received two-dimensional image information 803, and the switching component 814 receives the two-dimensional image information 812 (this two-dimensional image information 812 may be the same as the two-dimensional image information received by the image segmentation component). Dimensional image information 803) and the aforementioned first parallax image information 813 are provided to the first display area 815 and the second display area 816, so that the first image and the second image displayed in the two display areas The image has lateral parallax, even if the display mode of the display device 21 is the three-dimensional mode 817. On the other hand, the switching component is in the second state 820 according to the external instruction 801 and notifies the processing function of the image segmentation component to close 821. At this time, the switching component 824 receives the two-dimensional image information 822, and provides the two display areas (ie, the first display area 825 and the second display area 826) of the display with the same two-dimensional image information 822. (The two-dimensional image information 822 may be the same as the two-dimensional image information 803 received by the switching component), so that the first image and the second image displayed in the two display areas are the same, even if the display mode of the display device 21 is Two-dimensional model 827.

Further, in this embodiment, there is no particular limitation on the specific implementation method for the image segmentation component to obtain the first disparity image information according to the received two-dimensional image information. For example, the image segmentation component may perform image segmentation on the two-dimensional original image corresponding to the first original image information, thereby generating a depth map, and by adjusting the displacement of the segmentation block, the original image relative to the two-dimensional original image can be obtained. The image has a two-dimensional parallax image with lateral parallax, and outputs first parallax image information. When the switching component is switched to the first state by an external instruction, the display mode of the display device 21 is a three-dimensional mode. At this time, the switching component controls the processing function of the image segmentation component to be turned on. The image segmentation component receives the first original image information output by the two-dimensional endoscope, performs image segmentation on the two-dimensional original image, and then generates a depth map. By adjusting the displacement of the segmentation block, the The two-dimensional original image has a two-dimensional parallax image with lateral parallax, and outputs first parallax image information. The switching component transmits the first original image information and the first parallax image information to the two display areas of the display, so that the two display areas respectively receive the first original image information and the first parallax image information, and display the first original image information and the first parallax image information. One image and second image. Since the right eye of ordinary people is more sensitive, preferably, the first original image information is transmitted to the display area corresponding to the right eye, such as the second display area 211R. Since the first image and the second image displayed on the display 211 have lateral parallax, the first virtual image and the second virtual image formed through the optical path component also have lateral parallax. In addition, the first virtual image and the second virtual image are respectively observed by the left eye and the right eye of the observer. Therefore, a corresponding three-dimensional image is formed in the brain of the observer. When the switching component is switched to the second state by an external command, the display mode of the display device 21 is a two-dimensional mode. At this time, the switching component controls the processing function of the image segmentation component to turn off, and controls the first original image information output by the two-dimensional endoscope to be transmitted to two display areas, the first image displayed in the two display areas It is the same image as the second image, and the first virtual image and the second virtual image formed by the optical path component are also the same image. Therefore, what the observer observes is a two-dimensional image.

[Example 2]

The display device and the surgical robot of the second embodiment of the present invention are basically the same as those of the first embodiment, and the same parts will not be described, and only the different points will be described below.

Please refer to FIG. 8, which is a flowchart of image merging provided in the second embodiment of the present invention. In this embodiment, the endoscope 111 is a three-dimensional endoscope, and the three-dimensional endoscope is used to obtain two images with The two-dimensional original image with lateral parallax is output, and the second original image information including the information of the two two-dimensional original images with lateral parallax is output. The display device 21 further includes an image merging component, and the image merging component is respectively communicatively connected with the switching component. The image merging component is used to receive the second parallax image information and can process the second parallax image information to obtain a combined image information, wherein the second parallax image information includes two two-dimensional images with lateral parallax information. Similar to the foregoing embodiment, the second parallax image information received by the image merging component here may be the second original image information output by the three-dimensional endoscope, or it may be the second original image information mentioned above through other processing components using existing technology The processed image information, such as image denoising, image contrast adjustment, image distortion correction, image size cropping, image gray-scale processing, etc. (It should be understood that the processed image information also includes two two with lateral parallax Image information). In this regard, this embodiment does not impose restrictions.

As shown in FIG. 8, when the switching component is configured in the first state 910 according to an external instruction 901 obtained by an input device (for example, the above-mentioned switch), the switching component 902 notifies the image merging component The processing function is turned off 911, the switching component 914 receives the second parallax image information 903 (which includes the information 912 of the first two-dimensional image and the information 913 of the second two-dimensional image), and combines the information of the first two-dimensional image 912 and the information 913 of the second two-dimensional image are respectively transmitted to the two display areas (that is, the first display area 915 and the second display area 916), so that the first image and the second image displayed in the two display areas The image has lateral parallax, so that the display mode of the display device 21 is the three-dimensional mode 917. When the switching component is configured in the second state 920, the switching component notifies the processing function of the image merging component to start 921. At this time, the image merging component processes the second disparity image information 903 and obtains the merged image information 922, and the switching component 924 receives the merged image information 922 and provides it to the two display areas (ie, the first display area). 925 and the second display area 926), so that the first image and the second image displayed in the two display areas are the same, so that the display mode of the display device 21 is the two-dimensional mode 927.

This embodiment does not specifically limit the specific implementation method for the image merging component to obtain the merged image information according to the received second disparity image information. For example, when the switching component is switched to the second state by an external command, the display mode of the display device is a two-dimensional mode. At this time, the switching component controls the processing function of the image merging component to be turned on. The image merging component receives the second original image information output by the three-dimensional endoscope, and performs image segmentation on two two-dimensional original images in the second original image information, and corrects the segmented blocks to obtain a merge Image, and output the combined image information. The switching component transmits the combined image information to the two display areas. At this time, the first image and the second image displayed in the two display areas are the same image, and the first virtual image and the second virtual image formed by the optical path component are also The same image, therefore, what the observer observes is a two-dimensional image. When the switching component is switched to the first state by an external command, the display mode of the display device is the three-dimensional mode, the switching component controls the processing function of the image merging component to turn off, and controls the second output of the three-dimensional endoscope. The information of the two two-dimensional original images in the original image information is respectively transmitted to the two display areas. In this way, the first image and the second image displayed on the display 211 have lateral parallax. Further, the first virtual image and the second virtual image formed by the first image and the second image through the optical path component also have lateral parallax, and the first virtual image and the second virtual image are respectively observed by the left eye and the right eye of the observer, therefore, A corresponding three-dimensional image is formed in the observer's brain.

In some other embodiments, the switching component can also be used to directly switch the display mode of the display device. Specifically, when the switching component is configured in the second state, the display device is no longer equipped with an image merging module, but receives the second parallax image information through the switching component, and combines the second disparity image information contained therein. The information of the two-dimensional image is provided to the two display areas, and the information of the other two-dimensional image is discarded. This setting can also achieve a similar effect. Preferably, the observer can select any one of the two-dimensional original image information in the second parallax image information for output through the input device. In some cases, one of the two original two-dimensional images acquired by the three-dimensional endoscope may include the target position to be observed, while the other cannot be observed due to angle or parallax. , At this time, you can output the required two-dimensional image through the observer's choice.

The foregoing description is only a description of the preferred embodiments of the present invention, and is not intended to limit the scope of the present invention. Any changes or modifications made by a person of ordinary skill in the field of the present invention based on the foregoing disclosure shall fall within the protection scope of the claims.

Claims

A display device, characterized by comprising:

The display includes two display areas, a first display area and a second display area, the first display area and the second display area are respectively used to receive image information, and respectively display the first image and the second image;

The switching component includes a first state and a second state, and the switching component is configured to:

When the switching component is in the first state, enabling the two display areas to respectively receive two different image information with lateral parallax;

When the switching component is in the second state, enabling the two display areas to receive the same image information; and

Two light path components, respectively a first light path component and a second light path component, the first light path component is used to receive a first image and form a first virtual image for the observer's perception of an eye, the second light path component For receiving the second image and forming a second virtual image for perception by another eye of the observer, and the first virtual image and the second virtual image are arranged in the same direction.
The display device of claim 1, wherein the display device further comprises an observation surface and a symmetry surface;

The observation plane is arranged perpendicular to the symmetry plane, and the observation plane is used for the observer to perceive the first virtual image and the second virtual image;

The first display area and the second display area are symmetrically arranged with respect to the symmetry plane;

The first optical path component and the second optical path component are symmetrically arranged about the symmetry plane;

The first optical path component and the second optical path component are configured to make the first virtual image and the second virtual image parallel to the observation surface.
The display device according to claim 1 or 2, wherein the first optical path component and the second optical path component are configured to at least partially overlap the first virtual image and the second virtual image in the lateral direction.
The display device according to claim 1 or 2, wherein the first optical path component and the second optical path component are configured to make the image planes of the first virtual image and the second virtual image lie on the same virtual plane .
The display device according to claim 1 or 2, wherein the first virtual image and the second virtual image are both arranged in a vertical direction and located on the same level.
The display device according to claim 2, wherein each of the light path components is a reflective component, and the reflective component is used to reflect the image displayed by the display to form the first virtual image or the second virtual image. Virtual image.
7. The display device of claim 6, wherein each of the reflective components comprises:

The first reflecting mirror is arranged at an angle to the corresponding one of the display areas; and

The second reflecting mirror is arranged at an angle to the first reflecting mirror, one extending direction of the second reflecting mirror is the same as one extending direction of the observation surface, and the other extending direction of the second reflecting mirror is the same as The other extension direction of the observation surface is arranged at an angle;

Wherein, the reflective component is configured such that after the first image or the second image is displayed by the corresponding display area, it passes through the reflection of the corresponding first mirror and the second mirror in sequence , To form the first virtual image or the second virtual image.
8. The display device according to claim 7, wherein an extension direction of the second reflector and an extension direction of the observation surface both extend in a vertical direction.
4. The display device according to claim 1, wherein the display device further comprises an offset adjustment device, the offset adjustment device is used to adjust the position or angle of the optical path component to adjust the first virtual image And the lateral distance between the second virtual image.
The display device according to claim 1, wherein the display device further comprises an image segmentation component, the image segmentation component is in communication connection with the switching component, and the image segmentation component is configured to receive two-dimensional image information and Performing processing to obtain first parallax image information having lateral parallax with respect to the received two-dimensional image information;

When the switching component is in the first state, the switching component receives the two-dimensional image information and the first parallax image information output by the image segmentation component, and provides them to the first display area and A second display area, so that the first image and the second image displayed in the two display areas have lateral parallax;

When the switching component is in the second state, the switching component receives the two-dimensional image information, and provides the same two-dimensional image information to the two display areas, so that the second display area is displayed in the two display areas. One image is the same as the second image.
The display device according to claim 1, wherein the display device further comprises an image merging component, the image merging component is respectively communicatively connected with the switching component, and the image merging component is configured to receive the second parallax image Information and processing to obtain combined image information, wherein the second parallax image information includes information of two two-dimensional images with lateral parallax;

When the switching component is configured in the first state, the switching component receives the second parallax image information, and provides information of two two-dimensional images in the second parallax image information to the The first display area and the second display area, so that the first image and the second image displayed in the two display areas have lateral parallax;

When the switching component is configured in the second state, the switching component receives the merged image information output by the image merging component and provides it to the two display areas, so that the two display areas The first image and the second image displayed are the same.
The display device according to claim 1, wherein when the switching component is configured in the second state, the switching component receives second parallax image information, wherein the second parallax image information includes two Information of two two-dimensional images, the switching component provides information of one two-dimensional image in the second parallax image information to the two display areas, so that the first image and the one displayed in the two display areas The second image is the same.
A surgical robot, characterized in that the surgical robot comprises the display device according to claim 1, and further comprises an image acquisition device, and the image acquisition device is communicatively connected with the display device for acquiring images of the surgical environment And provide information to the display device.
The surgical robot according to claim 13, wherein the surgical robot further comprises:

The side cart includes at least one image arm and a tool arm, the image acquisition device is mounted on the image arm, and the surgical instrument is mounted on the tool arm; and

The doctor's console includes a main operator, and the display device is arranged on the doctor's console;

The master operating hand, the tool arm and the surgical instrument constitute a master-slave control relationship.