CN115737143A - Surgical imaging system and method for displaying imaging surgery - Google Patents
Surgical imaging system and method for displaying imaging surgery Download PDFInfo
- Publication number
- CN115737143A CN115737143A CN202211384038.4A CN202211384038A CN115737143A CN 115737143 A CN115737143 A CN 115737143A CN 202211384038 A CN202211384038 A CN 202211384038A CN 115737143 A CN115737143 A CN 115737143A
- Authority
- CN
- China
- Prior art keywords
- head
- display
- wearer
- predetermined value
- movement
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Abstract
The application discloses operation imaging system and imaging operation's display method, operation imaging system includes operation imaging device (10) and wear-type augmented reality/virtual reality display device (20), operation imaging device sets up to be able to with the image output who observes, wear-type augmented reality/virtual reality display device is including wear-type display (21) that can receive and show the image of microscope output, the control unit and the monitoring unit that is used for monitoring the removal of the person's head of wearing, the control unit is connected with the monitoring unit electricity, in order can be according to the display effect of the image of operation imaging device output of monitoring unit's feedback control wear-type display. This application can strengthen display device for the person of wearing through the head-mounted and show the image that operation imaging device observed to the display effect of corresponding control head mounted display when monitoring the person of wearing head and moving is monitored to the monitoring element, so that the person of wearing can conveniently carry out other operations when the skew operation area of sight.
Description
Technical Field
The present application relates to the field of imaging surgery, and more particularly, to a surgical imaging system and a method of displaying an imaging surgery.
Background
Surgical imaging systems (including surgical microscope systems, surgical endoscopic systems) have been widely used in surgical procedures.
The operation microscope system usually comprises a primary eyepiece and an auxiliary eyepiece, which are used by different operators respectively, and the primary eyepiece and the auxiliary eyepiece are physically connected, so that the distance between the operators is relatively short, and the respective operation range and degree of freedom are influenced. Although the external screen can be used for displaying the image observed by the surgical microscope, in order to reduce the frequent switching of the sight line of the operator among the positions of the screen, the operation area, the instrument area and the like in the surgical operation process, the screen needs to be arranged opposite to the operator, and the arrangement requirement seriously influences the position of the operator and auxiliary equipment which are fixed relatively during the surgery.
An endoscope system typically includes an endoscope, a processing device that processes images obtained by the endoscope, and a display that displays images output by the processing device. Similar to the surgical microscope system, in order to reduce the frequent switching of the sight line of the operator between the positions of the screen, the operation area, the instrument area and the like during the surgical operation, the screen needs to be arranged opposite to the operator, and the arrangement requirement seriously influences the standing position of the operator and the position of auxiliary equipment which are relatively fixed during the surgical operation.
Therefore, how to conveniently provide the operator with images of the surgical imaging system becomes a technical problem to be solved in the art.
Disclosure of Invention
In view of the above, the present application provides a surgical imaging system, wherein the surgical imaging system includes a surgical imaging device and a head-mounted augmented reality/virtual reality display device, the surgical imaging device is configured to output an observed image, the head-mounted augmented reality/virtual reality display device includes a head-mounted display capable of receiving and displaying the image output by the microscope, a control unit and a monitoring unit for monitoring the movement of the head of the wearer, the control unit is electrically connected to the monitoring unit to control the display effect of the image output by the surgical imaging device displayed by the head-mounted display according to the feedback of the monitoring unit.
Optionally, the control unit is configured to adjust a display effect of the image output by the surgical imaging device displayed by the head-mounted display when the monitoring unit monitors that the movement of the head of the wearer relative to a standard position reaches a predetermined value.
Optionally, the head-mounted display comprises a transparent display screen, and the control unit is configured to reduce the display effect of the head-mounted display when the monitoring unit monitors that the movement of the head of the wearer relative to the standard position reaches a predetermined value.
Optionally, the control unit controls at least one of brightness, contrast and saturation of a picture displayed by the head mounted display according to the feedback of the monitoring unit.
Optionally, the head-mounted augmented reality/virtual reality display device comprises a camera unit mounted on the head-mounted display and used for shooting an image in front of the head-mounted display, and the control unit is configured to control the head-mounted display to display the image shot by the camera unit when the monitoring unit monitors that the movement of the head of the wearer relative to a standard position reaches a predetermined value.
Optionally, the surgical imaging device is a surgical microscope and includes a primary eyepiece and a secondary eyepiece, the surgical imaging system including the head-mounted augmented reality/virtual reality display device for displaying an observed image of at least one of the primary eyepiece and the secondary eyepiece.
Optionally, the monitoring unit is mounted to the head mounted display to reflect movement of the wearer's head by monitoring movement of the head mounted display.
Optionally, the monitoring unit is configured to monitor at least one of a moving distance, a moving speed, a rotating angle, and a rotating speed of the head-mounted display.
Optionally, the monitoring unit comprises an accelerometer and/or a gyroscope.
Optionally, the predetermined value is at least one of a moving distance of the head-mounted display relative to a standard position reaching a first predetermined value or more, a moving speed reaching a second predetermined value or more, a rotation angle reaching a third predetermined value or more, and a rotation speed reaching a fourth predetermined value or more.
The application also provides a display method of the imaging operation, wherein the display method comprises the step of displaying an image output by the operation imaging device by using a head-mounted display of a head-mounted augmented reality/virtual reality display device, and the display method further comprises the steps of monitoring the movement of the head of a wearer and controlling the display effect of the image output by the operation imaging device displayed by the head-mounted display according to the movement.
Optionally, the display method comprises adjusting a display effect of an image output by the surgical imaging device displayed by the head-mounted display when movement of the wearer's head relative to a standard position is monitored to reach a predetermined value.
Optionally, the head mounted display comprises a transparent display screen, and the display method comprises reducing the display effect of the head mounted display when the movement of the wearer's head relative to a standard position is monitored to reach a predetermined value.
Optionally, the display method comprises adjusting at least one of brightness, contrast, saturation of a picture displayed by the head-mounted display when the movement of the wearer's head relative to a standard position is monitored to reach a predetermined value.
Optionally, the head-mounted augmented reality/virtual reality display device comprises a camera unit mounted on the head-mounted display and used for shooting an image in front of the head-mounted display, and the display method comprises controlling the head-mounted display to display the image shot by the camera unit when the movement of the head of the wearer relative to a standard position is monitored to reach a preset value.
Optionally, the predetermined value includes at least one of a moving distance of the head-mounted display relative to a standard position reaching a first predetermined value or more, a moving speed reaching a second predetermined value or more, a rotation angle reaching a third predetermined value or more, and a rotation speed reaching a fourth predetermined value or more.
According to the technical scheme of this application, can strengthen display device through the head-mounted to show the image that the imaging device surveyed for the person of wearing to the display effect of corresponding control head-mounted display when monitoring the person of wearing head and moving is monitored to make the person of wearing can conveniently carry out other operations when the operation region is deviated to the sight.
Additional features and advantages of the present application will be described in detail in the detailed description which follows.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this application, illustrate an embodiment of the invention and, together with the description, serve to explain the invention. In the drawings:
FIG. 1 is a schematic view of a surgical imaging system according to a preferred embodiment of the present application.
Detailed Description
The technical solutions of the present application will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
The application provides a surgery imaging system, wherein, surgery imaging system includes surgery imaging device 10 and wear-type augmented reality/virtual reality display device 20, surgery imaging device 10 sets up to be able to with the image output who observes, wear-type augmented reality/virtual reality display device 20 is including wear-type display 21, the control unit that can receive and show the image of microscope output and the monitoring unit that is used for monitoring the removal of wearer's head, the control unit with the monitoring unit electricity is connected, with can be according to the feedback control of monitoring unit wear-type display 21 shows the display effect of the image of surgery imaging device output.
The surgical imaging device 10 may include, among other things, a surgical microscope (as shown in fig. 1), a surgical endoscope (e.g., laparoscope, rhinoscope, etc.).
With the surgical imaging system of the present application, the image observed by the surgical imaging device 10 can be displayed for the wearer through the head-mounted enhanced display device 20, and the display effect of the head-mounted display 21 is controlled correspondingly when the monitoring unit monitors the head movement of the wearer, so that the wearer can conveniently perform other operations when the line of sight deviates from the operation area.
Specifically, the surgical imaging system may be set such that the head mounted augmented display device 20 displays the image output by the surgical imaging device 10 when the wearer is in the operating region constituted by the range of positions that the wearer is comfortable with, so that the wearer can perform the surgical operation conveniently. When the line of sight of the wearer deviates from the operating area, it indicates that the wearer needs to perform other operations (for example, taking and placing an instrument), and at this time, the wearer does not need to pay attention to the image output by the surgical imaging device 10, and the control unit may adjust the display effect of the head-mounted display 21 according to the movement of the head of the wearer monitored by the monitoring unit, for example, may weaken the display effect or stop displaying the image output by the surgical imaging device 10.
To ensure that the image output by the surgical imaging device 10 is always obtained by the head mounted display 21 when the line of sight of the wearer stays within the range of the operating area, and to avoid the mis-adjustment of the head mounted display 21 by the wearer due to minor head movements, the control unit is preferably configured to adjust the display effect of the image output by the surgical imaging device displayed by the head mounted display 21 when the monitoring unit monitors that the movement of the head of the wearer relative to a standard position reaches a predetermined value. The standard position is a comfortable position when the wearer operates, and can be set according to the specific situation of the wearer.
Wherein the monitoring unit may be provided in a position to monitor the movement of the wearer's head. Preferably, the monitoring unit may be mounted to the head mounted display 21 to reflect the movement of the wearer's head by monitoring the movement of the head mounted display 21. Thus, on the one hand, the movement of the head of the wearer can be reflected by monitoring the movement of the head mounted display 21, and on the other hand, the monitoring unit can be integrated with the head mounted display 21, simplifying the structure and setup.
It will be appreciated that when the wearer needs to perform other operations with a line of sight offset from the operating field, the wearer will typically rotate (either horizontally or vertically) about the wearer's head. To monitor such movement, the monitoring unit may be used to monitor at least one of a movement distance, a movement speed, a rotation angle, and a rotation speed of the head-mounted display 21. Wherein, the moving distance refers to the distance between the positions where the wearer stays for the preset time twice, and the moving speed refers to the linear speed of the head movement of the wearer. As described above, when the wearer rotates around the head as a center when the wearer deviates from the operation region, the movement locus of the head-mounted display 21 accordingly has an approximately arc-shaped curve, and whether the movement of the head-mounted display 21 has reached the degree of deviation from the operation region can be reflected by monitoring the movement distance, the movement speed, the rotation angle, and the rotation speed of the head-mounted display 21.
Specifically, to monitor the moving distance, moving speed, rotation angle, and rotation speed of the head mounted display 21, the monitoring unit may employ various appropriate sensors. For example, the monitoring unit may comprise an accelerometer and/or a gyroscope.
In the present application, the movement of the head of the wearer reaches the predetermined value, which indicates that the line of sight of the wearer deviates from the operating area, and the image output by the surgical imaging device 10 is no longer provided to the wearer through the head-mounted display 21. Preferably, the predetermined value may include at least one of a movement distance of the head mounted display 21 with respect to a standard position reaching a first predetermined value or more, a movement speed reaching a second predetermined value or more, a rotation angle reaching a third predetermined value or more, and a rotation speed reaching a fourth predetermined value or more. It is understood that the wearer may set a combination of the first predetermined value, the second predetermined value, the third predetermined value, the fourth predetermined value and the predetermined value (for example, the predetermined value includes the first predetermined value and the second predetermined value, or the predetermined value includes the first predetermined value and the third predetermined value) \8230;, the predetermined value is set according to the movement situation and the behavior habit of the wearer.
Wherein the standard position may be set as desired (e.g., the viewing position of the surgical imaging device 10, etc.). For example, the position of the head mounted display 21 when the wearer wears the head mounted display 21 with his comfortable position looking forward at the center of the operation area may be taken as the standard position. After setting the calibration position, a predetermined value may be set, for example, when the monitoring unit monitors that the distance from the head mounted display 21 to the standard position does not reach 5cm, the moving speed does not reach 5cm/s, the angle does not reach 5 °, and the rotation speed does not reach 5 °/s, the head mounted display 21 displays the output image of the surgical imaging device 10, and when the monitoring unit monitors that at least one of the distance from the standard position from the head mounted display 21 reaches 5cm, the moving speed reaches 5cm/s, the angle reaches 5 °, and the rotation speed reaches 5 °/s, the control unit feeds back the control unit, so that the control unit may control the display effect of the head mounted display 21 based on the feedback.
As described above, when the monitoring unit detects that the movement of the head of the wearer reaches the predetermined value, it indicates that the line of sight of the wearer deviates from the operation area at this time and intends to perform other related operations, and the output image of the surgical imaging device 10 does not need to be provided to the wearer as an important point, and specifically, the display of the output image of the surgical imaging device 10 may be weakened or stopped.
According to an embodiment of the present application, the head-mounted display 21 may include a transparent display screen, and the control unit is configured to reduce the display effect of the head-mounted display 21 when the monitoring unit monitors that the movement of the head of the wearer reaches a predetermined value. Thus, when the movement of the head of the wearer relative to the standard position reaches a predetermined value, the display effect of the head mounted display 21 can be reduced, so that the wearer can see the scene right in front of the wearer through the transparent display screen for other related operations. When the wearer's head is moved again so that the line of sight returns to the operating region, the control unit may return the display effect of the head mounted display 21 to normal to provide the output image of the surgical imaging device 10 to the wearer and to weaken or avoid the view seen by the wearer through the transparent display screen.
It is understood that, in order to reduce the display effect of the head-mounted display 21 when the movement of the head of the wearer relative to the standard position reaches a predetermined value, the display of the head-mounted display 21 may be directly turned off or weakened. Moreover, in order to reduce the display effect of the head mounted display 21, the display parameters of the head mounted display 21 may be adjusted. Preferably, the control unit may control at least one of brightness, contrast, and saturation of a picture displayed by the head mounted display 21 according to feedback of the monitoring unit.
In addition, in the present application, the head-mounted display 21 may include an outer lens disposed outside the transparent display screen, and the outer lens may be configured to be capable of adjusting the transmittance according to the feedback of the monitoring unit via the control unit (e.g., the transmittance of the outer lens may be adjusted via physical shading, electrochromic lenses, photochromic lenses, etc. specifically, when the line of sight of the wearer is in the operating area, the control unit reduces the transmittance of the outer lens to make the outer lens substantially opaque so as to display the output image of the surgical imaging device 10, and when the line of sight of the wearer deviates from the operating area (i.e., the movement of the head of the wearer relative to the standard position reaches a predetermined value), the control unit increases the transmittance of the outer lens to enable the wearer to see the scene directly in front through the transparent display screen and the outer lens.
According to another embodiment of the present application, as shown in fig. 1, the head-mounted augmented reality/virtual reality display device 20 may include a camera unit 22 mounted on the head-mounted display 21 and used for capturing an image in front of the head-mounted display, and the control unit is configured to control the head-mounted display 21 to display the image captured by the camera unit 22 when the monitoring unit monitors that the movement of the head of the wearer reaches a predetermined value. Thus, when the movement of the wearer's head reaches a predetermined value, the display effect of the image output by the surgical imaging device 10 displayed by the head-mounted display 21 can be reduced by stopping displaying the output image of the surgical imaging device 10 and switching to the image captured by the imaging unit 22, so that the wearer can see the scene directly in front of the image captured by the imaging unit 22 through the head-mounted display 21 to perform other related operations. When the wearer's head moves again so that the line of sight returns to the operating area, the control unit may cause the head mounted display 21 to stop displaying the image captured by the camera unit 22 and switch the head mounted display 21 back to displaying the output image of the surgical imaging device 10.
It will be appreciated that in a microscope hand, multiple operators may be required to interact, and to this end, as shown in FIG. 1, the surgical imaging device 10 may be a surgical microscope and include a primary eyepiece 11 and an eyepiece auxiliary eyepiece 12 for viewing by the primary operator and the auxiliary operator. To conveniently provide the operator with images of the surgical imaging device, the surgical imaging system includes the head mounted augmented reality/virtual reality display device 20 for displaying a viewing image of at least one of the eyepiece main lens 11 and the eyepiece hand lens 12. That is, at least one of the primary operator and the assistant operator may view the image output by the surgical imaging device 10 through the worn augmented reality/virtual reality display device 20.
In the case where the observation image of one of the eyepiece main lens 11 and the eyepiece hand lens 12 is displayed only by the head-mounted augmented reality/virtual reality display device 20, the other can be physically used by the corresponding operator (i.e., the corresponding operator can directly observe through the other), and the different operators can keep an appropriate distance from each other without mutual operation being affected.
It is understood that since the observation images of the eyepiece main lens 11 and the eyepiece hand lens 12 are substantially different angles of the same image (for example, the observation image of the eyepiece hand lens 12 is an image in which the observation image of the eyepiece main lens 11 is rotated by 90 ° or 180 °), in the case where the observation image of one of the eyepiece main lens 11 and the eyepiece hand lens 12 is displayed only by the head-mounted augmented reality/virtual reality display device 20, it is also possible to equip both the operators using the eyepiece main lens 11 and the eyepiece hand lens 12 with the head-mounted augmented reality/virtual reality display device 20, but the output images of the surgical imaging device 10 displayed by the two head-mounted augmented reality/virtual reality display devices 20 have a relationship of the rotation angle. Therefore, even if the observation image of one of the eyepiece main lens 11 and the eyepiece hand lens 12 is displayed only by the head-mounted augmented reality/virtual reality display device 20, it is possible to provide the two relevant operators with the required operation image.
Preferably, the surgical imaging system includes two head-mounted augmented reality/virtual reality display devices 20 for displaying the observation images of the eyepiece main lens 11 and the eyepiece hand lens 12, respectively, the main operator and the assistant operator can wear the two head-mounted augmented reality/virtual reality display devices 20, respectively, and each head-mounted augmented reality/virtual reality display device 20 can provide a corresponding display effect according to the head movement of the corresponding wearer.
When the surgical imaging device 10 and the head mounted display 21 are not in use, the head mounted display 21 may be placed at a predetermined position (e.g., in the same direction as the main eyepiece 11 of the surgical imaging device 10). When setting the standard position, the wearer is allowed to adjust his/her position to his/her comfortable position (the comfortable position is the set standard position) after wearing the head-mounted display 21, the output image of the surgical imaging device 10 is always displayed during the adjustment, and meanwhile, the monitoring unit monitors the position change (for example, angle change) between the end position and the initial position of the adjustment, and then the control unit may record the position change as a preset option for setting the standard position for the wearer. If the surgical imaging device 10 is disposed separately from the head-mounted display 21, the head-mounted display 21 may be disposed at a predetermined position before the surgical imaging device 10 is used for acquiring the above data. That is, different standard positions and different preset options may be set for different wearers.
According to another aspect of the present application, a display method of an imaging operation is provided, wherein the display method includes displaying an image output by an imaging operation apparatus 10 using a head mounted display 21 of a head mounted augmented reality/virtual reality display apparatus 20, and the display method further includes monitoring a movement of a head of a wearer and controlling a display effect of the image output by the imaging operation apparatus 10 displayed by the head mounted display 21 according to the movement.
With the display method of the imaging operation, the image observed by the operation imaging device 10 can be displayed for the wearer through the head-mounted enhanced display device 20, and the display effect of the head-mounted display 21 is correspondingly controlled when the head movement of the wearer is monitored, so that the wearer can conveniently perform other operations when the line of sight deviates from the operation area.
Specifically, the surgical imaging system may be set such that the head mounted enhanced display device 20 displays the image output by the surgical imaging device 10 when the wearer is in the operation area constituted by the position range comfortable for the wearer, so that the wearer can perform the surgical operation conveniently. When the line of sight of the wearer deviates from the operation area, it indicates that the wearer needs to perform other operations (for example, taking and placing an instrument), and at this time, the wearer does not need to pay attention to the image output by the surgical imaging device 10, and the display effect of the head-mounted display 21 may be adjusted accordingly according to the monitored movement of the head of the wearer, for example, the display effect may be weakened or the display of the image output by the surgical imaging device 10 may be stopped.
To ensure that the image output by the surgical imaging device 10 is always obtained by the head mounted display 21 when the line of sight of the wearer stays within the range of the operating area, and to avoid the mis-adjustment of the head mounted display 21 by the wearer due to a slight head movement, preferably, the display method comprises adjusting the display effect of the image output by the surgical imaging device 10 displayed by the head mounted display 21 when the movement of the head of the wearer relative to a standard position is monitored to reach a predetermined value. Wherein, the standard position is a comfortable position when the wearer operates, and can be set according to the specific situation of the wearer
Wherein the monitoring of the movement of the head of the wearer may be carried out in a suitable manner, for example by a monitoring unit. The monitoring unit may be provided in a suitable position to monitor the movement of the wearer's head. Preferably, the monitoring unit may be mounted to the head mounted display 21 to reflect the movement of the wearer's head by monitoring the movement of the head mounted display 21. Thus, on the one hand, the movement of the head of the wearer can be reflected by monitoring the movement of the head mounted display 21, and on the other hand, the monitoring unit can be integrated with the head mounted display 21, simplifying the structure and setup.
It will be appreciated that when the wearer needs to perform other operations with a line of sight offset from the operating field, the wearer will typically rotate (either horizontally or vertically) about the wearer's head. To monitor such movement, at least one of a movement distance, a movement speed, a rotation angle, and a rotation speed of the head mounted display 21 may be monitored. Wherein, the moving distance refers to the distance between the positions where the wearer stays for the preset time twice, and the moving speed refers to the linear speed of the head movement of the wearer. As described above, when the wearer rotates around the head when the wearer deviates from the operation area, the movement track of the head mounted display 21 accordingly forms a curve that approximates to an arc, and the movement distance, the movement speed, the rotation angle, and the rotation speed of the head mounted display 21 are monitored, so that whether the movement of the head mounted display 21 deviates from the operation area can be reflected.
Specifically, to monitor the moving distance, moving speed, rotation angle, and rotation speed of the head mounted display 21, various appropriate sensors may be used. For example, the monitoring unit may comprise an accelerometer and/or a gyroscope.
In the present application, the movement of the head of the wearer reaches a predetermined value, which indicates that the line of sight of the wearer deviates from the operating area, and the image output by the surgical imaging device 10 does not need to be provided to the wearer in a significant manner through the head-mounted display 21. Preferably, the predetermined value may include at least one of a movement distance of the head mounted display 21 with respect to a standard position reaching a first predetermined value or more, a movement speed reaching a second predetermined value or more, a rotation angle reaching a third predetermined value or more, and a rotation speed reaching a fourth predetermined value or more. It is understood that the wearer may set a combination of the first predetermined value, the second predetermined value, the third predetermined value, the fourth predetermined value and the predetermined value (for example, the predetermined value includes the first predetermined value and the second predetermined value, or the predetermined value includes the first predetermined value and the third predetermined value) \8230;, the predetermined value is set according to the movement situation and the behavior habit of the wearer.
Wherein the standard position may be set as desired (e.g., the viewing position of the surgical imaging device 10, etc.). For example, the position of the head mounted display 21 when the wearer wears the head mounted display 21 looking straight at the center of the operation area may be set as the standard position. After setting the calibration position, a predetermined value may be set to display an output image of the surgical imaging device 10 through the head mounted display 21 when, for example, it is monitored that the distance from the standard position to which the head mounted display 21 moves has not reached 5cm, the moving speed has not reached 5cm/s, the angle has not reached 5 °, and the rotation speed has not reached 5 °/s, and the display effect of the head mounted display 21 may be controlled based on the feedback when it is monitored that at least one of the distance from the standard position to which the head mounted display 21 moves has reached 5cm, the moving speed has reached 5cm/s, the angle has reached 5 °, and the rotation speed has reached 5 °/s.
As described above, when it is monitored that the movement of the head of the wearer relative to the standard position reaches the predetermined value, it indicates that the wearer is now out of sight of the operation area and intends to perform other related operations, and the output image of the surgical imaging device 10 is not required to be provided to the wearer as an emphasis, and in particular, the output image of the surgical imaging device 10 may be weakened or stopped from being displayed.
According to an embodiment of the present application, the head-mounted display 21 may comprise a transparent display screen, and the display method comprises reducing the display effect of the head-mounted display 21 when the movement of the head of the wearer relative to a standard position is monitored to reach a predetermined value. Thus, when the movement of the head of the wearer reaches a predetermined value, the display effect of the head-mounted display 21 can be reduced, so that the wearer can see the scene right in front of the wearer through the transparent display screen for other related operations. When the wearer's head is moved again so that the line of sight returns to the operating region, the display effect of the head mounted display 21 may be adjusted back to normal to provide the wearer with the output image of the surgical imaging device 10 and to weaken or avoid the view of the wearer looking through the transparent display screen.
It is understood that, in order to reduce the display effect of the head-mounted display 21 when the movement of the head of the wearer relative to the standard position reaches a predetermined value, the display of the head-mounted display 21 may be directly turned off or weakened. Also, to reduce the display effect of the head mounted display 21, the display parameters of the head mounted display 21 may be adjusted. Preferably, the display method comprises adjusting at least one of brightness, contrast and saturation of a picture displayed by the head-mounted display 21 when the movement of the head of the wearer is monitored to reach a predetermined value.
In addition, in the present application, the head-mounted display 21 may include an outer lens disposed outside the transparent display screen, and the outer lens may be configured to adjust the transmittance according to the feedback of the monitoring unit by the control unit (for example, the transmittance of the outer lens may be adjusted by physical shading, electrochromic lens, photochromic lens, etc.), so that the display method may control the transmittance according to the monitoring condition. Specifically, when the wearer's line of sight is in the operating area, the outer lens is made substantially opaque by decreasing the optical transparency of the outer lens so as to display the output image of the surgical imaging device 10; when the wearer's line of sight deviates from the operating zone (i.e. the movement of the wearer's head relative to the standard position reaches a predetermined value), the transparency of the outer lens is increased to allow the wearer to see the scene directly in front through the transparent display and the outer lens.
According to another embodiment of the present application, as shown in fig. 1, the head-mounted augmented reality/virtual reality display device 20 may include a camera unit 22 mounted to the head-mounted display 21 and configured to capture an image in front of the head-mounted display, and the display method includes controlling the head-mounted display 21 to display the image captured by the camera unit 22 when it is monitored that the movement of the head of the wearer reaches a predetermined value. Thus, when the movement of the head of the wearer reaches a predetermined value, the display effect of the image output by the surgical imaging device 10 displayed by the head mounted display 21 can be reduced by stopping displaying the output image of the surgical imaging device 10 and switching to the image captured by the imaging unit 22, so that the wearer can see the scene directly in front of the imaging unit 22 captured by the imaging unit 22 through the head mounted display 21 to perform other related operations. When the wearer's head moves again so that the line of sight returns to the operating area, the head mounted display 21 may be caused to stop displaying the image captured by the camera unit 22 and the head mounted display 21 switched back to displaying the output image of the surgical imaging device 10.
It will be appreciated that in microscopic surgery, multiple operators may be required to interact, and to this end, as shown in FIG. 1, the surgical imaging device 10 may be a surgical microscope and include a primary eyepiece 11 and an eyepiece hand lens 12 for viewing by the primary operator and the assistant operator. To conveniently provide the operator with an image of the surgical imaging device, the display method includes displaying a viewing image of at least one of the main eyepiece 11 and the eyepiece power eyepiece 12 through the head mounted augmented reality/virtual reality display device 20. That is, at least one of the primary operator and the assistant operator may observe the image output by the surgical imaging device 10 through the worn augmented reality/virtual reality display device 20.
In the case where the observation image of one of the eyepiece main lens 11 and the eyepiece hand lens 12 is displayed only by the wearable augmented reality/virtual reality display device 20, the other can be physically used by the corresponding operator (i.e., the corresponding operator can directly observe through the other), and the different operators can keep an appropriate distance from each other without mutual operation being affected.
It is understood that since the observation images of the eyepiece main lens 11 and the eyepiece hand lens 12 are substantially different angles of the same image (for example, the observation image of the eyepiece hand lens 12 is an image in which the observation image of the eyepiece main lens 11 is rotated by 90 ° or 180 °), in the case where the observation image of one of the eyepiece main lens 11 and the eyepiece hand lens 12 is displayed only by the head-mounted augmented reality/virtual reality display device 20, it is also possible to equip both the operators using the eyepiece main lens 11 and the eyepiece hand lens 12 with the head-mounted augmented reality/virtual reality display device 20, but the output images of the surgical imaging device 10 displayed by the two head-mounted augmented reality/virtual reality display devices 20 have a relationship of the rotation angle. Therefore, even if the observation image of one of the eyepiece main lens 11 and the eyepiece hand lens 12 is displayed only by the head-mounted augmented reality/virtual reality display device 20, it is possible to provide the two concerned operators with a desired operation image.
Preferably, the display method includes displaying the observation images of the eyepiece main lens 11 and the eyepiece auxiliary lens 12 through two head-mounted augmented reality/virtual reality display devices 20, respectively, and the main operator and the assistant operator may wear the two head-mounted augmented reality/virtual reality display devices 20, respectively, and each head-mounted augmented reality/virtual reality display device 20 may provide a corresponding display effect according to the corresponding head movement of the wearer.
When the surgical imaging device 10 and the head mounted display 21 are not in use, the head mounted display 21 may be placed at a predetermined position (e.g., in the same direction as the main eyepiece 11 of the surgical imaging device 10). When setting the standard position, the wearer is allowed to adjust his/her position to his/her comfortable position (the comfortable position is the set standard position) after wearing the head-mounted display 21, the output image of the surgical imaging device 10 is always displayed during the adjustment, and meanwhile, the monitoring unit monitors the position change (for example, angle change) between the end position and the initial position of the adjustment, and then the control unit may record the position change as a preset option for setting the standard position for the wearer. If the surgical imaging device 10 is to be placed separately from the head mounted display 21, the head mounted display 21 may also be placed in a predetermined position prior to use of the surgical imaging device 10 for obtaining the data. That is, different standard positions and different preset options may be set for different wearers.
The system and the method can be used for imaging operation and operation training, and have the following advantages:
1. physical connection between the objective lens of the operation microscope and the eyepiece main lens and the eyepiece assistant lens is eliminated, joint movement of the eyepiece lens when the angle of the objective lens is adjusted is avoided, and an operator can select a station according to needs;
2. the image output by the surgical imaging device 10 is displayed by the head-mounted augmented reality/virtual reality display device 20, and the operator does not need to fix the sight line on the optical path of the ocular lens of the surgical microscope when acquiring the image information;
3. when the position and the operation direction of an operator are changed, the direction of an ocular of the operating microscope does not need to be adjusted according to the operation direction and the position, and the operating microscope does not need to be adjusted and re-leveled according to the direction of the ocular, so that the operation steps are simplified;
4. the head-mounted augmented reality/virtual reality display device 20 adopts an augmented reality display technology or a virtual reality display technology, can output 3D image information with depth-of-field information and can adjust an image (for example, an enlarged image) as required to provide image information with better quality for an operator;
5. an additional external display device is not required to be arranged, a display can be omitted from the endoscope system, and instruments and spatial arrangement in an operation area are simplified;
6. the visual image provided for the wearer can be controlled according to the head movement of the wearer, manual adjustment of the wearer is not needed, and pollution and operation risks caused by the manual adjustment are prevented;
7. because manual contact is not required, the head-mounted augmented reality/virtual reality display device 20 may not be sterile, reducing the need for sterilization thereof.
The preferred embodiments of the present application have been described above in detail, however, the present application is not limited to the details of the above embodiments, and various simple modifications may be made to the technical solution of the present application within the technical idea of the present application, and these simple modifications all belong to the protection scope of the present application.
It should be noted that, in the foregoing embodiments, various features described in the above embodiments may be combined in any suitable manner, and in order to avoid unnecessary repetition, various possible combinations are not described in the present application.
In addition, any combination of the various embodiments of the present application can be made, and the same should be considered as the disclosure of the present invention as long as the combination does not depart from the spirit of the present application.
Claims (16)
1. A surgical imaging system, comprising a surgical imaging device (10) and a head-mounted augmented reality/virtual reality display device (20), wherein the surgical imaging device (10) is configured to output an observed image, the head-mounted augmented reality/virtual reality display device (20) comprises a head-mounted display (21) capable of receiving and displaying the image output by the microscope, a control unit and a monitoring unit for monitoring the movement of the head of a wearer, the control unit is electrically connected with the monitoring unit so as to control the display effect of the image output by the surgical imaging device displayed by the head-mounted display (21) according to the feedback of the monitoring unit.
2. A surgical imaging system according to claim 1, wherein the control unit is arranged to adjust the display of the image output by the surgical imaging device displayed by the head mounted display (21) when the monitoring unit monitors that the movement of the wearer's head relative to a standard position reaches a predetermined value.
3. A surgical imaging system according to claim 2, wherein the head mounted display (21) comprises a transparent display screen, the control unit being arranged to reduce the display effect of the head mounted display (21) when the monitoring unit monitors that the movement of the wearer's head relative to a standard position reaches a predetermined value.
4. The surgical imaging system according to claim 2, wherein the control unit controls at least one of brightness, contrast, and saturation of a picture displayed by the head mounted display (21) according to feedback of the monitoring unit.
5. A surgical imaging system according to claim 2, wherein the head mounted augmented reality/virtual reality display device (20) comprises a camera unit (22) mounted to the head mounted display (21) and adapted to take images in front of the head mounted display, the control unit being arranged to control the head mounted display (21) to display images taken by the camera unit (22) when the monitoring unit monitors that the movement of the wearer's head relative to a standard position reaches a predetermined value.
6. The surgical imaging system of claim 2, wherein the surgical imaging device (10) is a surgical microscope and includes a primary eyepiece (11) and a secondary eyepiece (12), the surgical imaging system including the head mounted augmented reality/virtual reality display device (20) for displaying a viewing image of at least one of the primary eyepiece (11) and the secondary eyepiece (12).
7. Surgical imaging system according to any of claims 2-6, characterized in that the monitoring unit is mounted to the head mounted display (21) to reflect the movement of the wearer's head by monitoring the movement of the head mounted display (21).
8. Surgical imaging system according to claim 7, wherein the monitoring unit is configured to monitor at least one of a movement distance, a movement speed, a rotation angle, a rotation speed of the head mounted display (21).
9. The surgical imaging system of claim 8, wherein the monitoring unit includes an accelerometer and/or a gyroscope.
10. The surgical imaging system according to claim 7, wherein the predetermined value is at least one of a movement distance of the head mounted display (21) relative to a standard position reaching a first predetermined value or more, a movement speed reaching a second predetermined value or more, a rotation angle reaching a third predetermined value or more, and a rotation speed reaching a fourth predetermined value or more.
11. A display method of an imaging procedure, the display method comprising displaying an image output by a surgical imaging device (10) using a head mounted display (21) of a head mounted augmented reality/virtual reality display device (20), the display method further comprising monitoring movement of a wearer's head and controlling display effects of the image output (10) by the surgical imaging device displayed by the head mounted display (21) in accordance with the movement.
12. The display method according to claim 11, characterized in that it comprises adjusting the display effect of the image output by the surgical imaging device (10) displayed by the head-mounted display (21) when the movement of the wearer's head with respect to a standard position is monitored to reach a predetermined value.
13. A display method as claimed in claim 12, characterised in that the head mounted display (21) comprises a transparent display screen, the display method comprising reducing the display effect of the head mounted display (21) when movement of the wearer's head relative to a standard position is monitored to reach a predetermined value.
14. The display method according to claim 13, characterized in that it comprises adjusting at least one of brightness, contrast, saturation of a picture displayed by the head-mounted display (21) when it is monitored that the movement of the wearer's head with respect to a standard position reaches a predetermined value.
15. The display method according to claim 12, wherein the head mounted augmented reality/virtual reality display device (20) comprises a camera unit (22) mounted to the head mounted display (21) and adapted to take images in front of the head mounted display, the display method comprising controlling the head mounted display (21) to display images taken by the camera unit (22) when it is monitored that the movement of the wearer's head relative to a standard position reaches a predetermined value.
16. The display method according to any one of claims 12 to 15, wherein the predetermined value includes at least one of a movement distance of the head-mounted display (21) with respect to a standard position reaching a first predetermined value or more, a movement speed reaching a second predetermined value or more, a rotation angle reaching a third predetermined value or more, and a rotation speed reaching a fourth predetermined value or more.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211384038.4A CN115737143A (en) | 2022-11-07 | 2022-11-07 | Surgical imaging system and method for displaying imaging surgery |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211384038.4A CN115737143A (en) | 2022-11-07 | 2022-11-07 | Surgical imaging system and method for displaying imaging surgery |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115737143A true CN115737143A (en) | 2023-03-07 |
Family
ID=85356879
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211384038.4A Pending CN115737143A (en) | 2022-11-07 | 2022-11-07 | Surgical imaging system and method for displaying imaging surgery |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115737143A (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE102014018056A1 (en) * | 2014-12-05 | 2016-06-09 | Audi Ag | Method of operating a virtual reality glasses and virtual reality glasses |
| CN106970697A (en) * | 2016-01-13 | 2017-07-21 | 华为技术有限公司 | Interface alternation device and method |
| CN108652824A (en) * | 2018-05-18 | 2018-10-16 | 深圳市莫廷影像技术有限公司 | Ophthalmic operating microscope system |
| CN109478346A (en) * | 2016-08-04 | 2019-03-15 | 诺华股份有限公司 | It is experienced using the ophthalmologic operation that virtual reality head-mounted display enhances |
| CN110494095A (en) * | 2017-04-20 | 2019-11-22 | 直观外科手术操作公司 | System and method for constraining virtual reality surgery systems |
| CN111405866A (en) * | 2017-11-30 | 2020-07-10 | 医学技术有限责任公司 | Immersive display system for eye treatment |
| CN113544765A (en) * | 2019-03-12 | 2021-10-22 | 索尼集团公司 | Information processing apparatus, information processing method, and program |
| CN215839550U (en) * | 2021-09-16 | 2022-02-18 | 深圳市数泽科技有限公司 | Surgical microscope system for displaying 3D images |
-
2022
- 2022-11-07 CN CN202211384038.4A patent/CN115737143A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE102014018056A1 (en) * | 2014-12-05 | 2016-06-09 | Audi Ag | Method of operating a virtual reality glasses and virtual reality glasses |
| CN106970697A (en) * | 2016-01-13 | 2017-07-21 | 华为技术有限公司 | Interface alternation device and method |
| CN109478346A (en) * | 2016-08-04 | 2019-03-15 | 诺华股份有限公司 | It is experienced using the ophthalmologic operation that virtual reality head-mounted display enhances |
| CN110494095A (en) * | 2017-04-20 | 2019-11-22 | 直观外科手术操作公司 | System and method for constraining virtual reality surgery systems |
| CN111405866A (en) * | 2017-11-30 | 2020-07-10 | 医学技术有限责任公司 | Immersive display system for eye treatment |
| CN108652824A (en) * | 2018-05-18 | 2018-10-16 | 深圳市莫廷影像技术有限公司 | Ophthalmic operating microscope system |
| CN113544765A (en) * | 2019-03-12 | 2021-10-22 | 索尼集团公司 | Information processing apparatus, information processing method, and program |
| CN215839550U (en) * | 2021-09-16 | 2022-02-18 | 深圳市数泽科技有限公司 | Surgical microscope system for displaying 3D images |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3204816B1 (en) | Head-mounted displaying of magnified images locked on an object of interest | |
| ES2899353T3 (en) | Digital system for capturing and visualizing surgical video | |
| US10874284B2 (en) | Display control device, display device, surgical endoscopic system and display control system | |
| JP6521982B2 (en) | Surgical visualization system and display | |
| TWI534476B (en) | Head-mounted display | |
| US10073515B2 (en) | Surgical navigation system and method | |
| US7768702B2 (en) | Medical stereo observation system | |
| JP2011205358A (en) | Head-mounted display device | |
| JP2017509925A (en) | 3D video microscope equipment | |
| EP3682790A1 (en) | Endoscope system | |
| JP2004320722A (en) | Stereoscopic observation system | |
| CN115087413A (en) | Method for operating a surgical microscope and surgical microscope | |
| CN115737143A (en) | Surgical imaging system and method for displaying imaging surgery | |
| US11150481B2 (en) | Reality viewer | |
| JP2015007722A (en) | Image display device | |
| JP7453403B2 (en) | head mounted visualization system | |
| US20200306003A1 (en) | Medical control apparatus and medical observation system | |
| JPH10174673A (en) | Endoscope apparatus | |
| US20220313085A1 (en) | Surgery 3D Visualization Apparatus | |
| US11504001B2 (en) | Surgery 3D visualization apparatus | |
| JP2018061850A (en) | Endoscope system, display control system, and display control device | |
| JPH07287187A (en) | Video display device | |
| JP2019101293A (en) | Head-mounted type display device, display system and display method | |
| JP2021027419A (en) | Imaging apparatus for forehead band and image control method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |