WO2022163362A1 - Dispositif de traitement d'image, procédé de traitement d'image et système de microscope chirurgical - Google Patents

Dispositif de traitement d'image, procédé de traitement d'image et système de microscope chirurgical Download PDF

Info

Publication number
WO2022163362A1
WO2022163362A1 PCT/JP2022/000855 JP2022000855W WO2022163362A1 WO 2022163362 A1 WO2022163362 A1 WO 2022163362A1 JP 2022000855 W JP2022000855 W JP 2022000855W WO 2022163362 A1 WO2022163362 A1 WO 2022163362A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
lens
stereo
display
display image
Prior art date
Application number
PCT/JP2022/000855
Other languages
English (en)
Japanese (ja)
Inventor
知之 大月
Original Assignee
ソニーグループ株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ソニーグループ株式会社 filed Critical ソニーグループ株式会社
Publication of WO2022163362A1 publication Critical patent/WO2022163362A1/fr

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
    • A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
    • A61B3/13—Ophthalmic microscopes
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61F—FILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, e.g. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS
    • A61F9/00—Methods or devices for treatment of the eyes; Devices for putting-in contact lenses; Devices to correct squinting; Apparatus to guide the blind; Protective devices for the eyes, carried on the body or in the hand
    • A61F9/007—Methods or devices for eye surgery

Definitions

  • the present disclosure relates to an image processing device, an image processing method, and a surgical microscope system.
  • intraocular lens As a method of refractive correction in ophthalmology, by inserting an artificial lens called an intraocular lens (IOL) into the eye, it is widely used to eliminate the refractive error of the crystalline lens and improve visual functions such as visual acuity. It is done.
  • the most widely used intraocular lens is an intraocular lens that is inserted into the lens capsule as a replacement for the lens removed by cataract surgery.
  • various intraocular lenses such as those that are fixed (dwelled) in the ciliary sulcus (Phakic IOL).
  • Patent Literature 1 a tomographic image of the eye is used to facilitate understanding of the positional relationship between the surgical tool and the treatment target portion of the eye.
  • OCT optical coherence tomography
  • an image processing apparatus and an image processing apparatus that are capable of simplifying the apparatus configuration and reducing the size of the apparatus while making it easier for the operator to understand the positional relationship between the surgical tool and the region to be treated of the eye.
  • a method and an operating microscope system are proposed.
  • An image processing apparatus includes an image input unit that receives an operative field image of a patient's eye, a three-dimensional model reception unit that receives a three-dimensional model of part or all of the lens of the eye, A stereo image generator that generates a stereo image of the crystalline lens from the three-dimensional model, and a display image generator that generates a display image including the surgical field image and the stereo image based on the surgical field image and the stereo image. And prepare.
  • an image processing device receives an operative field image of a patient's eye, receives a three-dimensional model of part or all of the lens of the eye, and from the three-dimensional model generating a stereo image of the lens; and generating a display image including the surgical field image and the stereo image based on the surgical field image and the stereo image.
  • a surgical microscope system includes a surgical microscope that obtains an surgical field image of a patient's eye, an image processing device that generates a display image, and a display device that displays the display image, wherein the image is
  • the processing device includes an image input unit that receives the operative field image, a three-dimensional model reception unit that receives a three-dimensional model of part or all of the lens of the eye, and a stereo image of the lens from the three-dimensional model. and a display image generator for generating the display image including the surgical field image and the stereo image based on the surgical field image and the stereo image.
  • FIG. 1 is a diagram showing an example of a schematic configuration of a surgical microscope system according to an embodiment of the present disclosure
  • FIG. 1 is a diagram showing an example of a schematic configuration of a surgical microscope according to an embodiment of the present disclosure
  • FIG. 1 is a diagram illustrating an example of a schematic configuration of an image processing device according to an embodiment of the present disclosure
  • FIG. 4 is a diagram showing an example of a tomographic image for 3D model generation according to an embodiment of the present disclosure
  • FIG. 3 is a diagram illustrating an example of a stereoscopic image for 3D model generation according to an embodiment of the present disclosure
  • FIG. 3 illustrates an example stereo image corresponding to a 3D model according to an embodiment of the present disclosure
  • FIG. 1 is a diagram showing an example of a schematic configuration of a surgical microscope system according to an embodiment of the present disclosure
  • FIG. 1 is a diagram showing an example of a schematic configuration of a surgical microscope according to an embodiment of the present disclosure
  • FIG. 1 is a diagram
  • FIG. 3 is a diagram showing Example 1 of a display image according to the embodiment of the present disclosure
  • FIG. FIG. 10 is a diagram showing Example 2 of a display image according to the embodiment of the present disclosure
  • FIG. 10 is a diagram showing Example 3 of a display image according to the embodiment of the present disclosure
  • FIG. 10 is a diagram showing Example 4 of a display image according to the embodiment of the present disclosure
  • FIG. 11 is a diagram showing Example 5 of a display image according to an embodiment of the present disclosure
  • 1 is a diagram illustrating an example of a schematic configuration of a computer according to an embodiment of the present disclosure
  • Embodiment 1-1 Example of schematic configuration of surgical microscope system 1-2.
  • Example of schematic configuration of surgical microscope 1-3 Schematic Configuration of Image Processing Apparatus and Example of Image Processing 1-4. Action and effect 2.
  • FIG. 1 is a diagram showing an example of a schematic configuration of a surgical microscope system 1 according to this embodiment.
  • the surgical microscope system 1 has a surgical microscope 10 and a patient bed 20.
  • This surgical microscope system 1 is a system used for eye surgery. The patient undergoes eye surgery while lying on the patient bed 20 . An operator, who is a doctor, performs surgery while observing the patient's eye through the surgical microscope 10 .
  • the surgical microscope 10 has an objective lens 11, an eyepiece lens 12, an image processing device 13, and a monitor 14.
  • the objective lens 11 and the eyepiece lens 12 are lenses for magnifying and observing the eye of the patient to be operated.
  • the image processing device 13 outputs various images, various information, etc. by performing predetermined image processing on the image captured through the objective lens 11 .
  • the monitor 14 displays an image captured through the objective lens 11, various images generated by the image processing device 13, various information, and the like. This monitor 14 may be provided separately from the surgical microscope 10 .
  • the operator looks into the eyepiece 12 and performs surgery while observing the patient's eye through the objective lens 11. Further, the operator performs surgery while confirming various images (for example, an image before image processing, an image after image processing, etc.) and various information displayed on the monitor 14 .
  • various images for example, an image before image processing, an image after image processing, etc.
  • FIG. 2 is a diagram showing an example of a schematic configuration of the surgical microscope 10 according to this embodiment.
  • the surgical microscope 10 includes, in addition to the objective lens 11, the eyepiece lens 12, the image processing device 13, and the monitor 14, a light source 51, an observation optical system 52, a front image photographing section 53, and a display It has a section 54 , an interface section 55 and a speaker 56 .
  • the monitor 14 and the presentation unit 54 correspond to a display device.
  • the light source 51 emits illumination light under the control of the control unit 13A included in the image processing device 13 to illuminate the eyes of the patient.
  • the observation optical system 52 is composed of optical elements such as the objective lens 11, a half mirror 52a, and lenses (not shown).
  • the observation optical system 52 guides the light (observation light) reflected from the patient's eye to the eyepiece 12 and the front image capturing section 53 .
  • the light reflected from the patient's eye enters the half mirror 52a as observation light via the objective lens 11, a lens (not shown), or the like.
  • Approximately half of the observation light incident on the half mirror 52 a passes through the half mirror 52 a as it is, and enters the eyepiece 12 via the transmissive presentation unit 54 .
  • the other half of the observation light incident on the half mirror 52 a is reflected by the half mirror 52 a and enters the front image capturing section 53 .
  • the front image capturing unit 53 is composed of, for example, a video camera.
  • the front image photographing unit 53 receives the observation light incident from the observation optical system 52 and photoelectrically converts it to obtain an image of the patient's eye observed from the front, that is, an image of the patient's eye photographed approximately in the eye axis direction. A front image is taken.
  • the front image capturing unit 53 captures (captures) a front image under the control of the image processing device 13 and supplies the obtained front image to the image processing device 13 .
  • the eyepiece 12 collects the observation light incident from the observation optical system 52 via the presentation unit 54 and forms an optical image of the patient's eye. An optical image of the patient's eye is thereby observed by the operator looking through the eyepiece 12 .
  • the presentation unit 54 is composed of a transmissive display device or the like, and is arranged between the eyepiece 12 and the observation optical system 52 .
  • the presentation unit 54 transmits observation light incident from the observation optical system 52 and makes it enter the eyepiece 12, and also displays various images (for example, a front image, a stereo image, etc.) and various information supplied from the image processing device 13. are also presented (displayed) as necessary.
  • various images, various information, and the like may be presented, for example, superimposed on the optical image of the patient's eye, or may be presented in the periphery of the optical image so as not to interfere with the optical image.
  • the image processing device 13 has a control section 13A that controls the operation of the surgical microscope 10 as a whole.
  • the control section 13A changes the illumination conditions of the light source 51 or changes the zoom magnification of the observation optical system 52 .
  • the control unit 13A controls image acquisition by the front image photographing unit 53 based on the operation information of the operator or the like supplied from the interface unit 55 and the like.
  • the interface unit 55 is composed of, for example, a communication unit and the like.
  • the communication unit receives commands from an operation unit such as a touch panel superimposed on the monitor 14, a controller, a remote controller (not shown), or the like, and communicates with external devices.
  • the interface unit 55 supplies the image processing apparatus 13 with information and the like according to the operation of the operator.
  • the interface unit 55 also outputs device control information for controlling the external device, which is supplied from the image processing apparatus 13, to the external device.
  • the monitor 14 displays various images such as front images and stereo images and various information according to control by the control unit 13A of the image processing device 13 .
  • the speaker 56 when a dangerous situation is detected during surgery, the speaker 56 emits a buzzer sound, a melody sound, or the like in order to notify the operator of the dangerous situation in accordance with the control by the control unit 13A of the image processing device 13. Outputs sound, message (voice), and the like.
  • the surgical microscope 10 may be provided with a rotating light or indicator light (lamp) for informing the operator or the like of a dangerous situation.
  • a display image is generated by the image processing device 13 based on various images such as a front image and a stereo image, and the generated display image is displayed on the monitor 14, the presentation unit 54, or the like. Displaying by the device allows the operator to grasp the positional relationship between the surgical instrument and the surgical target site of the eye.
  • FIG. 3 is a diagram showing an example of a schematic configuration (configuration and processing flow) of the image processing apparatus 13 according to this embodiment.
  • the treatment target site is the lens capsule
  • an inexperienced operator does not know the relationship between the surgical tool and the depth of the posterior capsule, and the surgical tool may damage the posterior capsule.
  • stereo image presentation processing based on a 3D model obtained by preoperatively measuring the lens (which may include the lens capsule) will be described.
  • the image processing device 13 includes a 3D model reception unit (three-dimensional model reception unit) 13a, a stereo image generation unit 13b, an image input unit 13c, and a display image generation unit 13d.
  • a 3D model reception unit three-dimensional model reception unit
  • a stereo image generation unit 13b stereo image generation unit
  • an image input unit 13c image input unit
  • a display image generation unit 13d display image generation unit
  • the 3D model receiving unit 13a receives from an external device a 3D model of the treatment target site measured preoperatively in the preoperative planning.
  • the 3D model may include only the site to be treated, or may include other sites for size adjustment and posture adjustment, which will be described later.
  • the 3D model may include the corneal limbus for size matching based on the cornea in the surgical field image, or may include the iris for posture matching using the iris pattern.
  • the 3D model may also include a preoperative frontal image of the target eye whose positional relationship with the 3D model is known for use in alignment. For example, blood vessels around the cornea in the preoperative front image can be used for posture adjustment.
  • FIG. 4 is a diagram showing an example of a tomographic image for 3D model generation according to this embodiment.
  • FIG. 5 is a diagram showing an example of a stereoscopic image (a stereoscopic image viewed from a specific direction) for generating a 3D model according to this embodiment.
  • a tomogram multiple tomograms
  • a stereoscopic image are used to construct a 3D model. This 3D model is supplied to the stereo image generator 13b.
  • the 3D model is obtained by OCT (Optical Coherence Tomography), so the 3D model of the lens behind the iris is missing.
  • OCT Optical Coherence Tomography
  • the 3D model of the lens behind the iris is missing.
  • OCT Optical Coherence Tomography
  • the following description is based on the premise that a 3D model of the lens without chipping has been obtained.
  • OCT irradiates the eye to be treated with light such as near-infrared light, reconstructs the reflected waves from each tissue of the eye, and creates an image (a tomographic image that is a cross-sectional image in the depth direction of the eye). It is a technology to generate.
  • images (tomographic images) captured by a Scheimpflug camera may be used for the 3D model.
  • the stereo image generating unit 13b generates two images of the crystalline lens viewed from two specific viewpoints (corresponding to the left eye and the right eye) based on the 3D model received by the 3D model receiving unit 13a. Generate a stereo image (stereogram) corresponding to the model.
  • the stereo image generator 13b supplies the stereo image of the treatment target site to the display image generator 13d.
  • FIG. 6 is a diagram showing an example of a stereo image corresponding to the 3D model according to this embodiment.
  • the stereo image shows, for example, the lens (which may include the lens capsule) by contour lines.
  • a crystalline lens corresponds to a crystalline lens alone or a crystalline lens having a lens capsule.
  • the stereo image is an image with contour lines to improve visibility, but is not limited to this, and may be an image that gives the operator a 3D feel.
  • the stereo image is an image with contour lines to improve visibility, but is not limited to this, and may be an image that gives the operator a 3D feel.
  • FIG. 6 and the examples of FIGS. 7 to 11 below only one side of the two images forming the stereo image is shown for the sake of simplification of explanation. shown.
  • the image input unit 13c receives an operating field image (front image) from the front image capturing unit 53 (see FIG. 2), and receives the operating field image (for example, an operating field image at the start of surgery or an operating field image). real-time operating field image, etc.) is supplied to the display image generation unit 13d. In addition, the image input unit 13c supplies the operative field image to the stereo image generation unit 13b as necessary.
  • an operating field image front image
  • the operating field image for example, an operating field image at the start of surgery or an operating field image.
  • the display image generation unit 13d generates a display image by adding the stereo image of the treatment target site supplied from the stereo image generation unit 13b to the surgical field image supplied from the image input unit 13c. For example, the display image generation unit 13d superimposes and synthesizes the stereo image on the surgical field image to generate a display image including the surgical field image and the stereo image.
  • FIG. 7 is a diagram showing Example 1 of a display image according to the present embodiment.
  • the stereo image is positioned on the upper right side of the operative field image (in FIG. 7) and superimposed on the operative field image to generate the display image.
  • This display image is displayed by both or one of the monitor 14 and the presentation unit 54 .
  • the operator cannot grasp the depth of the transparent posterior lens capsule from only the front image of the surgical field image received by the image input unit 13c.
  • the operator can grasp the three-dimensional shape of the crystalline lens by visually recognizing the stereo image showing the outline of the crystalline lens. This makes it easier for the operator to grasp the positional relationship between the surgical tool and the lens capsule.
  • the size, three-dimensional posture, and position of the stereo image of the treatment target site presented in the display image are not aligned with the surgical field image, but they may all be aligned.
  • the sizes, postures, and positions are aligned independently, the ease of grasping the positional relationship is improved, although it is inferior to the case where all are aligned.
  • the posture among these when the three-dimensional posture is aligned, it is particularly easy to grasp the positional relationship. It helps to accurately grasp the shape of the lens, which has anisotropic shape around the axis, and as a result, the grasp of the positional relationship is improved.
  • FIG. 8 is a diagram showing example 2 of a display image according to the present embodiment.
  • the stereo image is positioned on the upper right side of the operative field image (in FIG. 8) and superimposed on the operative field image to generate the display image.
  • a stereo image is generated so that its size and orientation are the same as those of the surgical target site (lens) in the operative field image.
  • the size of the stereo image of the treatment target area is aligned with the size of the treatment target area in the surgical field image.
  • the display image generator 13d may adjust the size of the cornea in the operating field image and the size of the cornea in the 3D model to be the same.
  • the size of the treatment target area matches the surgical field image, making it easier for the operator to guess the positional relationship between the treatment target area and the surgical tool.
  • the depth of the lens is known, so the operator can easily predict the depth relationship between the surgical tool and the posterior capsule.
  • the posture of the stereo image of the treatment target site is adjusted so that it is three-dimensionally aligned with the posture of the eye in the surgical field image.
  • the iris pattern and the blood vessel pattern in the corneal periphery in both the surgical field image and the 3D model may be used.
  • the display image generation unit 13d matches the patterns of the iris or blood vessels in both the operative field image and the 3D model, and aligns the posture of the stereo image of the treatment target region three-dimensionally with the posture of the eye in the operative field image. good too.
  • the stereo image of the treatment target area and the posture of the treatment target area in the surgical field image are aligned, so the operator can easily guess the positional relationship between the treatment target area and the surgical tool.
  • the orientation around the eye axis (the position in the rotational direction) may be adjusted.
  • the orientation around the eye axis may be adjusted so that the upper eye region (portion) in the displayed image in the stereo image of the treatment target site is the same as the surgical field image.
  • the iris pattern and the corneal peripheral blood vessel pattern in both the surgical field image and the 3D model may be used.
  • the display image generation unit 13d matches the patterns of the iris or blood vessels in both the operative field image and the 3D model, and changes the orientation of the stereo image of the treatment target region around the eye axis to the direction around the eye axis of the eye in the operative field image. You can align it.
  • the direction around the eye axis is defined, for example, by the angle in the direction of rotation around the eye axis with respect to a reference line orthogonal to the eye axis.
  • the treatment target site does not necessarily have the same shape around the eye axis. Therefore, it is effective to align the directions around the eye axis.
  • FIG. 9 is a diagram showing Example 3 of a display image according to this embodiment.
  • the stereo image is positioned on the treatment target region of the surgical field image and superimposed on the surgical field image to generate the display image.
  • a stereo image is generated so that the size, posture, and position are the same as those of the treatment target region (lens) in the surgical field image.
  • position alignment is performed so that the position of the stereo image of the treatment target area matches the position of the treatment target area in the surgical field image.
  • the iris pattern and the corneal peripheral blood vessel pattern in both the surgical field image and the 3D model may be used.
  • the display image generator 13d may match the iris or blood vessel patterns in both the surgical field image and the 3D model, and align the position of the stereo image of the treatment target site with the position of the eye in the surgical field image.
  • the stereo image is displayed superimposed on the treatment target site in the surgical field, making it easier for the operator to guess the positional relationship between the surgical tool and the treatment target site.
  • the operator can observe the actual positional relationship instead of guessing.
  • FIG. 10 is a diagram showing Example 4 of a display image according to this embodiment.
  • a part of the corresponding stereo image and a part of the treatment target part of the operative field image are enlarged and positioned at the upper left of the operative field image (in FIG. 10).
  • a display image is generated by being superimposed on the operative field image.
  • FIG. 10 shows an example of enlarged presentation of the surgical field image and the stereo image of the treatment target region, which are of the same size only.
  • two stereo images of the treatment target site are presented, but only the enlarged stereo image may be presented.
  • the same observation optical system as that for acquiring the surgical field image is used to capture the same stereo inward angle. Adjustments may be made to obtain results.
  • the three-dimensional position of the surgical tool estimated from the parallax information in the surgical field image, and the shape information of the 3D model of the treatment target site whose size, three-dimensional posture and position are aligned with the surgical field image are used to determine the surgical tool. and the treatment target site may be calculated, and information about the calculated distance and warning information about warning when the distance is short may be presented.
  • the information since the information is presented based on the measurement results rather than subjectively by the operator, the separation distance between the surgical tool and the treatment target site becomes easy to understand. Note that only the distance information may be presented without presenting the stereo image of the treatment target site. There are various methods of presenting information about this distance.
  • FIG. 11 is a diagram showing example 5 of a display image according to the present embodiment.
  • the separation distance is presented by a bar-shaped indicator. This enables the operator to grasp the separation distance between the surgical tool and the treatment target site.
  • the display image generation unit 13d aligns the size, orientation, and position of the lens of the stereo image with the size, orientation, and position of the lens of the surgical field image, and calculates the separation distance between the lens of the stereo image and the surgical instrument. Then, the display image generation unit 13d generates a bar-shaped indicator as separation distance information regarding the calculated separation distance and adds it to the display image.
  • the display image generation unit 13d may change the display mode, such as the color of the vicinity of the distal end of the surgical instrument and the color of the image frame attached to the periphery of the image, according to the separation distance.
  • the color may be green when the surgical tool is far from the site to be treated, and may change to orange or red as the surgical tool and the site to be treated are closer.
  • the sound may be presented by the speaker 56 so that the frequency of the presented sound increases as the surgical tool and the site to be treated approach.
  • a warning image or voice may be presented when the distance becomes less than a certain value.
  • the control unit 13A of the image processing device 13 determines whether or not the separation distance is equal to or less than a predetermined value, and if it is determined that the separation distance is equal to or less than the predetermined value, the controller instructs the display device to present a warning image. and control to instruct the speaker 56 to present audio.
  • the display image generation unit 13d aligns the size, orientation, and position of the lens in the stereo image with the size, orientation, and position of the lens in the surgical field image in accordance with the control by the control unit 13A, and displays the lens in the stereo image and the surgical tool. Calculate the distance between Then, according to the calculated separation distance, when the separation distance becomes equal to or less than a predetermined value, the display image generation unit 13d generates and displays warning information (for example, sentences, symbols, graphics, etc.) indicating that the separation distance is close. Add to image.
  • warning information for example, sentences, symbols, graphics, etc.
  • control unit 13A of the image processing device 13 determines whether or not the separation distance is equal to or less than a predetermined value. Performs control to instruct operation stop such as stopping ultrasonic vibration.
  • Various display images as described above are used, and these display images may be selectable by the operator, assistant, or the like.
  • the selection of the display image is realized by an input operation on the operation unit by the operator or the assistant.
  • an operator, an assistant, or the like operates an operation unit to select a display mode for displaying a desired display image.
  • the display image generator 13d generates a display image based on the selected display mode.
  • the size, position, etc. of the images may be changed by the operator, the assistant, or the like.
  • the display image generation unit 13d generates a display image by changing the size, position, etc. of the image according to the input operation to the operation unit by the operator or the assistant.
  • the image input unit 13c receives the operative field image
  • the 3D model receiving unit 13a receives a three-dimensional model of part or all of the crystalline lens of the patient's eye.
  • the stereo image generation unit 13b generates a stereo image of the crystalline lens from the three-dimensional model
  • the display image generation unit 13d generates a display image including the operative field image and the stereo image based on the operative field image and the stereo image.
  • display images including the operative field image and the stereo image can be presented to the operator. This makes it easier for the operator to understand the positional relationship between the surgical tool and the region of the eye to be treated. Therefore, it is possible for the operator to easily understand the positional relationship between the surgical instrument and the region of the eye to be treated, while simplifying the configuration of the device and reducing the size of the device.
  • the display image generation unit 13d generates a display image by aligning the size of the lens of the stereo image with the size of the lens of the surgical field image. This makes it possible to present a display image in which the size of the crystalline lens in the stereo image is the same as the size of the crystalline lens in the surgical field image. It can be easily understood.
  • the display image generation unit 13d generates a display image by aligning the orientation of the lens of the stereo image around the eye axis with the orientation of the lens of the surgical field image around the eye axis. This makes it possible to present a display image in which the orientation of the lens in the stereo image around the eye axis is the same as the orientation of the lens in the surgical field image around the eye axis. The positional relationship with the target part can be made easier to understand.
  • the display image generation unit 13d generates a display image by aligning the orientation of the lens of the stereo image with the orientation of the lens of the surgical field image. This makes it possible to present a display image in which the orientation of the crystalline lens in the stereo image is the same as the orientation of the crystalline lens in the surgical field image. It can be easily understood.
  • the display image generation unit 13d generates a display image by superimposing the stereo image on the operative field image.
  • the display image generation unit 13d generates a display image by superimposing the stereo image on the operative field image.
  • the display image generation unit 13d aligns the position of the lens of the stereo image with the position of the lens of the surgical field image to generate the display image. This makes it possible to present a display image in which the position of the crystalline lens in the stereo image is the same as the position of the crystalline lens in the surgical field image. It can be easily understood.
  • the display image generation unit 13d expands one or both of the lens of the stereo image and the lens of the surgical field image to generate a display image.
  • the display image generation unit 13d expands one or both of the lens of the stereo image and the lens of the surgical field image to generate a display image.
  • the display image generation unit 13d aligns the size, orientation, and position of the lens of the stereo image with the size, orientation, and position of the lens of the surgical field image, calculates the separation distance between the lens of the stereo image and the surgical tool, and calculates Spacing distance information about the spacing distance is generated and added to the display image.
  • the display image generation unit 13d aligns the size, orientation, and position of the lens of the stereo image with the size, orientation, and position of the lens of the surgical field image, calculates the separation distance between the lens of the stereo image and the surgical tool, and calculates Spacing distance information about the spacing distance is generated and added to the display image.
  • the display image generation unit 13d aligns the size, orientation, and position of the lens of the stereo image with the size, orientation, and position of the lens of the surgical field image, calculates the separation distance between the lens of the stereo image and the surgical tool, and calculates Warning information is generated according to the separation distance and added to the display image. As a result, it is possible to present warning information based on the measurement results rather than the operator's subjectivity.
  • Example of schematic configuration of computer> The series of processes described above can be executed by hardware or by software.
  • a program that constitutes the software is installed in the computer.
  • the computer includes, for example, a computer built into dedicated hardware and a general-purpose personal computer capable of executing various functions by installing various programs.
  • FIG. 12 is a diagram showing an example of a schematic configuration of a computer 500 that executes the series of processes described above by a program.
  • the computer 500 has a CPU (Central Processing Unit) 510, a ROM (Read Only Memory) 520, and a RAM (Random Access Memory) 530.
  • CPU Central Processing Unit
  • ROM Read Only Memory
  • RAM Random Access Memory
  • the CPU 510 , ROM 520 and RAM 530 are interconnected by a bus 540 .
  • An input/output interface 550 is also connected to the bus 540 .
  • An input unit 560 , an output unit 570 , a recording unit 580 , a communication unit 590 and a drive 600 are connected to the input/output interface 550 .
  • the input unit 560 is composed of a keyboard, mouse, microphone, imaging device, and the like.
  • the output unit 570 is configured with a display, a speaker, and the like.
  • the recording unit 580 is composed of a hard disk, a nonvolatile memory, or the like.
  • the communication unit 590 is configured by a network interface or the like.
  • a drive 600 drives a removable recording medium 610 such as a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory.
  • the CPU 510 loads, for example, the program recorded in the recording unit 580 into the RAM 530 via the input/output interface 550 and the bus 540, and executes it. A series of processes are performed.
  • a program executed by the computer 500 that is, the CPU 510 can be provided by being recorded on a removable recording medium 610 such as a package medium, for example. Also, the program can be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.
  • the program can be installed in the recording unit 580 via the input/output interface 550 by loading the removable recording medium 610 into the drive 600 . Also, the program can be received by the communication unit 590 and installed in the recording unit 580 via a wired or wireless transmission medium. In addition, the program can be installed in the ROM 520 or the recording unit 580 in advance.
  • the program executed by the computer 500 may be a program in which processing is performed in chronological order according to the order described in this specification, or in parallel or at a necessary timing such as when a call is made. It may be a program in which processing is performed in
  • a system means a set of multiple components (devices, modules (parts), etc.), and it does not matter whether all the components are in the same housing. Therefore, a plurality of devices housed in separate housings and connected via a network, and a single device housing a plurality of modules in one housing, are both systems. .
  • this technology can take the configuration of cloud computing in which one function is shared by multiple devices via a network and processed jointly.
  • each step described in the above process flow can be executed by a single device or shared by a plurality of devices.
  • one step includes multiple processes
  • the multiple processes included in the one step can be executed by one device or shared by multiple devices.
  • the present technology can also take the following configuration.
  • An image processing device comprising: (2) The display image generation unit is aligning the size of the lens of the stereo image with the size of the lens of the surgical field image to generate the display image;
  • the image processing apparatus according to (1) above.
  • the display image generation unit is generating the display image by aligning the orientation of the lens of the stereo image around the eye axis with the orientation of the lens of the surgical field image around the eye axis; The image processing apparatus according to (1) or (2) above.
  • the display image generation unit is generating the display image by aligning the orientation of the lens in the stereo image with the orientation of the lens in the surgical field image; The image processing apparatus according to any one of (1) to (3) above.
  • the display image generation unit is generating the display image by superimposing the stereo image on the operative field image; The image processing apparatus according to any one of (1) to (4) above.
  • the display image generation unit is generating the display image by aligning the position of the lens in the stereo image with the position of the lens in the surgical field image;
  • the image processing apparatus according to (5) above.
  • the display image generation unit is magnifying one or both of the lens of the stereo image and the lens of the surgical field image to generate the display image;
  • the image processing apparatus according to any one of (1) to (6) above.
  • the display image generation unit is Aligning the size, orientation, and position of the lens in the stereo image with the size, orientation, and position of the lens in the surgical field image, calculating the separation distance between the lens in the stereo image and the surgical instrument, and calculating the calculated separation generating separation distance information about the distance and adding it to the displayed image;
  • the image processing apparatus according to any one of (1) to (7) above.
  • the display image generation unit is Aligning the size, orientation, and position of the lens in the stereo image with the size, orientation, and position of the lens in the surgical field image, calculating the separation distance between the lens in the stereo image and the surgical instrument, and calculating the calculated separation generating warning information according to the distance and adding it to the display image;
  • the image processing apparatus according to any one of (1) to (8) above.
  • the image processing device receiving surgical field images for the patient's eye; receiving a three-dimensional model of part or all of the lens of the eye; generating a stereo image of the lens from the three-dimensional model; generating a display image including the operative field image and the stereo image based on the operative field image and the stereo image;
  • An image processing method comprising: (11) a surgical microscope for obtaining an operative field image of the patient's eye; an image processing device that generates a display image; a display device for displaying the display image; with The image processing device is an image input unit that receives the operative field image; a three-dimensional model receiving unit that receives a three-dimensional model of part or all of the lens of the eye; a stereo image generator that generates a stereo image of the crystalline lens from the three-dimensional model; a display image generation unit that generates the display image including the surgical field image and the stereo image based on the surgical field image and the stereo image;
  • a surgical microscope system having (12) An image processing method using the image processing apparatus according

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Ophthalmology & Optometry (AREA)
  • Engineering & Computer Science (AREA)
  • Public Health (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Surgery (AREA)
  • Veterinary Medicine (AREA)
  • Animal Behavior & Ethology (AREA)
  • General Health & Medical Sciences (AREA)
  • Vascular Medicine (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Physics & Mathematics (AREA)
  • Biophysics (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Microscoopes, Condenser (AREA)
  • Eye Examination Apparatus (AREA)

Abstract

Un dispositif de traitement d'image (13) selon un mode de réalisation de la présente divulgation comprend une unité d'entrée d'image (13c) qui reçoit une image de champ opératoire de l'œil d'un patient, une unité de réception de modèle tridimensionnel (13a) qui reçoit un modèle tridimensionnel d'une partie ou de la totalité du cristallin de l'œil, une unité de génération d'image stéréo (13b) qui génère une image stéréo du cristallin à partir du modèle tridimensionnel, ainsi qu'une unité de génération d'image d'affichage (13d) qui génère une image d'affichage comprenant l'image de champ opératoire et l'image stéréo sur la base de l'image de champ opératoire et de l'image stéréo.
PCT/JP2022/000855 2021-01-29 2022-01-13 Dispositif de traitement d'image, procédé de traitement d'image et système de microscope chirurgical WO2022163362A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2021012781A JP2022116559A (ja) 2021-01-29 2021-01-29 画像処理装置、画像処理方法及び手術顕微鏡システム
JP2021-012781 2021-01-29

Publications (1)

Publication Number Publication Date
WO2022163362A1 true WO2022163362A1 (fr) 2022-08-04

Family

ID=82653335

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/000855 WO2022163362A1 (fr) 2021-01-29 2022-01-13 Dispositif de traitement d'image, procédé de traitement d'image et système de microscope chirurgical

Country Status (2)

Country Link
JP (1) JP2022116559A (fr)
WO (1) WO2022163362A1 (fr)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2011030689A (ja) * 2009-07-31 2011-02-17 Nidek Co Ltd 眼底撮影システム及び立体眼底画像処理方法
JP2012506272A (ja) * 2008-10-22 2012-03-15 ゼンゾモトリック インストルメンツ ゲゼルシャフト フュア イノベイティブ ゼンゾリク ミット ベシュレンクテル ハフツング コンピューター支援眼部手術用の画像処理方法および装置
JP2015130911A (ja) * 2014-01-09 2015-07-23 パナソニックヘルスケアホールディングス株式会社 手術支援装置および手術支援プログラム
JP2016073409A (ja) * 2014-10-03 2016-05-12 ソニー株式会社 情報処理装置、情報処理方法及び手術顕微鏡装置
WO2018207466A1 (fr) * 2017-05-09 2018-11-15 ソニー株式会社 Dispositif de traitement d'images, procédé de traitement d'images et programme de traitement d'images
US20190099226A1 (en) * 2017-10-04 2019-04-04 Novartis Ag Surgical suite integration and optimization

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2012506272A (ja) * 2008-10-22 2012-03-15 ゼンゾモトリック インストルメンツ ゲゼルシャフト フュア イノベイティブ ゼンゾリク ミット ベシュレンクテル ハフツング コンピューター支援眼部手術用の画像処理方法および装置
JP2011030689A (ja) * 2009-07-31 2011-02-17 Nidek Co Ltd 眼底撮影システム及び立体眼底画像処理方法
JP2015130911A (ja) * 2014-01-09 2015-07-23 パナソニックヘルスケアホールディングス株式会社 手術支援装置および手術支援プログラム
JP2016073409A (ja) * 2014-10-03 2016-05-12 ソニー株式会社 情報処理装置、情報処理方法及び手術顕微鏡装置
WO2018207466A1 (fr) * 2017-05-09 2018-11-15 ソニー株式会社 Dispositif de traitement d'images, procédé de traitement d'images et programme de traitement d'images
US20190099226A1 (en) * 2017-10-04 2019-04-04 Novartis Ag Surgical suite integration and optimization

Also Published As

Publication number Publication date
JP2022116559A (ja) 2022-08-10

Similar Documents

Publication Publication Date Title
KR101451970B1 (ko) 안과용 수술장치 및 이의 제어 방법
JP5572161B2 (ja) 眼科手術システムならびに眼科手術システムの作動方法
JP2012152469A (ja) 眼科用手術顕微鏡
CN109963535A (zh) 集成式眼科手术系统
CN106999298A (zh) 图像处理装置、图像处理方法以及手术显微镜
US20130088414A1 (en) Surgical heads-up display that is adjustable in a three-dimensional field of view
JP6819223B2 (ja) 眼科情報処理装置、眼科情報処理プログラム、および眼科手術システム
AU2012264368A1 (en) System and method for using multiple detectors
US9138138B2 (en) Ophthalmic apparatus and recording medium having ophthalmic program stored therein
JP2024079703A (ja) 対象領域ボリュームの３ｄ再構成を確認するためのｏｃｔデバイスを備えるアセンブリ、コンピュータプログラム、及びそのためのコンピュータ実装方法
JP2008295973A (ja) 眼科測定装置、眼科測定プログラム、及び眼内レンズパワー決定方法
WO2022163362A1 (fr) Dispositif de traitement d'image, procédé de traitement d'image et système de microscope chirurgical
JP2018051208A (ja) 眼科情報処理装置、眼科情報処理プログラム、および眼科情報処理システム
KR102169674B1 (ko) 혼합 현실 안저카메라
JP7026988B1 (ja) 手術支援装置
EP4238484A1 (fr) Dispositif d'observation ophtalmologique, son procédé de commande, programme, et support d'enregistrement
JP7369148B2 (ja) 見当合わせ検証用の振動画像
WO2022163190A1 (fr) Dispositif de traitement d'image, procédé de traitement d'image et système de microscope chirurgical
WO2022163188A1 (fr) Dispositif de traitement d'images, procédé de traitement d'images et système de microscope chirurgical
US20240164850A1 (en) Ophthalmic visualization using spectrum-independent imagers, edge detection, and visible-ir image merging
JP7377331B2 (ja) 眼科装置
WO2022163383A1 (fr) Dispositif de traitement d'image, procédé de traitement d'image et système de microscope chirurgical
Gulkas et al. Intraoperative Optical Coherence Tomography
WO2022091210A1 (fr) Dispositif d'aide à la chirurgie
Miclos et al. A data acquisition, processing and storage system for an ophthalmic instrument: Fotobioftal-1

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22745587

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22745587

Country of ref document: EP

Kind code of ref document: A1