WO2014175514A1 - Operating microscope system of superimposing and outputting surface image and augmented reality image and apparatus of providing augmented reality image therefor - Google Patents

Operating microscope system of superimposing and outputting surface image and augmented reality image and apparatus of providing augmented reality image therefor Download PDF

Info

Publication number
WO2014175514A1
WO2014175514A1 PCT/KR2013/007735 KR2013007735W WO2014175514A1 WO 2014175514 A1 WO2014175514 A1 WO 2014175514A1 KR 2013007735 W KR2013007735 W KR 2013007735W WO 2014175514 A1 WO2014175514 A1 WO 2014175514A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
light
optoacoustic
surgical microscope
augmented reality
Prior art date
Application number
PCT/KR2013/007735
Other languages
French (fr)
Korean (ko)
Inventor
김지현
이창호
한승훈
강세희
김철홍
Original Assignee
경북대학교 산학협력단
포항공과대학교 산학협력단
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 경북대학교 산학협력단, 포항공과대학교 산학협력단 filed Critical 경북대학교 산학협력단
Publication of WO2014175514A1 publication Critical patent/WO2014175514A1/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/006—Mixed reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/20—Surgical microscopes characterised by non-optical aspects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G06T11/60—Editing figures and text; Combining figures or text
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—3D [Three Dimensional] image rendering
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/36—Image-producing devices or illumination devices not otherwise provided for
    • A61B2090/364—Correlation of different images or relation of image positions in respect to the body
    • A61B2090/365—Correlation of different images or relation of image positions in respect to the body augmented reality, i.e. correlating a live optical image with another image

Definitions

  • the present invention relates to a surgical microscope device, and more particularly, to a surgical microscope system for outputting by superimposing augmented reality image including a photoacoustic tomography image on a high magnification surface image and an apparatus for providing augmented reality image therefor.
  • a surgical microscope includes a light source 3 for irradiating light by a power source, an optical cable 4 for connecting the light of the light source 3 to a living body, and the living body as shown in FIG.
  • An objective lens 5 for enlarging the reflected light to generate an enlarged biometric image
  • an eyepiece 6 for visually confirming the enlarged biometric image.
  • Such a surgical microscope has a light source (3) provides light to the living body along the optical cable (4), the objective lens (5) magnifies and outputs the light reflected by the living body through the eyepiece (6), the operator -Observe and operate the living body based on the image output through the eyepiece (6).
  • the present invention provides an augmented reality image comprising an optoacoustic tomographic image from a PAT / PAM device in a high magnification surface image obtained by a surgical microscope.
  • Augmented reality that includes an optoacoustic tomographic image in a high magnification surface image that allows the operator to simultaneously receive a high magnification surface image and an optoacoustic tomographic image simultaneously through the eyepiece of the microscope without moving the eye during the procedure. It is an object of the present invention to provide a surgical microscope system for superimposing and outputting an image and an augmented reality image providing device therefor.
  • another object of the present invention is to include the information necessary for the operation in the augmented reality image, the operator through the eyepiece of the microscope through the eyepiece of the microscope without the movement of the eye during the procedure, as well as the information necessary for the surgery
  • the present invention provides a surgical microscope system and augmented reality image providing apparatus for superimposing and outputting the augmented reality image including a photoacoustic tomography image on a high magnification surface image.
  • an apparatus for providing an augmented reality image for a surgical microscope device which generates and outputs a low U light for generating an optoacoustic signal, and outputs the light from an object.
  • An optoacoustic image acquisition device receiving an acoustic signal to generate an optoacoustic image; Passing the second light provided from the surgical microscope device to the object, passing the second light reflected by the object to provide the surgical microscope device, and scanning the object to scan the object.
  • a scanning unit providing U light to the object, acquiring an optoacoustic signal generated by the first light incident on the object, and providing the photoacoustic signal to the optoacoustic image acquisition device;
  • a photoacoustic image provided by the surgical microscope device to transmit the two light beams to the object, and provide the second light reflected by the object to the surgical microscope device, the photoacoustic image acquisition device
  • a display unit configured to generate a beam according to the light incident surface of the surgical microscope device.
  • An augmented reality image providing apparatus for a surgical microscope device generates and outputs the first light for generating an optoacoustic signal, and receives the optoacoustic signal output from the object to receive the optoacoustic image Generating an optoacoustic image acquisition device;
  • a second provided from the surgical microscope device Passing light to the object, passing the second light reflected by the object to provide the surgical microscope device, and providing the first light to the object to scan the object, and
  • a scanning unit which acquires an optoacoustic signal generated by the first light incident on the object and provides the optoacoustic image acquisition device;
  • a computing device configured to receive an optoacoustic image from the optoacoustic image acquisition device and to generate and output an augmented reality image by including information related to the optoacoustic image in the optoacoustic image; Delivering a second light provided from the surgical microscope device to the object, providing the light to the surgical microscope device reflected by the object, and generating
  • the first light is preferably one of a pulse laser, a broadband wavelength pulse laser, a continuous wave laser, and a fill microwave, capable of generating an optoacoustic signal.
  • the photoacoustic image acquisition device is a photoacoustic microscopy (PAM) or photoacoustic tomography (PAT) It is preferable that it is one of ').
  • a surgical microscope system comprising: an optoacoustic image acquisition device that outputs a light for generating an optoacoustic signal, and receives an optoacoustic signal output from an object to generate an optoacoustic image; A surgical microscope device for irradiating a second light to the object and receiving and expanding the second light reflected by the object to generate and output a surface image of the object; A scanning unit providing a first light to the object so as to scan the object, and acquiring an optoacoustic signal output from the object to provide the optoacoustic image acquisition device; And a photoacoustic tomographic image superimposed on the surface image of the target object, including a display unit generating a beam according to the photoacoustic image provided from the photoacoustic image acquisition device and providing the beam to the light incident surface of the surgical microscope device.
  • a surgical microscope system includes: an optoacoustic image acquisition device configured to output a first light for generating an optoacoustic signal, and receive an optoacoustic signal output from an object to generate an optoacoustic image; A surgical microscope device for irradiating a second light to the object and receiving and expanding the second light reflected by the object to generate and output a surface image of the object; A scanning unit that provides one light to the object so as to scan the object, obtains an optoacoustic signal output from the object, and provides the photoacoustic signal to the optoacoustic image acquisition device; A computing device configured to receive an optoacoustic image from the optoacoustic image acquisition device, and generate and output an augmented reality image by including optoacoustic image related information in the optoacoustic image; A display unit for generating a pan in accordance with the augmented reality image provided from the computing device and providing the light incident surface of the
  • the scanning unit transmits the second light from the surgical microscope device to the object, and the second light reflected by the object. It passes through to provide to the surgical microscope device, and transmits the low U light to the object, to obtain an optoacoustic signal generated by the first light incident on the object to provide to the optoacoustic image acquisition device It is preferable.
  • the display unit passes the second light from the surgical microscope device to the object and transmits the second light reflected by the object. It is preferable to pass through to provide the surgical microscope device, and to generate a beam according to the optoacoustic image to provide a light incident surface of the surgical microscope device.
  • the present invention described above superimposes and outputs an augmented reality image including a photoacoustic tomography image from a PAT / PAM device in a high magnification surface image obtained by a surgical microscope, so that the operator can use a microscopic lens without moving the eye during the procedure.
  • Magnification surfaceUnknown optoacoustic stage At the same time it can be provided in real time to facilitate the operator's convenience.
  • the operator can view the high magnification surface image, the blood vessel distribution image through PAT / PAM, and the high resolution image of a specific substance through an external contrast agent at the same time in a non-invasive and real time through the eyepiece of the surgical microscope.
  • the present invention can facilitate the operator to perform a procedure such as blood vessel or cerebrovascular surgery.
  • the present invention includes the information necessary for surgery in the augmented reality image, the operator through the eyepiece of the microscope without movement of the eye during the procedure.
  • the high magnification surface image and the photoacoustic tomography image as well as the information necessary for the operation can be provided, there is an effect that can facilitate the operator.
  • FIG. 1 is a schematic structural diagram of a conventional surgical microscope.
  • FIG. 2 is a structural diagram of a surgical microscope system for superimposing and outputting an augmented reality image including a photoacoustic tomography image on a high magnification surface image according to an embodiment of the present invention.
  • FIG. 3 is a process diagram of a computing device according to a first preferred embodiment of the present invention.
  • FIGS. 4 and 5 are views showing an example in which a photoacoustic tomographic image is superimposed and displayed on a high magnification surface image according to a first preferred embodiment of the present invention.
  • FIG. 6 is a structural diagram of a surgical microscopy system for superposing and outputting an augmented reality image including an optoacoustic: tomographic image to a high magnification surface image according to a second preferred embodiment of the present invention.
  • an optoacoustic image acquisition device Prior to the description of the preferred embodiments of the present invention, an optoacoustic image acquisition device will be described.
  • the photoacoustic image acquisition apparatus is one of photoacoustic microscopy ( ⁇ ) or photoacoustic tomography (PAT), and in this specification, photoacoustic image acquisition.
  • the device is referred to as TAT / PAM device.
  • the PAT / PAM device is capable of microvascular imaging and blood oxygen saturation by mapping the distribution of absorbers of light, and requires a pulsed laser for this purpose.
  • the PAT / PAM device is generated when a pulsed laser enters a living body, the absorber in the living body absorbs the Fils laser, and the absorbed energy is converted into heat, and the heat is converted into pressure again.
  • the resulting photoacoustic signal is acquired by the transducer and imaged.
  • PAT / PAM device is a technology that combines optical imaging and ultrasonic imaging method has attracted much attention as the next generation high resolution medical imaging technology because it has a non-invasive, inexpensive, portable contrast contrast ratio and excellent spatial resolution.
  • the PAT / PAM device also supports tissues such as lymph nodes, which have been confused with the surroundings and difficult to identify in other technologies, thanks to eddy contrast agents such as hemoglobin, melanin, DNA, organic dyes, and genes engineered to release colored products. It can also image. Accordingly, the present invention overlaps and outputs an augmented reality image including an optoacoustic tomographic image from a PAT / PAM device in a high magnification surface image obtained by a surgical microscope, thereby allowing the operator to move the eyepiece of the microscope without moving the eye during the procedure. Through this, high magnification surface image and photoacoustic tomography image can be provided at the same time to facilitate the operator's convenience.
  • the operator can view the high-resolution surface image, the blood vessel distribution image acquired through PAT / PAM, or the high resolution image of the special 3 ⁇ 4 material through an external contrast agent through non-invasive and real time through the eyepiece of the surgical microscope.
  • the present invention can facilitate the operator to perform a procedure such as blood vessel or cerebrovascular surgery.
  • the present invention includes the information necessary for the operation in the augmented reality image, the operator to a high magnification surface through the microscope lens of the lens without shifting the eye to the procedure Images and optoacoustic tomography images as well as the information necessary for the operation can be provided to facilitate the operator's convenience.
  • the surgical microscope system includes a surgical microscope apparatus 100, a PAT / PAM apparatus 200, a PAT / PAM scanning unit 300, a display unit 320, and a computing device 400.
  • the surgical microscope device 100 includes a light source, an optical cable, an eyepiece, an objective lens, and the like, irradiates light onto the object 500, enlarges the light reflected by the object 500, and outputs the light through the eyepiece. do.
  • the apparatus 200 generates and generates a first light for generating an optoacoustic signal and provides the generated light to the PAT / PAM scanning unit 300, and processes the optoacoustic signal provided by the PAT / PAM scanning unit 300.
  • Optoacoustic tomography image information is generated and provided to the computing device 400.
  • the first light is incident on an object to generate an optoacoustic signal, and may be configured as one of a broadband wavelength pulse laser, a pulse laser, a CW laser, and a microwaves.
  • the pulsed microwave is an electromagnetic wave having a frequency from 1 GHz to 30 GHz.
  • the PAT / PAM scanning unit 300 is a surgical microscope device .
  • Crab 2 light from 100 is provided to object 500 to be reflected and passed to the multi-surgical microscope device 100 for transmission, and crab light from PAT / PAM device 200 is two-dimensional to the object.
  • the target 500 is provided to the object 500 while changing an optical path to scan the object for 3D imaging, and receives the photoacoustic signal generated by the object 500 to the PAT / PAM device 200. to provide.
  • the PAT / PAM scanning unit 300 is an optical path changing unit 302 ,.
  • the light path changing unit 302 may be configured as one of a galvanometer scanner, a polygon scanner, and a MEMSOnicroelectromechanical system scanner.
  • a galvanometer scanner is configured, and the galvanometer scanner can scan an object to acquire a two-dimensional or three-dimensional image of the first light from the PAT / PAM device 200.
  • the optical path is provided to the optical path guide unit 310 while changing the optical path to one light in the XY direction.
  • the light path guide unit 310 may be composed of a dichroic mirror or a pan splitter.
  • the induction part is a material that reflects only light of a specific wavelength, and is located between the object ⁇ and the objective lens of the surgical microscope device 100, Delivering the second light from the object 100 to the object 500 and the second light reflected by the Arler object 500 is provided to the surgical microscope device 100, from the surgical microscope device 100.
  • the objective lens 308 focuses the first light from the light path changing unit 302 on the light path inducing unit.
  • the photoacoustic signal acquisition gong-chi 312 acquires the optoacoustic signal generated when the object 500 absorbs the first light, and the absorbed energy is converted into heat, and the heat is converted into pressure.
  • the photoacoustic signal acquisition device may be configured of a contact transducer such as an ultrasonic transducer, a non-contact photoacoustic signal acquisition device, or the like.
  • the non-contact photoacoustic signal acquisition device is preferably a non-contact photoacoustic signal acquisition device using a Febri-Perot interferometer or an optical coherent tomography (OCT ') interferometer.
  • the computing device 400 generates augmented reality image information superimposed on the photoacoustic tomography image information provided by the PAT / PAM device 200 and the image information on the surgery-related information input by the operator, the augmented reality
  • the image information is provided to the display unit 320.
  • the display unit 320 is a first from the surgical microscope device 100
  • the light is provided to the object 500 to be reflected and passed back to the surgical microscope device 100, and the augmented reality image is converted into a beam and overlapped with the light incident path of the surgical microscope device 100. Let it enter.
  • the display unit 320 includes a beam projector 322 and a beam splitter 324.
  • the pan projector 322 converts augmented reality image information from the computing device 400 into a beam and emits the beam.
  • the beam splitter 324 is located in a part of the light exit and incident surface of the surgical microscope device 100, and divides the incident light by a certain ratio to partially pass and partially reflect the augmented reality image.
  • the beam is provided according to the information, and a portion of the beam increment is provided as a light incident path of the surgical microscope device 100.
  • the image output through the eyepiece of the surgical microscope device 100 is a high magnification surface image obtained by the surgical microscope device 100 and augmented reality image from the ⁇ 7 ⁇ device 200 and the computing device 400 It will include.
  • the computing device 400 receives optoacoustic tomographic image information from the PAT / PAM device 200 and is also operated by an operator.
  • the surgery-related information is input (step 700/702)
  • the photoacoustic tomography image information and the surgery-related information about the object are combined to generate augmented reality image information
  • the generated augmented reality: image information is displayed on the display unit 320.
  • the beam projector 322 step 704
  • a high magnification surface image, photoacoustic tomography image information, and augmented reality image according to surgery-related information are simultaneously output to the eyepiece of the surgical microscope device 100.
  • the augmented reality image information according to the optoacoustic tomography image information
  • the generated augmented reality image information is provided to the beam projector 322 of the display 320 (step 706).
  • the eyepiece of the surgical microscope device 100 as augmentation according to the high magnification surface image and photoacoustic tomography image information
  • the reality image is output at the same time.
  • FIG. 4 illustrates an image output to the surgical microscope device 100 of the surgical microscope system according to the first preferred embodiment of the present invention.
  • FIG. 4 is an image of a phantom in which four hairs (H1 to H4) and a needle (N) are arranged with a surgical microscope system according to a preferred embodiment of the present invention, and the eyepiece of the surgical microscope device 100 is shown in FIG.
  • the furnace displays a high-magnification surface image with a high magnification of four strands of hair (H1 H4) and an injection bar (N) together with a two-dimensional photoacoustic short image.
  • the high magnification surface image shows no difference between the height of the syringe and the hair, but the height difference between the syringe and the hair appears on the two-dimensional photoacoustic tomography image.
  • the height difference between the four hairs H1 to H4 and the injection needle N is clearly revealed.
  • the photoacoustic tomography image information from the PAT / PAM device is provided to the beam projector through the computing device, but the photoacoustic tomography image information from the PAT / PAM device is provided. It can also be provided directly to the beam projector.
  • the beam from the beam projector is configured to provide a partial region of the light incident surface of the surgical microscope device, and the size and position of the region is for the convenience of the operator. It is obvious by the present invention that it can be changed.
  • a surgical microscope device a PAT / PAM device, a PAT / PAM scanning unit, a display unit, and a computing device are combined to constitute a surgical microscope system.
  • the microscope system may be configured by combining a PAT / PAM device, a PAT / PAM scanning unit, a display unit, and a computing device to an existing surgical microscope device. It is possible to be provided with an augmented reality image according to the present invention without replacing the surgical microscope device provided in advance.
  • the PAT / PAM device, the PAT / PAM scanning unit, the display unit, and the computing device are referred to as an augmented reality image providing device.
  • the photoacoustic signal acquisition device only the ultrasonic transducer is disclosed as a contact type device which obtains the photoacoustic signal by contacting the skin through a medium such as gel or water.
  • Optical photoacoustic signal acquisition device of the optical system for acquiring the photoacoustic signal may be employed, which is obvious by the present invention.
  • a structure of a surgical microscope system for superposing and outputting an augmented reality image including an optoacoustic tomographic image on a high magnification surface image according to a second preferred embodiment of the present invention will be described in detail with reference to FIG. 6.
  • the surgical microscope system comprises a surgical microscope device 900, a PAT / PAM device 200, a PAT / PAM scanning unit 300, a display unit 800 and a computing device 400. Since the configurations and operations of the PAT / PAM device 200, the PAT / PAM scanning unit 300, and the computing device 400 are the same as those of the first embodiment, detailed descriptions thereof will be omitted.
  • the surgical microscope device 900 includes an eyepiece 902 and an objective lens 904, which have a general microscope structure, but have a display device 800 embedded in a portion of an upper surface of the eyepiece 902.
  • the upper surface of the eyepiece 902 refers to the surface in contact with the eyes of the user to output an image to the user.
  • the display device 800 is composed of a small display mode 802, the ⁇ -type display module 802 is located in a predetermined partial region of the upper surface of the eyepiece ⁇ (902) of the surgical microscope device 900
  • the augmented reality image information is received from the computing device 400 and displayed.
  • the surgical microscope device (.900) is a compact display and a high magnification surface image obtained by the eyepiece 902 and the objective lens 904.
  • Information displayed through the modules 802 may be output at the same time.
  • the small display modules 802 only display the augmented reality image information provided from the computing device 400, but the PAT / PAM device other than the computing device 400.
  • Optoacoustic tomography image information may be directly received from 200 and displayed, which is obvious by the present invention.
  • the mounting position of the small display mode 802 may be selected such that a high magnification surface image output through the eyepiece 902 can be observed while being within the visible range of the user. It is obvious by the present invention.
  • the device according to the invention can be widely used in surgical microscope devices.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Graphics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Surgery (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Software Systems (AREA)
  • Pathology (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Hardware Design (AREA)
  • Biomedical Technology (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Animal Behavior & Ethology (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Microscoopes, Condenser (AREA)

Abstract

The present invention relates to an operating microscope system which is capable of superimposing an augmented reality image containing a photoacoustic tomography image on a high magnification surface image for an object and outputting the superimposed image. The operating microscope system comprises: an operating microscope device which outputs a high magnification surface image for an object; and an augmented reality image providing device which generates an augmented reality image containing a two-dimensional or three-dimensional photoacoustic tomography image for the object and object-related information, superimposes the augmented reality image on the high magnification surface image and outputs the superimposed image. According to the present invention, an augmented reality image containing a photoacoustic tomography image is imposed on a high magnification surface image for an object, so as to simultaneously display the images.

Description

【명세서】  【Specification】
【발명의 명칭】  [Name of invention]
표면 이미지와 증강 현실 이미지를 중첩시켜 출력하는 수술용 현미경 시스템 및 그를 위한 증강 현실 이미지 제공 장치  Surgical microscope system for superimposing surface image and augmented reality image and outputting device for augmented reality image
【기술분야】 Technical Field
본 발명은 수술용 현미경 장치에 관한 것으로, 더욱 상세하게는 고배율 표면 이미지에 광음향 단층 이미지를 포함하는 증강 현실 이미지를 중첩시켜 출력하는 수술용 현미경 시스템 및 그를 위한 증강 현실 이미지 제공 장치에 관한 것이다.  The present invention relates to a surgical microscope device, and more particularly, to a surgical microscope system for outputting by superimposing augmented reality image including a photoacoustic tomography image on a high magnification surface image and an apparatus for providing augmented reality image therefor.
【배경기술】 Background Art
일반적으로 수술용 현미경은 도 1에 도시한 바와 같이 전원에 의해 광을 조사하는 광원 (3)과， 상기 광원 (3)의 광을 생체로 조사되도록 연결해주는 광케이블 (4)과, 상기 생체에 의해 반사된 광을 확대하여 확대된 생체 이미지를 생성하는 대물렌즈 (5)와, 상기 확대된 생체 이미지를 육안으로 확인하도록 한 접안렌즈 (6)로 구성된다.  In general, a surgical microscope includes a light source 3 for irradiating light by a power source, an optical cable 4 for connecting the light of the light source 3 to a living body, and the living body as shown in FIG. An objective lens 5 for enlarging the reflected light to generate an enlarged biometric image, and an eyepiece 6 for visually confirming the enlarged biometric image.
이러한 수술용 현미경은 광원 (3)이 광케이블 (4)을 따라 생체에 광을 제공하고， 그 생체에 의해 반사된 광을 대물렌즈 (5)가 확대하여 접안렌즈 (6)를 통해 출력하며， 시술자는 -상기 접안렌즈 (6)를 통해 출력되는 이미지를 토대로 생체를 관찰 및 수술한다.  Such a surgical microscope has a light source (3) provides light to the living body along the optical cable (4), the objective lens (5) magnifies and outputs the light reflected by the living body through the eyepiece (6), the operator -Observe and operate the living body based on the image output through the eyepiece (6).
상기한 수술용 현미경에 의해 획득된 이미지에는， 생체의 여러 층에서 발생하는 명암이 존재하며 , 그 명암은 생체 조직을 용이하게 구분할 수 없게 하는 원인이 되었다.  In the image obtained by the above-described surgical microscope, light and dark occur in various layers of the living body, and the light and dark cause the tissues to be indistinguishable.
이에 종래에는 수술용 현미경에 의해 획득된 이미지에 대해 다양한 부가정보참조 이미지를 제공하여 시술자가 생체 조직을 용이하게 파악할 수 있게 하는 기술의 개발이 절실하게 요망되었다.  Therefore, in the related art, there is an urgent need for the development of a technology that enables the operator to easily grasp the biological tissue by providing various additional information reference images for the image obtained by the surgical microscope.
【발명의 상세한 설명】 [Detailed Description of the Invention]
【기술적 과제】  [Technical problem]
본 발명은 수술용 현미경에 의해 획득된 고배율 표면 이미지내에 PAT/PAM 장치로부터의 광음향 단층 이미지를 포함하는 증강 현실 이미지를 중첩시켜 출력하여， 시술자가 시술중에 시선의 이동없이 현미경의 접안렌즈를 통해 고배율 표면 이미지 및 광음향 단층 이미지를 동시에 실시간으로 제공받을 수 있게 하는, 고배율 표면 이미지에 광음향 단층 이미지를 포함하는 증강 현실 이미지를 중첩시켜 출력하는 수술용 현미경 시스템 및 그를 위한 증강 현실 이미지 제공 장치를 제공하는 것을 그 목적으로 한다. The present invention provides an augmented reality image comprising an optoacoustic tomographic image from a PAT / PAM device in a high magnification surface image obtained by a surgical microscope. Augmented reality that includes an optoacoustic tomographic image in a high magnification surface image that allows the operator to simultaneously receive a high magnification surface image and an optoacoustic tomographic image simultaneously through the eyepiece of the microscope without moving the eye during the procedure. It is an object of the present invention to provide a surgical microscope system for superimposing and outputting an image and an augmented reality image providing device therefor.
또한 본 발명의 다른 목적은 증강 현실 이미지에 수술에 필요한 정보를 포함시켜， 시술자가 시술중에 시선의 이동없이 현미경의 접안렌즈를 통해 고배율 표면 이미지 및 광음향 단층 이미지는 물론아고 수술에 필요한 정보들까지 제공받을 수 있게 하는， 고배율 표면 이미지에 광음향 단층 이미지를 포함하는 증강 현실 이미지를 중첩시켜 출력하는 수술용 현미경 시스템 및 그를 위한 증강 현실 이미지 제공 장치를 제공하는 것이다.  In addition, another object of the present invention is to include the information necessary for the operation in the augmented reality image, the operator through the eyepiece of the microscope through the eyepiece of the microscope without the movement of the eye during the procedure, as well as the information necessary for the surgery The present invention provides a surgical microscope system and augmented reality image providing apparatus for superimposing and outputting the augmented reality image including a photoacoustic tomography image on a high magnification surface image.
【기술적 해결방법】 Technical Solution
전술한 기술적 과제를 달성하기 위한 본 발명의 게 1 특징은 수술용 현미경 장치를 위한 증강 현실 이미지 제공 장치에 관한 것으로서, 광음향 신호 생성을 위한 저 U 광을 생성하여 출력하며, 대상물로부터 출력된 광음향 신호를 제공받아 광음향 이미지를 생성하는 광음향 이미지 획득 장치; 상기 수술용 현미경 장치로부터 제공되는 제 2 광을 통과시켜 상기 대상물로 전달하고， 상기 대상물에 의해 반사된 제 2 광을 통과시켜 상기 수술용 현미경 장치로 제공하며, 상기 대상물을 스캐닝할 수 있도록 상기 거 U 광을 상기 대상물로 제공하고， 상기 대상물에 입사된 제 1 광에 의해 생성된 광음향 신호를 획득하여 상기 광음향 이미지 획득 장치로 제공하는 스캐닝부; 상기 수술용 현미경 장치로부터 제공되는 게 2 광을 상기 대상물로 전달하고， 상기 대상물에 의해 반사된 제 2 광을 상기 수술용 현미경 장치로 제공하며， 상기 광음향 이미지 획득 장치에 의해 생성된 광음향 이미지에 따른 빔을 생성하여 상기 수술용 현미경 장치의 광 입사면으로 제공하는 디스플레이부；를 포함한다.  According to an aspect of the present invention, there is provided an apparatus for providing an augmented reality image for a surgical microscope device, which generates and outputs a low U light for generating an optoacoustic signal, and outputs the light from an object. An optoacoustic image acquisition device receiving an acoustic signal to generate an optoacoustic image; Passing the second light provided from the surgical microscope device to the object, passing the second light reflected by the object to provide the surgical microscope device, and scanning the object to scan the object. A scanning unit providing U light to the object, acquiring an optoacoustic signal generated by the first light incident on the object, and providing the photoacoustic signal to the optoacoustic image acquisition device; A photoacoustic image provided by the surgical microscope device to transmit the two light beams to the object, and provide the second light reflected by the object to the surgical microscope device, the photoacoustic image acquisition device And a display unit configured to generate a beam according to the light incident surface of the surgical microscope device.
본 발명의 제 2 특징에 따른 수술용 현미경 장치를 위한 증강 현실 이미지 제공 장치는， 광음향 신호 생성을 위한 제 1 광을 생성하여 출력하며， 대상물로부터 출력된 광음향 신호를 제공받아 광음향 이미지를 생성하는 광음향 이미지 획득 장치; 상기 수술용 현미경 장치로부터 제공되는 제 2 광을 통과시켜 상기 대상물로 전달하고, 상기 대상물에 의해 반사된 제 2 광을 통과시켜 상기 수술용 현미경 장치로 제공하며, 상기 대상물을 스캐닝할 수 있도록 상기 제 1 광을 상기 대상물로 제공하고, 상기 대상물에 입사된 게 1 광에 의해 생성된 광음향 신호를 획득하여 상기 광음향 이미지 획득 장치로 제공하는 스캐닝부; 상기 광음향 이미지 획득 장치로부터 광음향 이미지를 제공받고， 상기 광음향 이미지에 상기 광음향 아미지와 관련된 정보를 포함시켜 증강 현실 이미지를 생성하여 출력하는 컴퓨팅 장치; 상기 수술용 현미경 장치로부터 제공되는 제 2 광을 상기 대상물로 전달하고, 상기 대상물에 의해 반사된 게 2 광을 상기 수술용 현미경 장치로 제공하며, 상기 컴퓨팅 장치로부터 제공된 증강 현실 이미지에 따른 빔을 생성하여 상기 수술용 현미경 장치의 광 입사면으로 제공하는 디스플레이부; 를 구비한다. An augmented reality image providing apparatus for a surgical microscope device according to a second aspect of the present invention, generates and outputs the first light for generating an optoacoustic signal, and receives the optoacoustic signal output from the object to receive the optoacoustic image Generating an optoacoustic image acquisition device; A second provided from the surgical microscope device Passing light to the object, passing the second light reflected by the object to provide the surgical microscope device, and providing the first light to the object to scan the object, and A scanning unit which acquires an optoacoustic signal generated by the first light incident on the object and provides the optoacoustic image acquisition device; A computing device configured to receive an optoacoustic image from the optoacoustic image acquisition device and to generate and output an augmented reality image by including information related to the optoacoustic image in the optoacoustic image; Delivering a second light provided from the surgical microscope device to the object, providing the light to the surgical microscope device reflected by the object, and generating a beam in accordance with the augmented reality image provided from the computing device A display unit for providing the light incident surface of the surgical microscope device; It is provided.
전술한 제 1 특징 및 제 2 특징에 따른 증강 현실 이미지 제공장치에 있어서， 상기 제 1 광은 광음향 신호 생성이 가능한 펄스 레이저， 광대역 파장 펄스 레이저, 연속파 레이저 및 필스 마이크로파 증 하나인 것이 바람직하다.  In the apparatus for providing augmented reality images according to the first and second features described above, the first light is preferably one of a pulse laser, a broadband wavelength pulse laser, a continuous wave laser, and a fill microwave, capable of generating an optoacoustic signal.
전술한 게 1 특징 및 게 2 특징에 따른 증강 현실 이미지 제공장치에 있어서, 상기 광음향 이미지 획득 장치는 광음향 현미경 (Photoacoustic microscopy; 'PAM' ) 또는 광음향 단층 촬영장치 (Photoacoust ic Tomography; 'PAT' ) 중 하나인 것이 바람직하다.  In the augmented reality image providing apparatus according to the above-described Crab 1 and Crab 2 features, the photoacoustic image acquisition device is a photoacoustic microscopy (PAM) or photoacoustic tomography (PAT) It is preferable that it is one of ').
본 발명의 제 3 특징에 따른 수술용 현미경 시스템은， 광음향 신호 생성을 위한 게 1 광을 출력하며, 대상물로부터 출력되는 광음향 신호를 제공받아 광음향 이미지를 생성하는 광음향 이미지 획득 장치; 대상물에 제 2 광을 조사하고 그 대상물에 의해 반사된 제 2 광을 입사받아 확대하여 대상물에 대한 표면 이미지를 생성하여 출력하는 수술용 현미경 장치; 대상물을 스캐닝할 수 있도록 제 1 광을 대상물로 제공하고, 상기 대상물로부터 출력되는 광음향 신호를 획득하여 상기 광음향 이미지 획득 장치로 제공하는 스캐닝부; 상기 광음향 이미지 획득 장치로부터 제공되는 광음향 이미지에 따른 빔을 생성하여 상기 수술용 현미경 장치의 광 입사면으로 제공하는 디스플레이부；를 포함하여, 대상물에 대한 표면 이미지에 광음향 단층 이미지를 중첩시켜 출력한다. 본 발명의 제 4 특징에 따른 수술용 현미경 시스템은， 광음향 신호를 생성하기 위한 제 1 광을 출력하며, 대상물로부터 출력된 광음향 신호를 제공받아 광음향 이미지를 생성하는 광음향 이미지 획득 장치 ; 대상물에 제 2 광을 조사하고 그 대상물에 의해 반사된 제 2 광을 입사받아 확대하여 대상물에 대한 표면 이미지를 생성하여 출력하는 수술용 현미경 장치; 대상물을 스캐닝할 수 있도록 게 1 광을 대상물로 제공하고， 상기 대상물로부터 출력되는 광음향 신호를 획득하여 상기 광음향 이미지 획득 장치로 제공하는 스캐닝부; 상기 광음향 이미지 획득 장치로부터 광음향 이미지를 제공받고, 상기 광음향 이미지에 광음향 이미지 관련 정보를 포함시켜 증강 현실 이미지를 생성하여 출력하는 컴퓨팅 장치 ; 상기 컴퓨팅 장치로부터 제공되는 증강 현실 이미지에 따른 범을 생성하여 상기 수슬용 현미경 장치의 광 입사면으로 제공하는 디스플레이부；를 포함하여， 대상물에 대한 표면 이미지에 광음향 단층 이미지를 포함한 증강 현실 이미지를 중첩시켜 출력한다. According to a third aspect of the present invention, there is provided a surgical microscope system, comprising: an optoacoustic image acquisition device that outputs a light for generating an optoacoustic signal, and receives an optoacoustic signal output from an object to generate an optoacoustic image; A surgical microscope device for irradiating a second light to the object and receiving and expanding the second light reflected by the object to generate and output a surface image of the object; A scanning unit providing a first light to the object so as to scan the object, and acquiring an optoacoustic signal output from the object to provide the optoacoustic image acquisition device; And a photoacoustic tomographic image superimposed on the surface image of the target object, including a display unit generating a beam according to the photoacoustic image provided from the photoacoustic image acquisition device and providing the beam to the light incident surface of the surgical microscope device. Output According to a fourth aspect of the present invention, a surgical microscope system includes: an optoacoustic image acquisition device configured to output a first light for generating an optoacoustic signal, and receive an optoacoustic signal output from an object to generate an optoacoustic image; A surgical microscope device for irradiating a second light to the object and receiving and expanding the second light reflected by the object to generate and output a surface image of the object; A scanning unit that provides one light to the object so as to scan the object, obtains an optoacoustic signal output from the object, and provides the photoacoustic signal to the optoacoustic image acquisition device; A computing device configured to receive an optoacoustic image from the optoacoustic image acquisition device, and generate and output an augmented reality image by including optoacoustic image related information in the optoacoustic image; A display unit for generating a pan in accordance with the augmented reality image provided from the computing device and providing the light incident surface of the confinement microscope device to the augmented reality image including an optoacoustic tomography image on the surface image of the object; Overlap the output.
전술한 제 3 특징 및 제 4 특징에 따른 수술용 현미경 시스템에 있어세 상기 스캐닝부는， 상기 수술용 현미경 장치로부터의 제 2 광을 통과시켜 상기 대상물로 전달하고, 상기 대상물에 의해 반사된 게 2 광을 통과시켜 상기 수술용 현미경 장치로 제공하며, 상기 저 U 광을 상기 대상물로 전달하고， 상기 대상물에 입사된 제 1 광에 의해 생성되는 광음향 신호를 획득하여 상기 광음향 이미지 획득 장치로 제공하는 것이 바람직하다.  In the surgical microscope system according to the third and fourth features described above, the scanning unit transmits the second light from the surgical microscope device to the object, and the second light reflected by the object. It passes through to provide to the surgical microscope device, and transmits the low U light to the object, to obtain an optoacoustic signal generated by the first light incident on the object to provide to the optoacoustic image acquisition device It is preferable.
전술한 제 3 특징 및 제 4 특징에 따른 수술용 현미경 시스템에 있어서， 상기 디스플레이부는, 상기 수술용 현미경 장치로부터의 제 2 광을 통과시켜 상기 대상물로 전달하고 상기 대상물에 의해 반사된 게 2 광을 통과시켜 상기 수.술용 현미경 장치로 제공하며, 상기 광음향 이미지에 따른 빔을 생성하여 상기 수술용 현미경 장치의 광 입사면으로 제공하는 것이 바람직하다.  In the surgical microscope system according to the third and fourth features described above, the display unit passes the second light from the surgical microscope device to the object and transmits the second light reflected by the object. It is preferable to pass through to provide the surgical microscope device, and to generate a beam according to the optoacoustic image to provide a light incident surface of the surgical microscope device.
【유리한 효과】 Advantageous Effects
상기한 본 발명은 수술용 현미경에 의해 획득된 고배율 표면 이미지내에 PAT/PAM 장치로부터의 광음향 단층 이미지를 포함하는 증강 현실 이미지를 중첩시켜 출력하여， 시술자가 시술중에 시선의 이동없이 미경의 렌즈를 통해ᅳ 배율 표면ᅵ 미지 광음향 단 쒜 동시에 실시간으로 제공받을 수 있게 하여 시술자의 편이를 도모할 수 있는 효과가 있다. The present invention described above superimposes and outputs an augmented reality image including a photoacoustic tomography image from a PAT / PAM device in a high magnification surface image obtained by a surgical microscope, so that the operator can use a microscopic lens without moving the eye during the procedure. Magnification surfaceUnknown optoacoustic stage At the same time it can be provided in real time to facilitate the operator's convenience.
즉,， 시술자는 수술용 현미경의 접안렌즈를 통해 고배율 표면 이미지와 PAT/PAM을 통한 혈관 분포 이미지 또는 외부 조영제를 통한 특정 물질의 ：고해상도 이미지를 비침습적이며 실시간으로 동시에 볼 수 있게 된다.  In other words, the operator can view the high magnification surface image, the blood vessel distribution image through PAT / PAM, and the high resolution image of a specific substance through an external contrast agent at the same time in a non-invasive and real time through the eyepiece of the surgical microscope.
이에 본 발명은 혈관이나 뇌혈관 수술 등을 시술하는 시술자의 편이를 도모할 수 있다.  Accordingly, the present invention can facilitate the operator to perform a procedure such as blood vessel or cerebrovascular surgery.
또한 본 발명은 증강 현실 이미지에 수술에 필요한 정보를 포함시켜， 시술자가 시술중에 시선의 이동없이 현미경의 접안렌즈를 통해. 고배율 표면 이미지 및 광음향 단층 이미지는 물론이고 수술에 필요한 정보들까지도 제공받을 수 있게 하여, 시술자의 편이를 도모할 수 있는 효과가 있다.  In another aspect, the present invention includes the information necessary for surgery in the augmented reality image, the operator through the eyepiece of the microscope without movement of the eye during the procedure. The high magnification surface image and the photoacoustic tomography image as well as the information necessary for the operation can be provided, there is an effect that can facilitate the operator.
【도면의 간단한 설명】 [Brief Description of Drawings]
1은 종래의 수술용 현미경의 개략적인 구조도.  1 is a schematic structural diagram of a conventional surgical microscope.
도 2는 본 발명의 바람직한 게 1실시예에 따른 고배율 표면 이미지에 광음향 단층 이미지를 포함하는 증강 현실 이미지를 중첩시켜 출력하는 수술용 현미경 시스템의 구조도.  2 is a structural diagram of a surgical microscope system for superimposing and outputting an augmented reality image including a photoacoustic tomography image on a high magnification surface image according to an embodiment of the present invention.
도 3은 본 발명의 바람직한 제 1실시예에 따른 컴퓨팅 장치의 처리 절차도 . i  3 is a process diagram of a computing device according to a first preferred embodiment of the present invention. i
도 4 및 도 5는 본 발명의 바람직한 제 1실시예에 따른 고배율 표면 이미지에 광음향 단층 이미지를 중첩시켜 표시한 예를 도시한 도면 .  4 and 5 are views showing an example in which a photoacoustic tomographic image is superimposed and displayed on a high magnification surface image according to a first preferred embodiment of the present invention.
도 6은 본 발명의 바람직한 제 2실시예에 따른 고배율 표면 이미지에 광음향 ：단층 이미지를 포함하는 증강 현실 이미지를 중첩시켜 출력하는 수술용 미경 시스템의 구조도.  6 is a structural diagram of a surgical microscopy system for superposing and outputting an augmented reality image including an optoacoustic: tomographic image to a high magnification surface image according to a second preferred embodiment of the present invention.
【발명의 실시를 위한 최선의 형태】 [Best form for implementation of the invention]
본 발명의 바람직한 실시예들에 대한 설명에 앞서 광음향 이미지 획득장치에 대해 설명한다.  Prior to the description of the preferred embodiments of the present invention, an optoacoustic image acquisition device will be described.
일반적으로 광음향 이미지 획득장치는 광음향 현미경 (Photoacoustic microscopy; 'ΡΑΜ') 또는 광음향 단층 촬영장치 (Photoacoust ic Tomography; 'PAT' ) 증 하나이며, 본 명세서에서는 광음향 이미지 획득 장치를 TAT/PAM 장치 '로 표기한다. PAT/PAM 장치는 광의 흡수체들의 분포를 맵핑함으로써 미세혈관 이미징 및 혈중 산소 포화농도 등의 이미징이 가능하며， 이를 위해 펄스 레이저 (Pulsed Laser)를 요구한다. In general, the photoacoustic image acquisition apparatus is one of photoacoustic microscopy (ΡΡΜ) or photoacoustic tomography (PAT), and in this specification, photoacoustic image acquisition. The device is referred to as TAT / PAM device. The PAT / PAM device is capable of microvascular imaging and blood oxygen saturation by mapping the distribution of absorbers of light, and requires a pulsed laser for this purpose.
좀 더 설명하면， 상기 PAT/PAM 장치는 펄스 레이저를 생체에 입사했을 때에， 그 생체 내의 흡수체가 상기 필스 레이저를 흡수하고, 그 흡수한 에너지가 열로， 다시 그 열이 다시 압력으로 변환될 때에 발생되는 광음향 신호를 트랜스듀서로 획득하여 이미징한다.  More specifically, the PAT / PAM device is generated when a pulsed laser enters a living body, the absorber in the living body absorbs the Fils laser, and the absorbed energy is converted into heat, and the heat is converted into pressure again. The resulting photoacoustic signal is acquired by the transducer and imaged.
즉， PAT/PAM 장치는 광학 영상법과 초음파 영상법을 결합한 기술로써 비침습 , 저렴함， 휴대성 우수한 명암대조비 및 우수한 공간분해능을 두루 갖추고 있기에 차세대 고분해능 의료영상 기술로 많은 관심을 받고 있다.  In other words, PAT / PAM device is a technology that combines optical imaging and ultrasonic imaging method has attracted much attention as the next generation high resolution medical imaging technology because it has a non-invasive, inexpensive, portable contrast contrast ratio and excellent spatial resolution.
그에 대한 응용으로는, 혈관 분포 및 산호 포화 농도 정보를 획득하여 조기 암진단 연구가 진행중이며， 안구내 혈관 분포 이미지까지 획득이 쟈능하여， 다양한 안구내 정보 획득에 관한 연구도 진행중이다.  As an application for this, early cancer diagnosis research is being conducted by acquiring vascular distribution and coral saturation concentration information, and the acquisition of various intraocular information is also in progress since it is possible to acquire intravascular intravascular distribution images.
또한 상기 PAT/PAM 장치는 혈색소, 멜라닌, DNA, 유기염료 및 색채를 띤 생성물을 방출하도록 조작된 유전자와 같은 와부 조영제들 덕분에, 다른 기술에서는 주변과 흔돈되어 판별이 쉽지 않았던 림프절과 같은 조직들도 이미징할 수 있다. 이에 본 발명은 수술용 현미경에 의해 획득된 고배율 표면 이미지내에 PAT/PAM 장치로부터의 광음향 단층 이머지를 포함하는 증강 현실 이미지를 증첩시켜 출력하여， 시술자가 시술중에 시선의 이동없이 현미경의 접안렌즈를 통해 고배율 표면 이미지 및 광음향 단층 이미지를 동시에 실시간으로 제공받을 수 있게 하여 시술자의 편이를 도모한다.  The PAT / PAM device also supports tissues such as lymph nodes, which have been confused with the surroundings and difficult to identify in other technologies, thanks to eddy contrast agents such as hemoglobin, melanin, DNA, organic dyes, and genes engineered to release colored products. It can also image. Accordingly, the present invention overlaps and outputs an augmented reality image including an optoacoustic tomographic image from a PAT / PAM device in a high magnification surface image obtained by a surgical microscope, thereby allowing the operator to move the eyepiece of the microscope without moving the eye during the procedure. Through this, high magnification surface image and photoacoustic tomography image can be provided at the same time to facilitate the operator's convenience.
즉， 시술자는 수술용 현미경의 접안렌즈를 통해 고배율 표면 이미지와 PAT/PAM을 통해 획득한 혈관 분포 이미지 또는 외부 조영제를 통한 특 ¾ 물질의 고해상도 이미지를 비침습적이며 실시간으로 볼 수 있게 된다.  In other words, the operator can view the high-resolution surface image, the blood vessel distribution image acquired through PAT / PAM, or the high resolution image of the special ¾ material through an external contrast agent through non-invasive and real time through the eyepiece of the surgical microscope.
이에 본 발명은 혈관이나 뇌혈관 수술 등을 시술하는 시술자의 편이를 도모할 수 있다.  Accordingly, the present invention can facilitate the operator to perform a procedure such as blood vessel or cerebrovascular surgery.
또한 본 발명은 증강 현실 이미지에 수술에 필요한 정보를 포함시켜， 시술자가 시술증에 시선의 이동없이 현미경의 접만렌즈를 통해 고배율 표면 이미지 및 광음향 단층 이미지는 물론이고 수술에 필요한 정보들까지도 제공받을 수 있게 하여， 시술자의 편이를 도모한다. In addition, the present invention includes the information necessary for the operation in the augmented reality image, the operator to a high magnification surface through the microscope lens of the lens without shifting the eye to the procedure Images and optoacoustic tomography images as well as the information necessary for the operation can be provided to facilitate the operator's convenience.
<제1실시예 > <First Embodiment>
상기한 본 발명의 제 1실시예에 따르는 고배율 표면 이미지에 광음향 단층 이'미지를 포함하는 증강 현실 이미지를 중첩시켜 출력하는 수술용 현미경 시스템의 구조를 도 2를 참조하여 상세히 설명한다. See photoacoustic tomography is the structure of Figure 2 a surgical microscope system for outputting to "overlap the augmented reality image that includes the high magnification image in the surface image in accordance with the first embodiment of the present invention will be described in detail.
상기 수술용 현미경 시스템은 수술용 현미경 장치 (100)와 PAT/PAM 장치 (200)와 PAT/PAM 스캐닝부 (300)와 디스플레이부 (320)와 컴퓨팅 장치 (400)로 구성된다.  The surgical microscope system includes a surgical microscope apparatus 100, a PAT / PAM apparatus 200, a PAT / PAM scanning unit 300, a display unit 320, and a computing device 400.
상기 수술용 현미경 장치 (100)는 광원과 광케이블， 접안렌즈， 대물렌즈 등으로 구성되며, 대상물 (500)에 광을 조사하고， 대상물 (500)에 의해 반사된 광을 확대하여 접안렌즈를 통해 출력한다.  The surgical microscope device 100 includes a light source, an optical cable, an eyepiece, an objective lens, and the like, irradiates light onto the object 500, enlarges the light reflected by the object 500, and outputs the light through the eyepiece. do.
상기 PAT/PAM . 장치 (200)는 광음향 신호 생성을 위한 제 1광을 생성하여 생성하여 PAT/PAM 스캐닝부 (300)에 제공함과 아울러, 상기 PAT/PAM 스캐닝부 (300)가 제공하는 광음향 신호를 처리하여 광음향 단층 이미지 정보를 생성하여 컴퓨팅 장치 (400)로 제공한다. 상기 제 1 광은 대상물에 입사되어 광음향 신호를 생성하기 위한 것으로서， 광대역 파장 펄스 레.이저， 펄스 레이저， 연속파 레이저 (CW laser) ， 필스 마이크로파 중 하나로 i구성될 수 있다. 여기서, 상기 펄스 마이크로파는 1GHz에서 30GHz까지의 주파수를 가지는 전자기파이다. 상기 PAT/PAM 스캐닝부 (300)는 수술용 현미경 장치. (100)로부터의 게 2 광은 대상물 (500)에 제공되어 반사되어 다사 수술용 현미경 장치 (100)로 전달되도록 통과시키고 상기 PAT/PAM 장치 (200)로부터의 게 1 광은 대상물에 대한 2차원 또는 3차원 촬상을 위해 대상물을 스캐닝할 수 있도톡 광경로를 변경하면서 대상물 (500)에 제공하며， 그 대상물 (500)에 의해 발생되는 광음향 신호를 수신하여 상기 PAT/PAM 장치 (200)로 제공한다. 상기 PAT/PAM 스캐닝부 (300)는 광경로 변경부 (302),. 오브젝티브 렌즈 (objective lens )(308), 광경로 유도부 (310) 및 광음향 신호 획득 장치 (312)를 구비한다. PAT / PAM. The apparatus 200 generates and generates a first light for generating an optoacoustic signal and provides the generated light to the PAT / PAM scanning unit 300, and processes the optoacoustic signal provided by the PAT / PAM scanning unit 300. Optoacoustic tomography image information is generated and provided to the computing device 400. The first light is incident on an object to generate an optoacoustic signal, and may be configured as one of a broadband wavelength pulse laser, a pulse laser, a CW laser, and a microwaves. Here, the pulsed microwave is an electromagnetic wave having a frequency from 1 GHz to 30 GHz. The PAT / PAM scanning unit 300 is a surgical microscope device . Crab 2 light from 100 is provided to object 500 to be reflected and passed to the multi-surgical microscope device 100 for transmission, and crab light from PAT / PAM device 200 is two-dimensional to the object. Alternatively, the target 500 is provided to the object 500 while changing an optical path to scan the object for 3D imaging, and receives the photoacoustic signal generated by the object 500 to the PAT / PAM device 200. to provide. The PAT / PAM scanning unit 300 is an optical path changing unit 302 ,. An objective lens 308, an optical path guide unit 310, and an optoacoustic signal acquisition device 312.
다이크로닉 미러 (dicronic mirror )(310) , 갈바노메터 스캐너 (galvano scnner)(302), 오브젝티브 렌즈)와 초음파 트랜스듀서 (ultrasonic transducer)(312)로 구성된다. Dichroic mirror (310), galvano scanner (galvano scnner 302, an objective lens and an ultrasonic transducer 312.
상기 광경로 변경부 (302)는 갈바노메터 스캐너 , 회전다면경 (polygon) 스캐너 및 MEMSOnicroelectromechanical system) 스캐너 중 하나로 구성될 수 있다. 본 실시예에서는 갈바노메터 스캐너로 구성하였으며， 갈바노메터 스캐너는 PAT/PAM 장치 (200)로부터의 제 1 광을 대상물에 대한 2차원 또는 3차원 이미지 획득을 위해 대상물을 스캐닝할 수 있도톡 제 1 광에 대한 광경로를 X-Y 방향으로 변경시키면서 상기 광경로 유도부 (310)로 제공한다. 또한 상기 광경로 유도부 (310)는 다이크로익 미러 또는 범 스플리터로 구성될 수 있다. 다이크로닉 미러 또는 빔 스플리터로 구성된 광경로 ：유도부는 특정 파장의 광에 대해서만 반사하는 소재로서， 대상물 ( θθ)과 수술용 현미경 장치 (100)의 대물렌즈 사이에 위치하여， 수술용 현미경 장치 (100)로부터의 제 2 광을 대상물 (500)로 전달함과 아을러 대상물 (500)에 의해 반사된 제 2 광은 수술용 현미경 장치 (100)로 제공하며， 수술용 현미경 장치 (100)로부터의 광 입사각과는 다른 입사각 예를들어 90도로 ：제공되는 갈바노메터 스캐너 (302)로부터의 제 1 광은 반사시켜 대상물 (500)로 제공한다.  The light path changing unit 302 may be configured as one of a galvanometer scanner, a polygon scanner, and a MEMSOnicroelectromechanical system scanner. In the present embodiment, a galvanometer scanner is configured, and the galvanometer scanner can scan an object to acquire a two-dimensional or three-dimensional image of the first light from the PAT / PAM device 200. The optical path is provided to the optical path guide unit 310 while changing the optical path to one light in the XY direction. In addition, the light path guide unit 310 may be composed of a dichroic mirror or a pan splitter. Optical path consisting of a dichroic mirror or beam splitter: The induction part is a material that reflects only light of a specific wavelength, and is located between the object θθ and the objective lens of the surgical microscope device 100, Delivering the second light from the object 100 to the object 500 and the second light reflected by the Arler object 500 is provided to the surgical microscope device 100, from the surgical microscope device 100. An angle of incidence different from the angle of incidence of light, for example: 90 degrees: The first light from the provided galvanometer scanner 302 is reflected and provided to the object 500.
상기 오브젝티브 렌즈 (308)는 상기 광경로 변경부 (302)로부터의 제 1 광을 광경로 유도부에 포커싱한다 .  The objective lens 308 focuses the first light from the light path changing unit 302 on the light path inducing unit.
상기 광음향 신호 획득 징-치 (312)는 대상물 (500)이 제 1광을 흡수하고， 그 흡수한 에너지가 열로, 그 열이 다시 압력으로 변환될 때에 발생되는 광음향 신호를 획득하여 상기 PAT/PAM 장치 (200)로 제공한다. 상기 광음향 신호 획득 장치는 초음파 트랜스듀서와 같은 접촉식의 트랜스듀서, 비접촉식의 광음향 신호 획득 장치 등으로 구성될 수 있다. 상기 비접촉식의 광음향 신호 획득 장치는 페브리 -페롯 (Febry— perot) 간섭계 또는 빛간섭단층촬영 (Optical Coherent Tomography: OCT' ) 간섭계를 이용한 비접촉식의 광음향 신호 획득 장치인 것이 바람직하다.  The photoacoustic signal acquisition gong-chi 312 acquires the optoacoustic signal generated when the object 500 absorbs the first light, and the absorbed energy is converted into heat, and the heat is converted into pressure. Provided with the / PAM device 200. The photoacoustic signal acquisition device may be configured of a contact transducer such as an ultrasonic transducer, a non-contact photoacoustic signal acquisition device, or the like. The non-contact photoacoustic signal acquisition device is preferably a non-contact photoacoustic signal acquisition device using a Febri-Perot interferometer or an optical coherent tomography (OCT ') interferometer.
상기 컴퓨팅 장치 (400)는 상기 PAT/PAM 장치 (200)가 제공하는 광음향 단층 이미지 정보와 조작자에 의해 입력된 수술 관련 정보에 대한 이미지 정보를 중첩시킨 증강 현실 이미지 정보를 생성하고, 이 증강 현실 이미지 정보를 디스플레이부 (320)로 제공한다.  The computing device 400 generates augmented reality image information superimposed on the photoacoustic tomography image information provided by the PAT / PAM device 200 and the image information on the surgery-related information input by the operator, the augmented reality The image information is provided to the display unit 320.
상기 디스플레이부 (320)는, 수술용 현미경 장치 (100)로부터의 제 1 광은 대상물 (500)에 제공되어 반사되어 다시 수술용 현미경 장치 (100)로 전달되도록 통과시킴과 아울러， 상기 증강 현실 이미지는 빔으로 변환하여 수술용 현미경 장치 (100)의 광 입사경로로 중첩시켜 입사시킨다. The display unit 320 is a first from the surgical microscope device 100 The light is provided to the object 500 to be reflected and passed back to the surgical microscope device 100, and the augmented reality image is converted into a beam and overlapped with the light incident path of the surgical microscope device 100. Let it enter.
상기한 디스플레이부 (320)는 빔 프로젝터 (322)와 빔 스플리터 (beam splitter)(324)로 구성된다.  The display unit 320 includes a beam projector 322 and a beam splitter 324.
상기 범 프로젝터 (322)는 상기 컴퓨팅 장치 (400)로부터의 증강 현실 이미지 정보를 빔으로 변환하여 출사한다.  The pan projector 322 converts augmented reality image information from the computing device 400 into a beam and emits the beam.
상기 빔 스플리터 (324)는 상기 수술용 현미경 장치 (100)의 광 출사 및 입사면의 일부 영역에 위치하여， 입사된 광을 특정 비율로 나누어 일부는 통과시키고 일부는 반사시키는 것으로, 상기 증강 현실 이미지 정보에 따른 빔을 제공받아 그 빔 증 일부를 상기 수술용 현미경 장치 (100)의 광 입사경로로 제공한다.  The beam splitter 324 is located in a part of the light exit and incident surface of the surgical microscope device 100, and divides the incident light by a certain ratio to partially pass and partially reflect the augmented reality image. The beam is provided according to the information, and a portion of the beam increment is provided as a light incident path of the surgical microscope device 100.
이에 따라 수술용 현미경 장치 (100)의 접안 렌즈를 통해 출력되는 이미지는 수술용 현미경 장치 (100)에 의해 획득된 고배율 표면 이미지와 상기 ΡΑΊ7ΡΑΜ 장치 (200) 및 컴퓨팅 장치 (400)로부터의 증강 현실 이미지를 포함하게 된다.  Accordingly, the image output through the eyepiece of the surgical microscope device 100 is a high magnification surface image obtained by the surgical microscope device 100 and augmented reality image from the ΡΑΊ7ΡΑΜ device 200 and the computing device 400 It will include.
상기 컴퓨팅 장치 (400)의 동작을 도 3의 흐름도를 참조하여 좀 더 설명한다.  The operation of the computing device 400 will be further described with reference to the flowchart of FIG. 3.
상기 컴퓨팅 장치 (400)는 PAT/PAM 장치 (200)로부터 광음향 단층 이미지 정보가 수신됨과 아울러 조작자에 의해 . 수술 관련 정보가 입력되면 (700/702단계)， 광음향 단층 이미지 정보와 대상물에 대한 수술 관련 정보를 결합하여 증강 현실 이미지 정보를 생성하고, 이 생성된 증강 현실 ：이미지 정보를 디스플레이부 (320)의 빔 프로젝터 (322)로 제공한다 (704단계). 이로서 수술용 현미경 장치 (100)의 접안 렌즈로는 고배율 표면 이미지와 광음향 단층 이미지 정보 및 수술 관련 정보에 따른 증강 현실 이미지가 동시에 출력된다.  The computing device 400 receives optoacoustic tomographic image information from the PAT / PAM device 200 and is also operated by an operator. When the surgery-related information is input (step 700/702), the photoacoustic tomography image information and the surgery-related information about the object are combined to generate augmented reality image information, and the generated augmented reality: image information is displayed on the display unit 320. To the beam projector 322 (step 704). As a result, a high magnification surface image, photoacoustic tomography image information, and augmented reality image according to surgery-related information are simultaneously output to the eyepiece of the surgical microscope device 100.
상기한 바와 달리 PAT/PAM 장치 (200)로부터 광음향 단층 이미지 정보가 수신되는 상태에서 조작자에 의해 수술 관련 정보가 입력되지 않으면 (700， 702단계), 광음향 단층 이미지 정보에 따르는 증강 현실 이미지 정보를 생성하고, 이 생성된 증강 현실 이미지 정보를 디스플레이부 (320)의 빔 프로젝터 (322)로 제공한다 (706단계). 이로서 수술용 현미경 장치 (100)의 접안 렌즈로는 고배율 표면 이미지와 광음향 단층 이미지 정보에 따른 증강 현실 이미지가 동시에 출력된다. Unlike the above, when the operation related information is not input by the operator in the state in which the optoacoustic tomography image information is received from the PAT / PAM device 200 (steps 700 and 702), the augmented reality image information according to the optoacoustic tomography image information Next, the generated augmented reality image information is provided to the beam projector 322 of the display 320 (step 706). Thus, the eyepiece of the surgical microscope device 100 as augmentation according to the high magnification surface image and photoacoustic tomography image information The reality image is output at the same time.
상기한 본 발명의 바람직한 제 1실사예에 따른 수술용 현미경 시스템의 수술용 현미경 장치 (100)로 출력되는 이미지를 예시한 것이 도 4이다. ；  4 illustrates an image output to the surgical microscope device 100 of the surgical microscope system according to the first preferred embodiment of the present invention. ;
상기 도 4는 머리카락 4가닥 (H1~H4)과 주사 바늘 (N)을 배치한 팬텀을 본 발명의 바람직한 실시예에 따른 수술용 현미경 시스템으로 이미징한 것으로, 수술용 현미경 장치 (100)의 접안렌즈로는 머리카락 4가닥 (H1 H4)과 주사 바 (N)이 고배율로 확대된 고배율 표면 이미지와 2차원 광음향 단충 이미지가 함께 표시된다.  4 is an image of a phantom in which four hairs (H1 to H4) and a needle (N) are arranged with a surgical microscope system according to a preferred embodiment of the present invention, and the eyepiece of the surgical microscope device 100 is shown in FIG. The furnace displays a high-magnification surface image with a high magnification of four strands of hair (H1 H4) and an injection bar (N) together with a two-dimensional photoacoustic short image.
상기 고배율 표면 이미지에는 주사기와 머리카락 사이의 높아 차이가 없는 것으로 나타나나， 2차원 광음향 단층 이미지에는 주사기와 머리카락의 높이 차이가 나타나게 표시된다. 상기 2차원 광음향 단층 이미지에는 도 5에 도시한 바와 같이 머리카락 4가닥 (H1~H4)과 주사 바늘 (N) 사이의 높이 차이가 명확하게 드러난다. The high magnification surface image shows no difference between the height of the syringe and the hair, but the height difference between the syringe and the hair appears on the two-dimensional photoacoustic tomography image. In the two-dimensional photoacoustic tomography image, as shown in FIG. 5, the height difference between the four hairs H1 to H4 and the injection needle N is clearly revealed.
상기한 본 발명의 바람직한 제 1실시예에서는 PAT/PAM 장치로부터의 광음향 단층 이미지 정보를 컴퓨팅 장치를 통해 빔 프로젝터에 제공하는 것만을 예시하였으나， 상기 PAT/PAM 장치로부터의 광음향 단층 이미지 정보를 빔 프로젝터로 직접 제공할 수도 있다.  In the first preferred embodiment of the present invention, only the photoacoustic tomography image information from the PAT / PAM device is provided to the beam projector through the computing device, but the photoacoustic tomography image information from the PAT / PAM device is provided. It can also be provided directly to the beam projector.
또한 상기한 본 발명의 바람직한 제 1실시예에서는 빔 프로젝터로부터의 빔을 수술용 현미경 장치의 광 입사면의 일부영역으로 제공하도록 구성되어 있으며, 그 영역의 크기 및 위치는 시술자의 편이를 도모하기 위해 변경될 수 있음은 본 발명에 의해 자명하다.  In addition, in the first preferred embodiment of the present invention, the beam from the beam projector is configured to provide a partial region of the light incident surface of the surgical microscope device, and the size and position of the region is for the convenience of the operator. It is obvious by the present invention that it can be changed.
또한 상기한 본 발명의 바람직한 게 1실시예에서는 수술용 현미경 장치와 PAT/PAM 장치와 PAT/PAM 스캐닝부와 디스플레이부와 컴퓨팅 장치가 결합되어 수술용 현미경 시스템을 구성한 것만을 예시하였으나， 상기 수술용 현미경 시스템은 기존의 수술용 현미경 장치에 PAT/PAM 장치와 PAT/PAM 스캐닝부와 디스플레이부와 컴퓨팅 장치를 결합하여 구성될 수도 있다. 어는 기존에 구비된 수술용 현미경 장치를 교체하자 않으면서도 본 발명에 따른 증강 현실 이미지를 제공받을 수 있게 한다. 상기 PAT/PAM 장치와 PAT/PAM 스캐닝부와 디스플레이부와 컴퓨팅 장치는 증강 현실 이미지 제공 장치로 명명한다. 또한 상기한 본 발명의 제 1실시예에서는 2차원 광음향 단층 이미지를 고배율 표면 이미지에 중첩하여 표시하는 것만을 예시하였으나, PAT/PAM 장치를 ：통해 획득될 수 있는 3차원 광음향 이미지 또는 MAP(maximum amplitude projection) 이미지를 고배율 표면 이미지에 중첩하여 표시할 수도 있으며， 이는 본 발명에 의해 자명하다. In addition, in the preferred embodiment of the present invention described above, only a surgical microscope device, a PAT / PAM device, a PAT / PAM scanning unit, a display unit, and a computing device are combined to constitute a surgical microscope system. The microscope system may be configured by combining a PAT / PAM device, a PAT / PAM scanning unit, a display unit, and a computing device to an existing surgical microscope device. It is possible to be provided with an augmented reality image according to the present invention without replacing the surgical microscope device provided in advance. The PAT / PAM device, the PAT / PAM scanning unit, the display unit, and the computing device are referred to as an augmented reality image providing device. In addition, in the first embodiment of the present invention described above, only the two-dimensional photoacoustic tomographic image is superimposed and displayed on the high magnification surface image, but the three-dimensional photoacoustic image or MAP (which can be obtained through the PAT / PAM device) is displayed. A maximum amplitude projection image may be superimposed on a high magnification surface image, which is apparent by the present invention.
또한 상기한 본 발명의 바람직한 제 1실시예에서는 광음향 신호 획득장치로서 젤이나 물 등과 같은 매개체를 통해 피부와 접촉하여 광음향 신호를 획득하는 접촉식 장치로서 초음파 트랜스듀서만을 개시하였으나， 피부와 비접촉하여 광음향 신호를 획득하는 광학적인 방식의 광음향 신호 획득장치가 채용될 수도 있으며, 이는 본 발명에 의해 자명하다..  In addition, in the first preferred embodiment of the present invention, as the photoacoustic signal acquisition device, only the ultrasonic transducer is disclosed as a contact type device which obtains the photoacoustic signal by contacting the skin through a medium such as gel or water. Optical photoacoustic signal acquisition device of the optical system for acquiring the photoacoustic signal may be employed, which is obvious by the present invention.
【발명의 실시를 위한 형태】 [Form for implementation of invention]
^제 2실시예 >  ^ Second Embodiment>
본 발명의 바람직한 제 2실시예에 따르는 고배율 표면 이미지에 광음향 단층 이미지를 포함하는 증강 현실 이미지를 중첩시켜 출력하는 수술용 현미경 시스템의 구조를 도 6을 참조하여 상세히 설명한다.  A structure of a surgical microscope system for superposing and outputting an augmented reality image including an optoacoustic tomographic image on a high magnification surface image according to a second preferred embodiment of the present invention will be described in detail with reference to FIG. 6.
상기 수술용 현미경 시스템은 수술용 현미경.. 장치 (900)와 PAT/PAM 장치 (200)와 PAT/PAM 스캐닝부 (300)와 디스플레이부 (800)와 컴퓨팅 장치 (400)로 구성된다. 상기 PAT/PAM 장치 (200)와 PAT/PAM 스캐닝부 (300)와 컴퓨팅 장치 (400)의 구성 및 동작은 제 1실시예와 동일하므로 그 상세한 설명은 생략한다.  The surgical microscope system comprises a surgical microscope device 900, a PAT / PAM device 200, a PAT / PAM scanning unit 300, a display unit 800 and a computing device 400. Since the configurations and operations of the PAT / PAM device 200, the PAT / PAM scanning unit 300, and the computing device 400 are the same as those of the first embodiment, detailed descriptions thereof will be omitted.
상기 수술용 현미경 장치 (900)는 접안렌즈 (902)와 대물렌즈 (904)를 구비하 ^ 일반적인 현미경 구조를 가지나, 다만 상기 접안렌즈 (902)의 상면 중 일부영역에 디스플레이 장치 (800)를 내장한다ᅳ . 여기서， 상기 접안렌즈 (902)의 상면은 사용자의 눈과 맞닿아 사용자에게 이미지를 출력하는 면을 일컫는다.  The surgical microscope device 900 includes an eyepiece 902 and an objective lens 904, which have a general microscope structure, but have a display device 800 embedded in a portion of an upper surface of the eyepiece 902. ᅳ Here, the upper surface of the eyepiece 902 refers to the surface in contact with the eyes of the user to output an image to the user.
상기 디스플레이 장치 (800)는 소형 디스플레이 모들 (802)로 구성되며， 상기 ^형 디스플레이 모듈 (802)은 수술용 현미경 장치 (900)의 접안렌 ^(902)의 상면 중 미리 정해진 일부영역에 위치하여 컴퓨팅 장치 (400)로부터 증강 현실 이미지 정보를 제공받아 표시한다.  The display device 800 is composed of a small display mode 802, the ^ -type display module 802 is located in a predetermined partial region of the upper surface of the eyepiece ^ (902) of the surgical microscope device 900 The augmented reality image information is received from the computing device 400 and displayed.
이로서， 상기 수술용 현미경 장치 (.900)는 접안렌즈 (902) 및 대물렌즈 (904)에 의해 획득된 고배율 표면 이미지와 상기 소형 디스플레이 모들 (802)을 통해 표시되는 정보를 동시에 출력할 수 있다. 상기한 본 발명의 바람직한 실시예에서는 상기 소형 디스플레이 모들 (802)이 상기 컴퓨팅 장치 (400)로부터 제공되는 증강 현실 이미지 정보를 표시하는 것만을 개시하였으나 , 상기 컴퓨팅 장치 (400)가 아닌 PAT/PAM 장치 (200)로부터 광음향 단층 이미지 정보를 직접 제공받아 표시할 수도 있으며, 이는 본 발명에 의해 자명하다. As such, the surgical microscope device (.900) is a compact display and a high magnification surface image obtained by the eyepiece 902 and the objective lens 904. Information displayed through the modules 802 may be output at the same time. In the above-described preferred embodiment of the present invention, the small display modules 802 only display the augmented reality image information provided from the computing device 400, but the PAT / PAM device other than the computing device 400. Optoacoustic tomography image information may be directly received from 200 and displayed, which is obvious by the present invention.
^한 상기 소형 디스플레이 모들 (802)의 설치위치는 가능한 사용자의 가시영역내이면서도 상기 접안렌즈 (902)를 통해 출력되는 고배율 표면 이미지를 층분히 관찰할 수 있을 정도의 위치로 선택될 수 있으며， 이는 본 발명에 의해 자명하다.  The mounting position of the small display mode 802 may be selected such that a high magnification surface image output through the eyepiece 902 can be observed while being within the visible range of the user. It is obvious by the present invention.
【산업상 이용가능성】 Industrial Applicability
본 발명에 따른 장치는 수술용 현미경 장치에 널리 사용될 수 있다.  The device according to the invention can be widely used in surgical microscope devices.

Claims

【청구의 범위】 [Range of request]
【청구항 11  [Claim 11
수술용 현미경 장치를 위한 증강 현실 이미지 제공 장치에 있어서， 광음향 신호 생성을 위한 게 1 광을 생성하여 출력하며， 대상물로부터 출력된 광음향 신호를 제공받아 광음향 이미지를 생성하는 광음향 이미지 획득 장치;  An apparatus for providing augmented reality images for a surgical microscope device, the apparatus for generating an optoacoustic signal, which generates and outputs one light, and receives an optoacoustic signal output from an object to generate an optoacoustic image. ;
상기 수술용 현미경 장치로부터 제공되는 제 2 광을 통과시켜 상기 대상물로 전달하고, 상기 대상물에 의해 반사된 제 2 광을 통과시켜 상기 수술용 현미경 장치로 제공하며，  Passing the second light provided from the surgical microscope device to the object, passing the second light reflected by the object to provide the surgical microscope device,
상기 대상물을 스캐닝할 수 있도록 상기 제 1 광을 상기 대상물로 제공하고， 상기 대상물에 입사된 게 1 광에 의해 생성된 광음향 신호를 획득하여 상기 광음향 이미지 획득 장치로 제공하는 스캐닝부;  A scanning unit providing the first light to the object to scan the object, and acquiring an optoacoustic signal generated by the first light incident on the object to provide the optoacoustic image acquisition device;
상기 광음향 이미지 획득 장치에 의해 생성된 광음향 이미지에 따른 빔을 생성하여 상기 수술용 현미경 장치로 제공하는 디스플레이부;  A display unit generating a beam according to the photoacoustic image generated by the photoacoustic image acquisition device and providing the beam to the surgical microscope device;
를 포함하는 것을 특징으로 하는 수술용 현미경 장치를 위한 증강 현실 이미지 제공 장치.  Augmented reality image providing device for a surgical microscope device comprising a.
【청구항 2】 [Claim 2]
수술용 현미경 장치를 위한 증강 현실 이미지 제공 장치에 있어.서, 광음향 신호 생성을 위한 제 1 광을 생성하여 출력하며, 대상물로부터 출력된 광음향 신호를 제공받아 광음향 이미지를 생성하는 광음향 이미지 획득 장 ; In augmented reality image providing device for surgical microscope device . A photoacoustic image acquisition method for generating and outputting a first light for generating an optoacoustic signal, and receiving an optoacoustic signal output from an object to generate an optoacoustic image;
상기 수술용 현미경 장치로부터 제공되는 게 2 광을 통과시켜 상기 대상물로 전달하고, 상기 대상물에 와해 반사된 제 2 광을 통과시켜 상기 수술용 현미경 장치로 제공하며，  What is provided from the surgical microscope device passes through the two light to the object, passing through the second light reflected by the object through the surgical microscope device,
상기 대상물을 스캐닝할 수 있도록 상기 제 1 광을 상기 대상물로 제공하고， 상기 대상물에 입사된 제 1 광에 의해 생성된 광음향 신호를 획득하여 상기 광음향 이미지 획득 장치로 제공하는 스캐닝부;  A scanning unit providing the first light to the object to scan the object, and acquiring an optoacoustic signal generated by the first light incident on the object to provide the optoacoustic image acquisition device;
상기 광음향 이미지 획득 장치로부터 광음향 이미지를 제공받고， 상기 광음향 이미지에 상기 광음향 이미지와 관련된 정보를 포함시켜 증강 현실 이미지를 생성하여 출력하는 컴퓨팅 장치;  A computing device configured to receive an optoacoustic image from the optoacoustic image acquisition device and to generate and output an augmented reality image by including information related to the optoacoustic image in the optoacoustic image;
상기 컴퓨팅 장치로부터 제공된 증강 현실 이미지에 따른 빔을 생성하여 상기 수술용 현미경 장치로 제공하는 디스플레이부; 를 구비하는 것을 특징으로 하는 수술용 현미경 장치를 위한 증강 현실 이미지 제공 장치. A beam according to the augmented reality image provided from the computing device. A display unit for generating and providing the surgical microscope device; Augmented reality image providing device for a surgical microscope device comprising a.
【청구항 3】 [Claim 3]
제 2항에 있어서, 상기 증강 현실 이미지에 포함된 광음향 이미지 관련 정보는 상기 컴퓨팅 장치로 입력된 대상물에 대한 수술 관련 정보인 것을 특징으로 하는 수술용 현미경 장치를 위한 증강 현실 이미지 제공 장치 ·  The apparatus of claim 2, wherein the photoacoustic image related information included in the augmented reality image is surgery related information about an object input to the computing device.
【청구항 4】 [Claim 4]
제 1항 및 제 2항 중 어느 한 항에 있어서， 상기 제 1 광은 광음향 신호 생성이 가능한 펄스 레이저, 광대역 파장 펄스 레이저， 연속파 레이저 및 펄스 마이크로파 중 하나인 것을 특징으로 하는 수술용 현미경 장치를 위한 증강 현실 이미지 제공 장치.  The surgical microscope apparatus according to any one of claims 1 to 3, wherein the first light is one of a pulse laser, a broadband wavelength pulse laser, a continuous wave laser, and a pulse microwave capable of generating an optoacoustic signal. Augmented reality image providing device for.
【청구항 5】 [Claim 5]
제 1항 및 제 2항 중 어느 한 항에 있어서，  The method according to any one of claims 1 and 2,
상기 광음향 이미지 획득 장치는 광음향 현미경 (Photoacoustic microscopy; 'ΡΑΜ' ) 또는 광음향 단층 촬영장치 (Photoacoust ic Tomography; 'ΡΑΤ') 중 하나인 것을 특징으로 하는 수술용 현미경 장치를 위한 증강 현실 이미지 제공 장치ᅳ  The photoacoustic image acquisition device is one of a photoacoustic microscopy ('ΡΑΜ') or a photoacoustic tomography apparatus (Photoacoust ic Tomography; 'ΡΑΤ') providing an augmented reality image for the surgical microscope device, characterized in that Device
【청구항 6】 [Claim 6]
제 1항 및 게 2항 중 어느 한 항에 있어서, 상기 스캐닝부는， 대상물을 스캐닝할 수 있도록， 상기 광음향 이미지 획득 장치로부터 출력된 제 1 광의 경로를 변경하는 광 경로 변경부;  The apparatus of claim 1, wherein the scanning unit comprises: an optical path changing unit configured to change a path of the first light output from the optoacoustic image acquisition device so as to scan an object;
상기 수술용 현미경 장치의 대물 렌즈와 대상물의 사이에 위치하며， 상기 수술용 현미경 장치로부터 제공되는 제 2 광을 상기 대상물로 제공하고, 상기 대상물에 의해 반사된 제 2 광을 상기 수술용 현미경 장치로 제공하며，  Located between the objective lens of the surgical microscope device and the object, providing a second light provided from the surgical microscope device to the object, the second light reflected by the object to the surgical microscope device Provided by
상기 광경로 변경부로부터 제공되는 제 1 광을 유도하여 대상물로 제공하는 광경로 유도부; Induces the first light provided from the light path changing unit to the object An optical path guidance unit provided;
상기 광경로 변경부와 상기 광경로 유도부의 사이에 위치하여 제 1 광을 포커싱하여 광경로 유도부로 제공하는 오브젝트 렌즈; 및  An object lens positioned between the light path changing unit and the light path inducing unit to focus the first light and provide the first light to the light path inducing unit; And
상기 대상물로부터 출력된 광음향 신호를 획득하여 상기 광음향 이미지 획득 장치로 제공하는 광음향 신호 획득 장치 ;  An optoacoustic signal acquisition device for acquiring an optoacoustic signal output from the object and providing the optoacoustic image acquisition device;
를 포함하는 것을 특징으로 하는 수술용 현미경 장치를 위한 증강 현실 이미지 제공 장치. * Augmented reality image providing device for a surgical microscope device comprising a. *
【청구항 7】 [Claim 7]
제 6항에 있어서， 상기 광음향 이미지 획득 장치는 접촉식의 트랜스듀서 또는 비접촉식의 광음향 신호 획득 장치인 것을 특징으로 하는 수술용 ¾미경 장치를 위한 증강 현실 이미지 제공 장치.  7. The apparatus of claim 6, wherein the optoacoustic image acquisition device is a contact transducer or a non-contact optoacoustic signal acquisition device.
【청구항 8】 [Claim 8]
게 7항에 있어서， 상기 비접촉식의 광음향 신호 획득 장치는 페브리- 페롯 (Febry-perot) 간섭계 또는 빛간섭단층촬영 (Opt ical Coherent Tomography: OCT') 간섭계를 이용한 비접촉식의 광음향 신호 획득 장치인 것을 특정으로 하는 수술용 현미경 장치를 위한 증강 현실 이미지 게공 장치 .  The apparatus of claim 7, wherein the non-contact photoacoustic signal acquisition device is a non-contact photoacoustic signal acquisition device using a Febri-perot interferometer or an optical coherent tomography (OCT ') interferometer. Augmented Reality Image Processing Device for Surgical Microscope Device
【청구향 93  Claim 93
제 6항에 있어서， 상기 광경로 변경부는,  The method of claim 6, wherein the light path changing unit,
찰바노메터 스캐너， 회전 다면경 스캐너 및 MEMS(microelectro- mechanical system) 스캐너 중 하나로 구성된 것을 특징으로 하는 수술용 현미경 장치를 위한 증강 현실 이미지 제공 장치.  An augmented reality image providing device for a surgical microscope device, characterized in that consisting of one of a galvanometer scanner, a rotary multifaceted scanner and a microelectromechanical system (MEMS) scanner.
【청구항 10] [Claim 10]
겨 16항에 있어서， 상기 광경로 유도부는,  According to claim 16, The optical path guide portion,
다이크로닉 미러 및 빔 스플리터 중 하나로 구성된 것을 특징으로 하는 수술용 현미경 장치를 위한 증강 현실 이미지 제공 장치ᅳ  Augmented reality image providing device for surgical microscope device characterized in that consisting of one of the dichroic mirror and the beam splitter 터
【청구항 11】 [Claim 11]
제 1항 및 제 2항 중 어느 한 항에 있어서， 상기 디스플레이부는， The method according to any one of claims 1 and 2, The display unit,
외부로부터 제공되는 이미지를 빔으로 변환시켜 출력하는 빔 프로젝터; 및 상기 범 프로젝터로부터의 빔을 상기 수술용 현미경 장치의 광 입사면으로 반사하여 전달하는 범 스플리터;를 포함하여 구성되거나， 상기 수술용 현미경 장치의 접안 렌즈의 상면 중 미리. 정해진 위치에 위치하고， 외부로부터 제공되는 이미지를 광 입사면의 방향으로 표시하는 소형 디스플레이 모들로 구성됨을 특징으로 하는 수술용 현미경 장치를 위한 증강 현실 이미지 제공 장치.  A beam projector converting an image provided from the outside into a beam and outputting the beam; And a pan splitter which reflects and transmits a beam from the pan projector to a light incident surface of the surgical microscope device, in advance of an upper surface of the eyepiece of the surgical microscope device. An augmented reality image providing device for a surgical microscope device which is located at a predetermined position, consisting of small display modules for displaying an image from the outside in the direction of the light incident surface.
【청구항 12】 [Claim 12]
제 1항 및 제 2항 중 어느 한 항에 있어서, 상기 광음향 이미지는 2차원 또는 3차원 광음향 이미지， 2차원 또는 3차원 광음향 단층 이미지 또는 MAP maximum am litude projection) 이미지임을 특징으로 하는 수술용 현미경 장치를 위한 증강 현실 이미지 제공 장치.  The surgery according to claim 1 or 2, wherein the optoacoustic image is a two-dimensional or three-dimensional optoacoustic image, a two-dimensional or three-dimensional optoacoustic tomography image, or a MAP maximum am litude projection image. Augmented reality image providing device for the microscope device for use.
【청구항 13] [Claim 13]
광음향 신호 생성을 위한 제 1 광을 출력하며， 대상물로부터 출력되는 광음향 신호를 제공받아 광음향 이미지를 생성하는 광음향 이미지 획득 장치；  An optoacoustic image acquisition device for outputting a first light for generating an optoacoustic signal, and receiving an optoacoustic signal output from an object to generate an optoacoustic image;
대상물에 게 2 광을 조사하고 그 대상물에 의해 반사된 게 2 광을 입사받아 확대하여 대상물에 대한 표면 이미지를 생성하여 출력하는 수술용 현미경 장치 ;  A surgical microscope device that irradiates 2 light onto an object and receives and enlarges 2 light reflected by the object to generate and output a surface image of the object;
대상물을 스캐닝할 수 있도록 게 1 광을 대상물로 제공하고， 상기 대상물로부터 출력되는 광음향 신호를 획득하여 상기 광음향 이미지 획득 장치로 제공하는 스캐닝부;  A scanning unit that provides one light to the object so as to scan the object, obtains an optoacoustic signal output from the object, and provides the photoacoustic signal to the optoacoustic image acquisition device;
상기 광음향 이미지 획득 장치로부터 제공되는 광음향 이미지에 따른 빔을 생성하여 상기 수술용 현미경 장치의 광 입사면으로 제공하는 디스플레이부; ' A display unit generating a beam according to an optoacoustic image provided from the optoacoustic image acquisition device and providing the beam to the light incident surface of the surgical microscope device; '
를 포함하여, 대상물에 대한 표면 이미지에 광음향 단층 이미지를 중첩시켜 출력하는 것을 특징으로 하는 수술용 현미경 시스템.  Including, Surgical microscope system, characterized in that for outputting by overlaying the photoacoustic tomography image on the surface image for the object.
【청구항 14】 광음향 신호를 생성하기 위한 제 1 광을 출력하며， 대상물로부터 출력된 광음향 신호를 제공받아 광음향 이미지를 생성하는 광음향 이미지 획득 장치; [Claim 14] An optoacoustic image acquisition device for outputting a first light for generating an optoacoustic signal, and receiving an optoacoustic signal output from an object to generate an optoacoustic image;
대상물에 제 2 광을 조사하고 그 대상물에 의해 반사된 제 2 광을 입사받아 확대하여 대상물에 대한 표면 이미지를 생성하여 출력하는 수슬용 현미경 장치;  A shoulder microscope apparatus for irradiating a second light to an object and receiving and enlarging the second light reflected by the object to generate and output a surface image of the object;
대상물을 스캐닝할 수 있도록 제 1 광을 대상물로 제공하고， 상기 대상물로부터 출력되는 광음향 신호를 획득하여 상기 광음향 이미지 획득 장치로 체공하는 스캐닝부;  A scanning unit configured to provide a first light to the object so as to scan the object, and acquire an optoacoustic signal output from the object and post the ball to the optoacoustic image acquisition device;
상기 광음향 이미지 획득 장치로부터 광음향 이미지를 제공받고, 상기 광음향 이미지에 광음향 이미지 관련 정보를 포함시켜 증강 현실 이미지 ¾ 생성하여 출력하는 컴퓨팅 장치;  A computing device configured to receive an optoacoustic image from the optoacoustic image acquisition device, and generate and output an augmented reality image ¾ by including optoacoustic image related information in the optoacoustic image;
상기 컴퓨팅 장치로부터 제공되는 증강 현실 이미지에 따른 빔을 생성하여 상기 수술용 현미경 장치의 광 입사면으로 제공하는 디스플레이부;  A display unit generating a beam according to an augmented reality image provided from the computing device and providing the beam to a light incident surface of the surgical microscope device;
를 포함하여, 대상물에 대한 표면 이미지에 광음향 단층 이미지를 포함한 증강 현실 이미지를 중첩시켜 출력하는 것을 특징으로 하는 수술용 현미경 시스템 .  Including, a surgical microscope system, characterized in that for outputting the superimposed augmented reality image including the photoacoustic tomography image on the surface image for the object.
【청구항 15】 [Claim 15]
제 13항 및 제 14항 중 어느 한 항에 있어서， 상기 스캐닝부는, 상기 수술용 현미경 장치로부터의 제 2 광을 통과시켜 상기 대상물로 전달하고， 상기 대상물에 의해 반사된 제 2 광을 통과시켜 상기 수술용 현미경 장치로 제공하며，  The method of claim 13, wherein the scanning unit passes the second light from the surgical microscope device to the object, and passes the second light reflected by the object to pass the second light. Provided as a surgical microscope device
상기 제 1 광을 상기 대상물로 전달하고， 상기 대상물에 입사된 제 1 광에 의:해 생성되는 광음향 신호를 획득하여 상기 광음향 이미지 획득 장치로 체공하는 것을 특징으로 하는 수술용 현미경 시스템.  And transmitting the first light to the object, acquiring an optoacoustic signal generated by the first light incident on the object, and passing the first light to the optoacoustic image acquisition device.
【청구항 16】 [Claim 16]
제 13항 및 제 14항 중 어느 한 항에 있어서, 상기 디스플레이부는， 상기 수술용 현미경 장치로부터의 게 2 광을 통과시켜 상기 대상물로 전달하고， 상기 대상물에 의해 반사된 제 2 광을 통과시켜 상기 수술용 현미경 장치로.제공하며， The method according to any one of claims 13 to 14, wherein the display unit, passing through the crab 2 light from the surgical microscope device to the object, passing through the second light reflected by the object to the For surgery As a microscope device.
상기 광음향 이미지에 따른 빔을 생성하여 상기 수술용 현미경 장치의 광 입사면으로 제공하는 것을 특징으로 하는 수술용 현미경 시스템.  Surgical microscope system, characterized in that for generating a beam according to the optoacoustic image to provide a light incident surface of the surgical microscope device.
【청구항 17】 [Claim 17]
제 14항에 있어서， 상기 증강 현실 이미지에 포함된 광음향 이미지 관련 정보는 상기 컴퓨팅 장치로 입력된 대상물에 대한 수술 관련 정보인 것을 특징으로 하는 수술용 현미경 시스템.  The surgical microscope system of claim 14, wherein the photoacoustic image related information included in the augmented reality image is surgery related information about an object input to the computing device.
【청구항 18】 [Claim 18]
제 13항 및 제 14항 중 어느 한 항에 있어서, 상기 제 1 광은 광음향 신호 생성이 가능한 펄스 레이저， 광대역 파장 펼스 레이저， 연속파 레이저 및 펄스 마이크로파 중 하나인 것을 특징으로 하는 수술용 현미경 시스템 .  15. The surgical microscope system of claim 13 or 14, wherein the first light is one of a pulse laser, a broadband wavelength pull laser, a continuous wave laser, and a pulsed microwave capable of generating an optoacoustic signal.
【청구항 19】 [Claim 19]
제 13항 및 제 14항 중 어느 한 항에 있어서，  The method according to any one of claims 13 and 14,
상기 광음향 이미지 획득 장치는 광음향 현미경 (Photoacoustic microscopy; 'ΡΑΜ' ) 또는 광음향 단층 촬영장치 (Photoacoust ic Tomography; 'PAT') 중 하나인 것을 특징으로 하는 수술용 현미경 시스템.  The photoacoustic image acquisition device is a surgical microscope system, characterized in that one of the photoacoustic microscopy ("ΡΑΜ") or photoacoustic tomography ("PAT").
【청구항 20] [Claim 20]
^113항 및 제 14항 중 어느 한 항에 있어서， 상기 스캐닝부는, ' 대상물을 스캐닝할 수 있도록， 상기 광음향 이미지 획득 장치로부터 출력된 제 1 광의 경로를 변경하는 광 경로 변경부;  15. The apparatus of any one of claims 113 and 14, wherein the scanning unit comprises: an optical path changing unit for changing a path of the first light output from the optoacoustic image acquisition device to scan an object;
상기 수술용 현미경 장치의 대물 렌즈와 대상물의 사이에 위치하며， 상기 수술용 현미경 장치로부터 제공되는 제 2 광을 상기 대상물로 제공하고， 상기 대상물에 의해 반사된 제 2 광을 상기 수술용 현미경 장치로 제공하며,  Located between the objective lens of the surgical microscope device and the object, providing a second light provided from the surgical microscope device to the object, the second light reflected by the object to the surgical microscope device Provide,
상기 광경로 변경부로부터 제공되는 제 1 광을 유도하여 대상물로 제공하는 광경로 유도부;  A light path inducing unit for inducing a first light provided from the light path changing unit and providing the light to an object;
상기 광경로 변경부와 상기 광경로 유도부의 사이에 위치하여 제 1 광을 포커싱하여 광경로 유도부로 제공하는 오브젝트 렌즈； 및 상기 대상물로부터 출력된 광음향 신호를 획득하여 상기 광음향 이미지 획득 장치로 제공하는 광음향 신호 획득 장치; An object lens positioned between the light path changing unit and the light path inducing unit to focus the first light and provide the first light to the light path inducing unit; and An optoacoustic signal acquisition device for acquiring the optoacoustic signal output from the object and providing the optoacoustic image acquisition device;
를 포함하는 것을 특징으로 하는 수술용 현미경 시스템.  Surgical microscope system comprising a.
【청구항 21】 [Claim 21]
제 20항에 있어서, 상기 광음향 이미지 획득. 장치는 접촉식의 트랜스듀서 또는 비접촉식의 광음향 신호 획득 장치인 것을 특징으로 하는 수술용 현미경 시스템. 21. The method of claim 20, wherein the optoacoustic image acquisition . The device is a surgical microscope system, characterized in that the contact transducer or the non-contact photoacoustic signal acquisition device.
【청구항 22】 [Claim 22]
제 21항에 있어서， 상기 비접촉식의.광음향 신호 획득 장치는 페브리- 페롯 (Febry-perot) 간섭계 또는 빛간섭단충촬영 (Opt ical Coherent Tomography: ' OCT') 간섭계를 이용한 비접촉식의 광음향 신호 획득 장치인 것을 특징으로 하는 수술용 현미경 시스템.  22. The method of claim 21, wherein the non-contact photo-acoustic signal acquisition device is a non-contact photo-acoustic signal acquisition using a Febry-Perot interferometer or OCT (Opt ical Coherent Tomography (OCT)) interferometer Surgical microscope system, characterized in that the device.
I  I
【청구항 23】  [Claim 23]
제 20항에 있어서, 상기 광경로 변경부는,  The method of claim 20, wherein the light path changing unit,
갈바노메터 스캐너， 회전 다면경 스캐너 및 MEMSG icroelectro- mechanical system) 스캐너 증 하나로 구성된 것을 특징으로 하는 수술용 현미경 스템.  Surgical microscope stems consisting of a galvanometer scanner, a rotary polyhedron scanner and a MEMSG icroelectro-mechanical system scanner.
【청구항 24】 [Claim 24]
제 20항에 있어서， 상기 광경로 유도부는,  The method of claim 20, wherein the optical path guidance unit,
이크로닉 미러 및 범 스플리터 중 하나로 구성된 것을 특징으로 하는 수술용 현미경 시스템 .  Surgical microscope system comprising one of an etronic mirror and a pan splitter.
【청구항 25】 [Claim 25]
제 13항 및 제 14항 중 어느 한 항에 있어서，  The method according to any one of claims 13 and 14,
상기 디스플레이부는,  The display unit,
쵸ᅵ부로부터 제공되는 이미지를 빔으로 변환시켜 출력하는 빔 프로젝터; 및 상기 빔 프로젝터로부터의 범을 상기 수술용 현미경 장치의 광 입사면으로 반사하여 전달하는 빔 스플리터 ;를 포함하여 구성되거나， 상기 수술용 현미경 장치의 접안 렌즈의 상면 중 미리 정해진 위치에 위치하고， 외부로부터 제공되는 이미지를 광 입사면의 방향으로 표시하는 소형 디스플레이 모들로 구성됨을 특징으로 하는 수술용 현미경 시스템 . A beam projector for converting an image provided from the cho unit into a beam and outputting the beam; And a beam splitter for reflecting and transmitting a beam from the beam projector to a light incident surface of the surgical microscope device. Surgical microscope system is characterized in that it is located in a predetermined position of the upper surface of the eyepiece of the surgical microscope device, consisting of small display modules for displaying an image provided from the outside in the direction of the light incident surface.
【청구항 25] [Claim 25]
제 13항 및 제 14항 증 어느 한 항에 있어서， 상기 광음향 이미지는 2차원 또는 3차원 광음향 이미지, 2차원 또는 3차원 광음향 단층 이미지 또는 MAP (maximum amplitude projection) 이미지임을 특징으로 하는 수술용 현미경 시스템.  The surgery according to claim 13 or 14, wherein the optoacoustic image is a two-dimensional or three-dimensional optoacoustic image, a two-dimensional or three-dimensional optoacoustic tomography image, or a MAP (maximum amplitude projection) image. Microscope system.
PCT/KR2013/007735 2013-04-25 2013-08-28 Operating microscope system of superimposing and outputting surface image and augmented reality image and apparatus of providing augmented reality image therefor WO2014175514A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR10-2013-0045970 2013-04-25
KR1020130045970A KR101418405B1 (en) 2013-04-25 2013-04-25 medical microscope system of overlapping and outputting high magnification surface image and augmented reality image having photoacoustic tomography Image and apparatus of providing augmented reality image therefor

Publications (1)

Publication Number Publication Date
WO2014175514A1 true WO2014175514A1 (en) 2014-10-30

Family

ID=51741900

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2013/007735 WO2014175514A1 (en) 2013-04-25 2013-08-28 Operating microscope system of superimposing and outputting surface image and augmented reality image and apparatus of providing augmented reality image therefor

Country Status (2)

Country Link
KR (1) KR101418405B1 (en)
WO (1) WO2014175514A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109102573A (en) * 2018-08-06 2018-12-28 百度在线网络技术（北京）有限公司 Image processing method, device and storage medium

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101749602B1 (en) * 2016-04-05 2017-06-21 포항공과대학교 산학협력단 Optical resolution photoacoustic microscopy using non-conductive fluid, and photoacoustic image acquisition system and method using the same

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20050003948A (en) * 2003-07-02 2005-01-12 가부시끼가이샤 도시바 Method and apparatus for forming an image that shows information about a subject
JP2006242816A (en) * 2005-03-04 2006-09-14 Mitsui Eng & Shipbuild Co Ltd Photoacoustic microscope device
JP4282120B2 (en) * 1998-01-06 2009-06-17 オリンパス株式会社 Surgical microscope
KR20090088909A (en) * 2006-12-19 2009-08-20 코닌클리케 필립스 일렉트로닉스 엔.브이. Combined photoacoustic and ultrasound imaging system
KR20130014338A (en) * 2011-07-28 2013-02-07 한국전자통신연구원 Image diagnosis device including x-ray image tomosynthesis device and photoacoustic image device and image diagnosis method thereof

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4282120B2 (en) * 1998-01-06 2009-06-17 オリンパス株式会社 Surgical microscope
KR20050003948A (en) * 2003-07-02 2005-01-12 가부시끼가이샤 도시바 Method and apparatus for forming an image that shows information about a subject
JP2006242816A (en) * 2005-03-04 2006-09-14 Mitsui Eng & Shipbuild Co Ltd Photoacoustic microscope device
KR20090088909A (en) * 2006-12-19 2009-08-20 코닌클리케 필립스 일렉트로닉스 엔.브이. Combined photoacoustic and ultrasound imaging system
KR20130014338A (en) * 2011-07-28 2013-02-07 한국전자통신연구원 Image diagnosis device including x-ray image tomosynthesis device and photoacoustic image device and image diagnosis method thereof

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
CAI, X. ET AL.: "Photoacoustic Microscopy in Tissue Engineering.", MATERIAL TODAY., vol. 16, no. 3, March 2013 (2013-03-01), pages 67 - 77 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109102573A (en) * 2018-08-06 2018-12-28 百度在线网络技术（北京）有限公司 Image processing method, device and storage medium
CN109102573B (en) * 2018-08-06 2023-05-02 百度在线网络技术（北京）有限公司 Image processing method, device and storage medium

Also Published As

Publication number Publication date
KR101418405B1 (en) 2014-07-09

Similar Documents

Publication Publication Date Title
US11029287B2 (en) Multi-focus optical-resolution photoacoustic microscopy with ultrasonic array detection
US20200128187A1 (en) Subject information obtaining apparatus, display method, program, and processing apparatus
Chen et al. Progress of clinical translation of handheld and semi-handheld photoacoustic imaging
JP5743957B2 (en) Photoacoustic image generation apparatus and method
US9746318B2 (en) Imaging apparatus and imaging method
JP7164339B2 (en) Photocoagulator
JP5840069B2 (en) Photoacoustic image generation apparatus, system, and method
Ahn et al. Fully integrated photoacoustic microscopy and photoplethysmography of human in vivo
KR101513032B1 (en) Microscope system and method of providing augmented reality image using microscope system
Wang et al. Recent advances in high-speed photoacoustic microscopy
KR101610837B1 (en) Photoacoustic imaging system for diagnosis animals of the ophthalmic diseases
Truong et al. Ultra-widefield photoacoustic microscopy with a dual-channel slider-crank laser-scanning apparatus for in vivo biomedical study
KR101418405B1 (en) medical microscope system of overlapping and outputting high magnification surface image and augmented reality image having photoacoustic tomography Image and apparatus of providing augmented reality image therefor
Choi et al. Volumetric cutaneous microangiography of human skin in vivo by VCSEL swept-source optical coherence tomography
Huang et al. Review of low-cost light sources and miniaturized designs in photoacoustic microscopy
KR101296369B1 (en) Optical coherence tomography system using a visible laser beam
Jiang et al. Photoacoustic imaging plus X: a review
CN110384480B (en) Subject information acquisition device, subject information processing method, and storage medium
KR101509516B1 (en) Endoscope, handpiece of endocscope, calibration method for the endoscope, using method for the endoscope
Baran et al. Video-rate volumetric optical coherence tomography-based microangiography
KR101350739B1 (en) Optical coherence tomography system using a selecting the target on monitoring image
KR102204426B1 (en) Image acquisition system and image acquisition method using the same
JP7164338B2 (en) Photocoagulator, fundus observation device, program, and recording medium
JP2018169246A (en) Outgoing beam controller of optical deflector
Lee et al. Virtual intraoperative surgical photoacoustic microscopy

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13883200

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 13883200

Country of ref document: EP

Kind code of ref document: A1