US11638000B2 - Medical observation apparatus - Google Patents

Medical observation apparatus Download PDF

Info

Publication number
US11638000B2
US11638000B2 US16/525,587 US201916525587A US11638000B2 US 11638000 B2 US11638000 B2 US 11638000B2 US 201916525587 A US201916525587 A US 201916525587A US 11638000 B2 US11638000 B2 US 11638000B2
Authority
US
United States
Prior art keywords
medical
image
points
positions
display
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
US16/525,587
Other languages
English (en)
Other versions
US20200045293A1 (en
Inventor
Takeshi Honda
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Olympus Medical Solutions Inc
Original Assignee
Sony Olympus Medical Solutions Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Olympus Medical Solutions Inc filed Critical Sony Olympus Medical Solutions Inc
Assigned to SONY OLYMPUS MEDICAL SOLUTIONS INC. reassignment SONY OLYMPUS MEDICAL SOLUTIONS INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: HONDA, TAKESHI
Publication of US20200045293A1 publication Critical patent/US20200045293A1/en
Application granted granted Critical
Publication of US11638000B2 publication Critical patent/US11638000B2/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00147—Holding or positioning arrangements
    • A61B1/00149—Holding or positioning arrangements using articulated arms
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
    • A61B1/00009—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope
    • A61B1/000094—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope extracting biological structures
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00043—Operational features of endoscopes provided with output arrangements
    • A61B1/00045—Display arrangement
    • A61B1/0005—Display arrangement combining images e.g. side-by-side, superimposed or tiled
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00163—Optical arrangements
    • A61B1/00193—Optical arrangements adapted for stereoscopic vision
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
    • A61B1/045—Control thereof
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/06—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements
    • A61B1/0655—Control therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/172—Processing image signals image signals comprising non-image signal components, e.g. headers or format information
    • H04N13/183—On-screen display [OSD] information, e.g. subtitles or menus
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/204—Image signal generators using stereoscopic image cameras
    • H04N13/239—Image signal generators using stereoscopic image cameras using two 2D image sensors having a relative position equal to or related to the interocular distance
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/204—Image signal generators using stereoscopic image cameras
    • H04N13/243—Image signal generators using stereoscopic image cameras using three or more 2D image sensors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/296—Synchronisation thereof; Control thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/50—Constructional details
    • H04N23/555—Constructional details for picking-up images in sites, inaccessible due to their dimensions or hazardous conditions, e.g. endoscopes or borescopes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/69—Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/695—Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/272—Means for inserting a foreground image in a background image, i.e. inlay, outlay
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
    • A61B1/00006—Operational features of endoscopes characterised by electronic signal processing of control signals
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00011—Operational features of endoscopes characterised by signal transmission
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00064—Constructional details of the endoscope body
    • A61B1/00105—Constructional details of the endoscope body characterised by modular construction
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00163—Optical arrangements
    • A61B1/00186—Optical arrangements with imaging filters
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
    • A61B1/042—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances characterised by a proximal camera, e.g. a CCD camera
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
    • A61B1/043—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances for fluorescence imaging
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/06—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements
    • A61B1/0638—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements providing two or more wavelengths
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/06—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements
    • A61B1/07—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements using light-conductive means, e.g. optical fibres
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10068—Endoscopic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30244—Camera pose

Definitions

  • the present disclosure relates to a medical observation apparatus.
  • a medical observation apparatus which enables magnified observation of an observation target, such as a lesion, may be used, for example, for support of a microsurgery, such as a neurosurgical operation, or for an endoscopic surgical operation.
  • this medical observation apparatus include: a medical observation apparatus including an optical microscope; and a medical observation apparatus including an imaging device that functions as an electronic imaging microscope.
  • a medical observation apparatus including an optical microscope as mentioned above will be referred to as an “optical medical observation apparatus”.
  • a medical observation apparatus including an imaging device as mentioned above may be referred to as an “electronic imaging medical observation apparatus” or simply as a “medical observation apparatus”.
  • a captured image (a moving image or a static image, the same applying hereinafter) having an observation target captured therein by an imaging device included in a medical observation apparatus will be referred to as a “captured medical image”.
  • Electronic imaging medical observation apparatuses now enable acquisition of image quality equivalent to or better than that by optical medical observation apparatuses because of increase in image quality of imaging devices and increase in image quality of display devices where captured images are displayed. Furthermore, a user who uses an electronic imaging medical observation apparatus (for example, a medical worker, such as a surgical operator or an assistant of the surgical operator, the same applying hereinafter) does not need to look into an eyepiece forming an optical microscope as done in a case where an optical medical observation apparatus is used, and thus position of its imaging device is able to be moved more freely. Therefore, due to an advantage that use of electronic imaging medical observation apparatuses enables microsurgeries and the like to be supported more flexibly, use of electronic imaging medical observation apparatuses in medical settings has been promoted.
  • an electronic imaging medical observation apparatus for example, a medical worker, such as a surgical operator or an assistant of the surgical operator, the same applying hereinafter
  • GUI graphical user interface
  • Display of a GUI object, such as a diameter measuring object, on a captured medical image is realized by use of the technique described in Japanese Translation of PCT International Application, Publication No. 2007-521864.
  • use of the technique described in Japanese Translation of PCT International Application, Publication No. 2007-521864 realizes display of the diameter measuring object for a contour included in the captured medical image.
  • the diameter measuring object is displayed for the contour included in the captured medical image as described above, a person looking at the captured medical image displayed on a display screen is able to visually recognize the diameter of the contour.
  • a medical observation apparatus includes: an imaging device configured to capture an observation target to obtain a captured right eye medical image and a captured left eye medical image; and circuitry configured to: acquire positions of at least two points in the observation target, the positions being determined based on predetermined operation on the observation target; and cause the captured right eye medical image, the captured left eye medical image, and an annotation image, to be displayed on a display screen of a display device, the annotation image indicating a distance between two points at the acquired positions.
  • FIG. 1 is an explanatory diagram illustrating a first example of a configuration of a medical observation system according to an embodiment
  • FIG. 2 is an explanatory diagram illustrating an example of a use case where the medical observation system according to the embodiment is used;
  • FIG. 3 is an explanatory diagram for explanation of an example of a configuration of an imaging device included in a medical observation apparatus according to the embodiment
  • FIG. 4 is an explanatory diagram illustrating a second example of the configuration of the medical observation system according to the embodiment.
  • FIG. 5 is a functional block diagram illustrating an example of a configuration of the medical observation apparatus according to the embodiment.
  • FIG. 6 is an explanatory diagram for explanation of an example of an annotation image displayed by a display control method according to the embodiment
  • FIG. 7 is an explanatory diagram for explanation of another example of the annotation image displayed by the display control method according to the embodiment.
  • FIG. 8 is a functional block diagram illustrating a first example of a configuration of a processing unit included in the medical observation apparatus according to the embodiment.
  • FIG. 9 is a flow chart illustrating an example of processing in an acquiring unit that the processing unit included in the medical observation apparatus according to the embodiment has;
  • FIG. 10 is a flow chart illustrating an example of processing in a distance calculating unit that the processing unit included in the medical observation apparatus according to the embodiment has;
  • FIG. 11 is a functional block diagram illustrating a second example of the configuration of the processing unit included in the medical observation apparatus according to the embodiment.
  • FIG. 12 is a functional block diagram illustrating a third example of the configuration of the processing unit included in the medical observation apparatus according to the embodiment.
  • FIG. 13 is an explanatory diagram for explanation of an example of an image displayed by the display control method according to the embodiment.
  • a medical observation apparatus performs processing related to the display control method according to the embodiment
  • an apparatus that is able to perform the processing related to the display control method according to the embodiment is not limited to the medical observation apparatus according to the embodiment.
  • any apparatus such as a medical controller, may perform the processing related to the display control method according to the embodiment.
  • FIG. 1 is an explanatory diagram illustrating a first example of a configuration of a medical observation system 1000 according to the embodiment.
  • the medical observation system 1000 illustrated in FIG. 1 has, for example, a medical observation apparatus 100 , and a display device 200 .
  • a medical observation system according to the first example is not limited to the example illustrated in FIG. 1 .
  • the medical observation system according to the first example may further have a medical control device (not illustrated in the drawings) that controls various types of operation in the medical observation apparatus 100 .
  • a medical control device (not illustrated in the drawings) that controls various types of operation in the medical observation apparatus 100 .
  • the medical observation system 1000 illustrated in FIG. 1 represents an example where the medical observation apparatus 100 has functions of the medical control device (not illustrated in the drawings) by the medical observation apparatus 100 including a control unit (described later).
  • Examples of the medical control device include: a “medical controller”; and “a computer, such as a server” Furthermore, the medical control device (not illustrated in the drawings) may be, for example, an integrated circuit (IC) that is able to be incorporated in a device as described above.
  • IC integrated circuit
  • the medical observation system according to the first example may be configured to have: plural medical observation apparatuses 100 ; plural display devices 200 ; or both plural medical observation apparatuses 100 and plural display devices 200 . If the medical observation system has plural medical observation apparatuses 100 , the processing related to the display control method described later is performed in each of the medical observation apparatuses 100 . Moreover, if the medical observation system according to the first example is configured to have plural medical observation apparatuses 100 and plural display devices 200 , the medical observation apparatuses 100 and the display devices 200 may have one-to-one correspondence, or more than one medical observation apparatus 100 may be associated with one display device 200 . If more than one medical observation apparatus 100 is associated with one display device 200 , which one of captured medical images respectively captured by these medical observation apparatuses 100 is to be displayed on a display screen is changed in the display device 200 by, for example, a switch-over manipulation.
  • the medical observation system may further have a navigation device.
  • the navigation device is a medical device for realizing a so-called medical navigation system.
  • the navigation device detects a spatial position of a position detecting probe, and causes an image corresponding to the detected spatial position to be displayed on a display screen of an arbitrary display device.
  • the navigation device is driven by, for example, electric power supplied from an internal power source, such as a battery, which is included in the navigation device, or electric power supplied from an external power source connected to the navigation device.
  • the navigation device detects a position of a site of surgical operation in a patient by detecting a spatial position of the position detecting probe by any position detecting method, such as an optical position detecting method where infrared light or the like is used, or a magnetic field type position detecting method.
  • a position sensor that detects the spatial position of the position detecting probe may be included in the navigation device, or may be provided at any position outside the navigation device.
  • a medical worker such as a surgical operator, is able to visually recognize which part of the patient a portion being treated corresponds to.
  • the navigation device may have a function of transmitting positional information indicating the detected spatial position of the position detecting probe, to an external device, such as the medical observation apparatus 100 .
  • the navigation device corresponds to an example of a detecting device that detects a position in an observation target.
  • FIG. 2 is an explanatory diagram illustrating an example of a use case where the medical observation system 1000 according to the embodiment is used, and illustrates an example of a use case where the medical observation system 1000 according to the first example is used.
  • An image of a patient PA (a patient to receive medical intervention) who is an observation target is captured by an imaging device (described later) included in the medical observation apparatus 100 .
  • a captured image having the patient PA captured therein, the patient PA being a target that receives the medical intervention, corresponds to an example of a captured medical image.
  • the captured medical image captured in the medical observation apparatus 100 is displayed on the display screen of the display device 200 .
  • a surgical operator (OP) (an example of a user of the medical observation apparatus 100 ) who performs the medical intervention by using the medical observation apparatus 100 performs the medical intervention on the patient PA while looking at the captured medical image being displayed on the display screen of the display device 200 .
  • the surgical operator OP causes an arm (described later), the imaging device (described later), or the like, which is included in the medical observation apparatus 100 , to operate and the medical observation apparatus 100 to be brought into a desired state, by manipulating a manipulation device, such as a foot switch FS, which is external to the medical observation apparatus 100 , or a manipulation device (described later) included in the medical observation apparatus 100 .
  • a manipulation device such as a foot switch FS, which is external to the medical observation apparatus 100 , or a manipulation device (described later) included in the medical observation apparatus 100 .
  • the display device 200 is a display means in the medical observation system 1000 according to the first example, and corresponds to a display device external to the medical observation apparatus 100 .
  • the display device 200 displays, for example, various images, such as a captured medical image captured in the medical observation apparatus 100 and an image related to a user interface (UI), on the display screen.
  • UI user interface
  • the display device 200 may have a configuration enabling 3D display by an arbitrary method. Display in the display device 200 is controlled by, for example, the medical observation apparatus 100 or the medical control device (not illustrated in the drawings).
  • the display device 200 is installed at an arbitrary place, such as a wall surface, a ceiling, or a floor surface, of a surgical operating room, the arbitrary place being visually recognizable by a person, such as a surgical operator, who is involved in a surgical operation in the surgical operating room.
  • Examples of the display device 200 include a liquid crystal display, an organic electro-luminescence (EL) display, and a cathode ray tube (CRT) display.
  • EL organic electro-luminescence
  • CRT cathode ray tube
  • the display device 200 is not limited to the example described above.
  • the display device 200 may be any wearable device, such as a head mounted display or an eyewear device, which is used by being worn by the surgical operator or the like on the body.
  • the display device 200 is driven by, for example, electric power supplied from an internal power source, such as a battery, which is included in the display device 200 , or electric power supplied from an external power source connected to the display device 200 .
  • an internal power source such as a battery
  • an external power source connected to the display device 200 .
  • the medical observation apparatus 100 illustrated in FIG. 1 is an example of an electronic imaging medical observation apparatus.
  • a surgical operator an example of a user of the medical observation apparatus 100 , the same applying hereinafter observes a site of surgical operation (a lesion) while referring to a captured medical image captured by the medical observation apparatus 100 and displayed on the display screen of the display device 200 , and performs various treatments, such as maneuvers, according to operative surgical procedures, on the site of surgical operation.
  • the medical observation apparatus 100 includes, for example, a base 102 , an arm 104 , and an imaging device 106 .
  • the medical observation apparatus 100 may include, for example: one or more processors (not illustrated in the drawings) each formed of an arithmetic operation circuit, such as a micro processing unit (MPU); a read only memory (ROM, not illustrated in the drawings); a random access memory (RAM, not illustrated in the drawings); a recording medium (not illustrated in the drawings); and a communication device (not illustrated in the drawings), which are all not illustrated in FIG. 1 .
  • the medical observation apparatus 100 is driven by, for example, electric power supplied from an internal power source, such as a battery, which is included in the medical observation apparatus 100 , or electric power supplied from an external power source connected to the medical observation apparatus 100 .
  • the processor functions as the control unit (described later) in the medical observation apparatus 100 .
  • the ROM (not illustrated in the drawings) stores therein a program used by the processor (not illustrated in the drawings) and control data, such as arithmetic operation parameters.
  • the RAM (not illustrated in the drawings) temporarily stores therein the program executed by the processor (not illustrated in the drawings).
  • the recording medium functions as a storage unit (not illustrated in the drawings) in the medical observation apparatus 100 .
  • the recording medium has, stored therein, various data, such as data related to the display control method according to the embodiment, and various applications.
  • Examples of the recording medium (not illustrated in the drawings) include a magnetic recording medium, such as a hard disk, and a non-volatile memory, such as a flash memory.
  • the recording medium (not illustrated in the drawings) may be attachable to and detachable from the medical observation apparatus 100 .
  • the communication device (not illustrated in the drawings) is a communication means included in the medical observation apparatus 100 , and plays a role of performing communication wirelessly or wiredly with an external device, such as the display device 200 .
  • the communication device may be, for example: an IEEE 802.15.1 port and a transmitting and receiving circuit (wireless communication); an IEEE 802.11 port and a transmitting and receiving circuit (wireless communication); a communication antenna and a radio frequency (RF) circuit (wireless communication); or a local area network (LAN) terminal and a transmitting and receiving circuit (wired communication).
  • the base 102 is a base of the medical observation apparatus 100 , has one end of the arm 104 connected thereto, and supports the arm 104 and the imaging device 106 .
  • the base 102 has, for example, wheels provided therein, and the medical observation apparatus 100 contacts a floor surface via the wheels.
  • the medical observation apparatus 100 is able to easily move on the floor surface by means of the wheels.
  • the arm 104 is formed of plural links connected to one another via joints.
  • the arm 104 supports the imaging device 106 .
  • the imaging device 106 supported by the arm 104 is three-dimensionally movable, and the position and posture of the imaging device 106 that has been moved are retained by the arm 104 .
  • the arm 104 is formed of, for example, plural joints 110 a , 110 b , 110 c , 110 d , 110 e , and 110 f , and plural links 112 a , 112 b , 112 c , 112 d , 112 e , and 112 f that are rotatably connected to one another via the joints 110 a , 110 b , 110 c , 110 d , 110 e , and 110 f .
  • a rotatable range of each of the joints 110 a , 110 b , 110 c , 110 d , 110 e , and 110 f is arbitrarily set in the design phase and manufacturing phase, such that desired movement of the arm 104 is realized.
  • six degrees of freedom are realized with respect to movement of the imaging device 106 by means of six rotation axes (a first axis O 1 , a second axis O 2 , a third axis O 3 , a fourth axis O 4 , a fifth axis O 5 , and a sixth axis O 6 ) corresponding to the six joints 110 a , 110 b , 110 c , 110 d , 110 e , and 110 f that form the arm 104 . More specifically, in the medical observation apparatus 100 illustrated in FIG. 1 , movement of six degrees of freedom, which are three degrees of translational freedom and three degrees of rotational freedom, is realized.
  • the joints 110 a , 110 b , 110 c , 110 d , 110 e , and 110 f each have an actuator (not illustrated in the drawings) provided therein, and the joints 110 a , 110 b , 110 c , 110 d , 110 e , and 110 f respectively rotate about the rotational axes corresponding thereto by drive of the actuators (not illustrated in the drawings).
  • the drive by the actuators is controlled by, for example, the processor that functions as the control unit described later, or the external medical control device (not illustrated in the drawings).
  • the joints 110 a , 110 b , 110 c , 110 d , 110 e , and 110 f may respectively have, provided therein, angle sensors (not illustrated in the drawings) that are able to detect rotation angles about the six rotation axes respectively.
  • the angle sensors may be any sensors, such as rotary encoders or angular velocity sensors, which are able to obtain the rotation angles about the six rotation axes respectively.
  • the joint 110 a has a substantially columnar shape, and supports the imaging device 106 (an upper end portion of the imaging device 106 in FIG. 1 ) rotatably about the rotation axis (the first axis O 1 ) parallel to a center axis of the imaging device 106 , by means of a distal end position (a lower end portion in FIG. 1 ) of the joint 110 a .
  • the medical observation apparatus 100 is formed such that the first axis O 1 coincides with the optical axis in the imaging device 106 . That is, by rotation of the imaging device 106 about the first axis O 1 illustrated in FIG. 1 , a captured medical image captured by the imaging device 106 becomes an image where the field of view is changed as if being rotated.
  • the link 112 a is a member that is substantially rod shaped, and supports the joint 110 a fixedly.
  • the link 112 a is, for example, extended in a direction orthogonal to the first axis O 1 , and connected to the joint 110 b.
  • the joint 110 b has a substantially columnar shape, and supports the link 112 a rotatably about the rotation axis (the second axis O 2 ) orthogonal to the first axis O 1 . Furthermore, the joint 110 b has the link 112 b fixedly connected thereto.
  • the link 112 b is a member that is substantially rod shaped, and is extended in a direction orthogonal to the second axis O 2 . Furthermore, each of the joint 110 b and joint 110 c is connected to the link 112 b.
  • the joint 110 c has a substantially columnar shape, and supports the link 112 b rotatably about the rotation axis (the third axis O 3 ) orthogonal to each of the first axis O 1 and second axis O 2 . Furthermore, one end of the link 112 c is fixedly connected to the joint 110 c.
  • the imaging device 106 By rotation of a distal end (an end where the imaging device 106 is provided) of the arm 104 about the second axis O 2 and third axis O 3 , the imaging device 106 is able to be moved such that position of the imaging device 106 is changed in a horizontal plane. That is, in the medical observation apparatus 100 , control of rotation about the second axis O 2 and third axis O 3 enables movement of the field of view of a captured medical image in a plane.
  • the link 112 c is a member, which has one end having a substantially columnar shape, and another end that is substantially rod shaped.
  • the joint 110 c is fixedly connected to the one end of the link 112 c , such that the center axis of the joint 110 c coincides with the center axis of the substantially columnar shape.
  • the joint 110 d is connected to the other end of the link 112 c .
  • the joint 110 d has a substantially columnar shape, and supports the link 112 c rotatably about the rotation axis (the fourth axis O 4 ) orthogonal to the third axis O 3 .
  • the joint 110 d has the link 112 d connected fixedly thereto.
  • the link 112 d is a substantially rod shaped member, and is extended orthogonally to the fourth axis O 4 .
  • One end of the link 112 d is fixedly connected to the joint 110 d , so as to abut a side surface of the substantially columnar shape of the joint 110 d .
  • the joint 110 e is connected to the other end of the link 112 d (an end opposite to the end connected to the joint 110 d ).
  • the joint 110 e has a substantially columnar shape, and supports the other end of the link 112 d rotatably about the rotation axis (the fifth axis O 5 ) parallel to the fourth axis O 4 . Furthermore, the one end of the link 112 e is fixedly connected to the joint 110 e.
  • the fourth axis O 4 and the fifth axis O 5 are rotation axes that allow the imaging device 106 to be moved in a vertical direction.
  • rotation of the distal end (the end where the imaging device 106 is provided) of the arm 104 about the fourth axis O 4 and fifth axis O 5 the position of the imaging device 106 in the vertical direction is changed. Therefore, by the rotation of the distal end (the end where the imaging device 106 is provided) of the arm 104 about the fourth axis O 4 and fifth axis O 5 , the distance between the imaging device 106 and an observation target, such as a site of surgical operation in a patient, is able to be changed.
  • the link 112 e is a member formed of a combination of: a first member substantially having an L-shape with one side thereof extending in a vertical direction and another side thereof extending in a horizontal direction; and a second member, which extends vertically downward from a portion of the first member, the portion extending in the horizontal direction, and which is rod-shaped.
  • the joint 110 e is fixedly connected to a portion of the first member of the link 112 e , the portion extending in the vertical direction.
  • the second member of the link 112 e has the joint 110 f connected thereto.
  • the joint 110 f has a substantially columnar shape, and supports the link 112 e rotatably about the rotation axis (the sixth axis O 6 ) parallel to the vertical direction. Furthermore, the joint 110 f has the link 112 f fixedly connected thereto.
  • the link 112 f is a member that is substantially rod shaped, and is extended in the vertical direction.
  • the joint 110 f is connected to one end of the link 112 f . Furthermore, the other end (an end opposite to the end connected to the joint 110 f ) of the link 112 f is fixedly connected to the base 102 .
  • the arm 104 having the above described configuration, in the medical observation apparatus 100 , six degrees of freedom are realized with respect to movement of the imaging device 106 .
  • the configuration of the arm 104 is not limited to the example described above.
  • the joints 110 a , 110 b , 110 c , 110 d , 110 e , and 110 f of the arm 104 may respectively have brakes provided therein, the brakes respectively restricting rotation at the joints 110 a , 110 b , 110 c , 110 d , 110 e , and 110 f .
  • the brakes according to the embodiment include brakes of any form, such as brakes that are mechanically driven, and electromagnetic brakes that are electrically driven.
  • Driving of the brakes is controlled by, for example, the processor that functions as the control unit described later, or the external medical control device (not illustrated in the drawings).
  • operation modes of the arm 104 are set. Examples of the operation modes of the arm 104 include a fixed mode and a free mode.
  • the fixed mode is an operation mode where, for example, the position and posture (the position and posture of an imaging unit 150 described later) of the imaging device 106 are fixed by restriction of rotation about the rotation axes provided in the arm 104 by means of the brakes.
  • the arm 104 being brought into the fixed mode, a state of operation of the medical observation apparatus 100 is brought into a fixed state where the position and posture of the imaging device 106 are fixed.
  • the free mode according to the embodiment is an operation mode where the rotation axes provided in the arm 104 are freely rotatable by the brakes being released.
  • the position and posture (the position and posture of the imaging unit 150 described later) of the imaging device 106 are allowed to be adjusted through direct manipulation by a surgical operator.
  • the direct manipulation according to the embodiment means, for example, manipulation where a surgical operator holds the imaging device 106 in the hand and directly moves the imaging device 106 .
  • the imaging device 106 is supported by the arm 104 , and captures an image of an observation target, such as, for example, a site of surgical operation of a patient. Imaging in the imaging device 106 is controlled by, for example, the processor that functions as the control unit described later, or the external medical control device (not illustrated in the drawings).
  • the imaging device 106 has a configuration corresponding to, for example, an electronic imaging microscope.
  • FIG. 3 is an explanatory diagram for explanation of an example of the configuration of the imaging device 106 included in the medical observation apparatus 100 according to the embodiment.
  • the imaging device 106 has, for example, an imaging member 120 , and a cylindrical member 122 having a substantially cylindrical shape, and the imaging member 120 is provided in the cylindrical member 122 .
  • a cover glass (not illustrated in the drawings) for protecting the imaging member 120 is provided.
  • a light source (not illustrated in the drawings) is provided inside the cylindrical member 122 , and at the time of imaging, illumination light is emitted from the light source to a subject through the cover glass. Reflected light (observation light) from the subject irradiated with the illumination light enters the imaging member 120 via the cover glass (not illustrated in the drawings), and an image signal representing the subject (an image signal representing a captured medical image) is thereby acquired by the imaging member 120 .
  • any one of configurations used in various known electronic imaging microscopes may be adopted for the imaging member 120 .
  • the imaging member 120 is formed of, for example: an optical system 120 a ; and an image sensor 120 b including an imaging element that captures an image of an observation target by means of light that has passed through the optical system 120 a .
  • the optical system 120 a is formed of, for example: one or more of lenses including an objective lens, a zoom lens, and a focus lens; and an optical element, such as a mirror.
  • Examples of the image sensor 120 b include an image sensor having plural imaging elements used therein, the imaging elements being, for example, complementary metal oxide semiconductors (CMOSs) or charge coupled devices (CCDs).
  • CMOSs complementary metal oxide semiconductors
  • CCDs charge coupled devices
  • the imaging member 120 functions as a so-called stereo camera by including, for example, two or more imaging devices each formed of the optical system 120 a and the image sensor 120 b .
  • the optical system may be a Galilean optical system, or a Greenough-type optical system.
  • the medical observation apparatus 100 includes plural imaging devices that function as a stereo camera, and plural captured medical images including a captured right eye medical image and a captured left eye medical image are respectively acquired by imaging of the plural imaging devices will be described below as an example.
  • an imaging device that captures a captured right eye medical image will be referred to as a “first imaging device”
  • an imaging device that captures a captured left eye medical image will be referred to as a “second imaging device”.
  • a captured right eye medical image and a captured left eye medical image may be collectively referred to as “captured medical images”.
  • the imaging devices forming the imaging member 120 are each installed with one or more functions generally included in an electric imaging microscope, such as a zoom function (one or both of an optical zoom function and an electronic zoom function) and an auto-focus (AF) function.
  • a zoom function one or both of an optical zoom function and an electronic zoom function
  • AF auto-focus
  • the imaging member 120 may be configured to enable imaging at so-called high-definition, for example, 4K or 8K.
  • the imaging member 120 being configured to enable high definition imaging, display of an image by the display device 200 including a large display screen of, for example, 50 inches or more, is enabled with predetermined definition (for example, full HD image quality) ensured, and thus visual recognizability by a surgical operator looking at the display screen is improved.
  • predetermined definition for example, full HD image quality
  • predetermined definition is able to be ensured even if an image captured is displayed enlarged by the electronic zoom function on the display screen of the display device 200 .
  • the imaging device 106 is provided with, for example, various manipulation devices for controlling the operation of the imaging device 106 .
  • a zoom switch 124 a focus switch 126 , and an operation mode changing switch 128 are provided in the imaging device 106 .
  • the positions and forms where the zoom switch 124 , the focus switch 126 , and the operation mode changing switch 128 are provided are not limited to the example illustrated in FIG. 3 .
  • the zoom switch 124 and the focus switch 126 are examples of the manipulation devices for adjusting imaging conditions in the imaging device 106 .
  • the zoom switch 124 is formed of, for example, a zoom-in switch 124 a that increases the zoom magnification (the magnifying power), and a zoom-out switch 124 b that decreases the zoom magnification. By manipulation of the zoom switch 124 , the zoom magnification is adjusted and zooming is adjusted.
  • the focus switch 126 is formed of, for example, a distant view focus switch 126 a that increases the focal distance to an observation target (a subject), and a near view focus switch 126 b that decreases the focal distance to the observation target. By the manipulation of the focus switch 126 , the focal distance is adjusted and focusing is adjusted.
  • the operation mode changing switch 128 is an example of a manipulation device for changing the operation mode of the arm 104 in the imaging device 106 . By manipulation of the operation mode changing switch 128 , the operation mode of the arm 104 is changed. Examples of the operation mode of the arm 104 include, as described above, the fixed mode and the free mode.
  • Examples of the manipulation of the operation mode changing switch 128 include manipulation where the operation mode changing switch 128 is pressed down. For example, while a surgical operator is holding the operation mode changing switch 128 down, the operation mode of the arm 104 is in the free mode, and when the surgical operator is not holding the operation mode changing switch 128 down, the operation mode of the arm 104 is in the fixed mode.
  • the imaging device 106 is provided with, for example, a non-slip member 130 and a protruding member 132 , for further improvement in operability and convenience upon manipulation by an operator performing manipulation of the various manipulation devices.
  • the non-slip member 130 is a member provided for prevention of slippage of a manipulating body, such as a hand, when, for example, the operator performs manipulation of the cylindrical member 122 with the manipulating body.
  • the non-slip member 130 is formed of a material having a large friction coefficient, and has a less slippery structure, such as bumps and dips.
  • the protruding member 132 is a member provided for prevention of: the manipulating body, such as the hand, blocking the field of view of the optical system 120 a when the operator manipulates the cylindrical member 122 with the manipulating body; and the cover glass (not illustrated in the drawings) becoming unclean by the manipulating body touching the cover glass when manipulation with the manipulating body is performed.
  • positions and forms where the non-slip member 130 and protruding member 132 are respectively provided are not limited to the example illustrated in FIG. 3 .
  • the imaging device 106 may be not provided with one or both of the non-slip member 130 and the protruding member 132 .
  • An image signal (image data) generated by imaging in the imaging device 106 is subjected to image processing in, for example, the processor functioning as the control unit described later.
  • Examples of the image processing according to the embodiment include one or more of: gamma correction, white balance adjustment, image enlargement or reduction related to the electronic zoom function, and correction among pixels.
  • the medical observation system has the medical control device (not illustrated in the drawings) that controls various types of operation in the medical observation apparatus 100
  • the image processing according to the embodiment may be performed in the medical control device (not illustrated in the drawings).
  • the medical observation apparatus 100 transmits, for example, a display control signal, and an image signal subjected to image processing as mentioned above, to the display device 200 .
  • a captured medical image having an observation target captured therein (for example, a captured image having a site of surgical operation captured therein) is displayed on the display screen of the display device 200 .
  • the captured medical image having the observation target captured therein may be displayed enlarged or reduced to a desired magnification by one or both of the optical zoom function and electronic zoom function, on the display screen of the display device 200 .
  • the medical observation apparatus 100 illustrated in FIG. 1 has, for example, the hardware configuration described above by reference to FIG. 1 and FIG. 3 .
  • the hardware configuration of the medical observation apparatus according to the embodiment is not limited to the configuration described above by reference to FIG. 1 and FIG. 3 .
  • the medical observation apparatus may be configured to have the arm 104 directly attached to a ceiling or wall surface of a surgical operating room, without including the base 102 .
  • the medical observation apparatus according to the embodiment has a configuration where the arm 104 is suspended from the ceiling.
  • FIG. 1 illustrates the example where the arm 104 is configured to realize six degrees of freedom with respect to driving of the imaging device 106 , but the configuration of the arm 104 is not limited to the configuration where the number of degrees of freedom related to the driving of the imaging device 106 is six.
  • the arm 104 may just be configured to be able to move the imaging device as appropriate according to a use, and the number and arrangement of the joints and links, and the directions of the drive axes of the joints may be set as appropriate such that the arm 104 has desired freedom.
  • FIG. 1 and FIG. 3 illustrate the example where the various manipulation devices for controlling the operation of the imaging device 106 are provided in the imaging device 106 , but a part or all of the manipulation devices illustrated in FIG. 1 and FIG. 3 may be not provided in the imaging device 106 .
  • the various manipulation devices for controlling the operation of the imaging device 106 may be provided in another part forming the medical observation apparatus according to the embodiment, instead of in the imaging device 106 .
  • the various manipulation devices for controlling the operation of the imaging device 106 may include an external manipulation device, such as a foot switch FS or a remote controller.
  • the imaging device 106 may be configured to enable switch-over among plural observation modes.
  • the observation modes according to the embodiment include: an observation mode where imaging is performed with natural light; an observation mode where imaging is performed with special light; and an observation mode where imaging is performed by use of an image enhancement observation technique, such as narrow band imaging (NBI).
  • NBI narrow band imaging
  • the special light according to the embodiment include light of specific wavelength bands, such as: light of a near infra-red wavelength band; and light of a fluorescent wavelength band of fluorescent observation where 5-aminolevulinic acid (5-ALA) is used.
  • 5-ALA 5-aminolevulinic acid
  • Examples of the configuration of the imaging device 106 the configuration enabling the plural observation modes to be switched from one to another, include “a configuration including a filter that transmits therethrough light of a specific wavelength band and does not transmit therethrough light of other wavelength bands, and a moving mechanism that selectively places the filter onto an optical path”.
  • Examples of the specific wavelength band transmitted through the filter according to the embodiment include: a near infra-red wavelength band (for example, a wavelength band from about 0.7 micrometers to about 2.5 micrometers); a fluorescent wavelength band according to fluorescent observation by use of 5-ALA (for example, a wavelength band from about 0.6 micrometers to about 0.65 micrometers); and a fluorescent wavelength band of indocyanine green (ICG) (for example, a wavelength band from about 0.82 micrometers to about 0.85 micrometers).
  • a near infra-red wavelength band for example, a wavelength band from about 0.7 micrometers to about 2.5 micrometers
  • a fluorescent wavelength band according to fluorescent observation by use of 5-ALA for example, a wavelength band from about 0.6 micrometers to about 0.65 micrometers
  • a fluorescent wavelength band of indocyanine green (ICG) for example, a wavelength band from about 0.82 micrometers to about 0.85 micrometers.
  • the imaging device 106 may be provided with plural filters where wavelength bands transmitted therethrough are different from one another. Furthermore, the above description is on the example where imaging is performed with light a specific wavelength band by the arrangement of the filter on the optical path, but needless to say, the configuration of the imaging device 106 for imaging with light of a specific wavelength band is not limited to the above described example.
  • the medical observation system 1000 is not limited to the configuration represented by the first example illustrated in FIG. 1 .
  • An example of the configuration of a medical observation system 1000 including a medical observation apparatus 100 that functions as an endoscope apparatus will be described next as another example of the medical observation system 1000 .
  • FIG. 4 is an explanatory diagram illustrating a second example of the configuration of the medical observation system 1000 according to the embodiment.
  • the medical observation system 1000 illustrated in FIG. 4 has, for example, the medical observation apparatus 100 and the display device 200 .
  • a surgical operator observes a site of surgical operation while referring to a captured medical image captured by the medical observation apparatus 100 and displayed on the display screen of the display device 200 , and performs various treatments, such as maneuvers according to operative surgical procedures, on the site of surgical operation.
  • the medical observation system according to the second example is not limited to the example illustrated in FIG. 4 .
  • the medical observation system according to the second example may further have a medical control device (not illustrated in the drawings) that controls various types of operation in the medical observation apparatus 100 .
  • the medical observation system according to the second example may be configured to have plural medical observation apparatuses 100 , plural display devices 200 , or both plural medical observation apparatus 100 and plural display devices 200 .
  • the display device 200 is a display means in the medical observation system 1000 according to the second example, and corresponds to a display device external to the medical observation apparatus 100 .
  • the display device 200 forming the medical observation system 1000 according to the second example is similar to the display device 200 forming the medical observation system 1000 according to the first example.
  • the medical observation apparatus 100 illustrated in FIG. 4 includes, for example, an insertion member 134 , a light source unit 136 , a light guide 138 , a camera head 140 , a cable 142 , and a control unit 144 .
  • the medical observation apparatus 100 is driven by, for example, electric power supplied from an internal power source, such as a battery, which is included in the medical observation apparatus 100 , or electric power supplied from an external power source connected to the medical observation apparatus 100 .
  • the insertion member 134 has an elongated shape, and includes therein an optical system that condenses incident light.
  • a distal end of the insertion member 134 is inserted into, for example, a body cavity of a patient.
  • a rear end of the insertion member 134 is attachably and detachably connected to a distal end of the camera head 140 .
  • the insertion member 134 is connected to the light source unit 136 via the light guide 138 , and light from the light source unit 136 is supplied to the insertion member 134 .
  • the insertion member 134 may be, for example, formed of a material not having flexibility, or formed of a material having flexibility.
  • the medical observation apparatus 100 may be called a rigid endoscope or a flexible endoscope, depending on the material forming the insertion member 134 .
  • the light source unit 136 is connected to the insertion member 134 via the light guide 138 .
  • the light source unit 136 supplies light to the insertion member 134 via the light guide 138 .
  • the light source unit 136 has, for example, plural light sources that emit light of different wavelengths.
  • the plural light sources that the light source unit 136 has include: a light source that emits red light, a light source that emits green light, and a light source that emits blue light.
  • the light source that emits red light may be, for example, one or more red light emitting diodes.
  • the light source that emits green light may be, for example, one or more green light emitting diodes.
  • the light source that emits blue light may be, for example, one or more blue light emitting diodes.
  • the plural light sources that the light source unit 136 has are not limited to the above described examples.
  • the light source unit 136 may have the plural light sources on a single chip, or may have the plural light sources on plural chips.
  • the light source unit 136 is connected wiredly or wirelessly to the control unit 144 , and light emission in the light source unit 136 is controlled by the control unit 144 .
  • Light supplied to the insertion member 134 is emitted from the distal end of the insertion member 134 , and an observation target, such as a tissue in a body cavity of a patient, is irradiated with the emitted light. Reflected light from the observation target is condensed by the optical system in the insertion member 134 .
  • the camera head 140 has a function of capturing an image of an observation target.
  • the camera head 140 is connected to the control unit 144 via the cable 142 , which is a signal transmission member.
  • the camera head 140 has an image sensor, captures an image of an observation target by photoelectrically converting reflected light from the observation target, the reflected light having been condensed by the insertion member 134 , and outputs an image signal (a signal representing a captured medical image) acquired by imaging, to the control unit 144 via the cable 142 .
  • the image sensor that the camera head 140 has may be, for example, an image sensor having, used therein, plural imaging elements, such as CMOSs or CCDs.
  • the insertion member 134 serves as “an imaging device that is inserted into a body of a patient and captures an image of the interior of the body”.
  • the medical observation apparatus 100 that functions as an endoscope apparatus includes, for example, plural imaging devices that function as a so-called stereo camera.
  • the optical system may be a Galilean optical system, or a Greenough-type optical system.
  • the control unit 144 controls the imaging devices. More specifically, the control unit 144 controls each of the light source unit 136 and the camera head 140 .
  • control unit 144 includes a communication device (not illustrated in the drawings), and transmits an image signal output from the camera head 140 , to the display device 200 by arbitrary wireless communication or arbitrary wired communication.
  • the control unit 144 may transmit an image signal and a display control signal, to the display device 200 .
  • the communication device (not illustrated in the drawings) included in the control unit 144 may be, for example: an IEEE 802.15.1 port and a transmitting and receiving circuit (wireless communication); an IEEE 802.11 port and a transmitting and receiving circuit (wireless communication); a communication antenna and an RF circuit (wireless communication); an optical communication device (wired communication or wireless communication); or a LAN terminal and a transmitting and receiving circuit (wired communication).
  • the communication device (not illustrated in the drawings) may be configured to be able to perform communication with one or more external devices by plural communication methods.
  • control unit 144 may perform predetermined processing on an image signal output to the camera head 140 , and transmit the image signal that has been subjected to the predetermined processing, to the display device 200 .
  • predetermined processing on the image signal include: white balance adjustment, image enlargement or reduction related to an electronic zoom function, and correction among pixels.
  • the control unit 144 may store therein a captured medical image that is based on an image signal.
  • the control unit 144 may be, for example, a camera control unit (CCU).
  • CCU camera control unit
  • the medical observation apparatus 100 that functions as an endoscope apparatus has, for example, the hardware configuration described by reference to FIG. 4 .
  • the insertion member 134 , the light source unit 136 , and the camera head 140 serve as an imaging device, and imaging in the imaging device is controlled by the control unit 144 .
  • FIG. 5 is a functional block diagram illustrating an example of a configuration of the medical observation apparatus 100 according to the embodiment.
  • the medical observation apparatus 100 includes, for example, the imaging unit 150 , a communication unit 152 , and a control unit 154 .
  • the imaging unit 150 captures an image of an observation target.
  • the imaging unit 150 is formed of, for example, “the imaging device 106 ” (for the medical observation apparatus 100 illustrated in FIG. 1 ), or “the insertion member 134 , the light source unit 136 , and the camera head 140 ” (for the medical observation apparatus 100 illustrated in FIG. 4 ). Imaging in the imaging unit 150 is controlled by, for example, the control unit 154 .
  • the communication unit 152 is a communication means included in the medical observation apparatus 100 , and plays a role of performing communication wirelessly or wiredly with an external device, such as the display device 200 .
  • the communication unit 152 is formed of, for example, the above described communication device (not illustrated in the drawings). Communication in the communication unit 152 is controlled by, for example, the control unit 154 .
  • the control unit 154 is formed of, for example, the above described processor (not illustrated in the drawings), and plays a role of controlling the whole medical observation apparatus 100 . Furthermore, the control unit 154 plays a role of proactively performing processing related to the display control method described later.
  • the processing related to the display control method in the control unit 154 may be distributed and performed among plural processing circuits (for example, plural processors) More specifically, the control unit 154 has, for example, an imaging control unit 156 and a processing unit 158 .
  • the imaging control unit 156 controls the imaging device forming the imaging unit 150 .
  • the control of the imaging device may be, for example, control of one or more functions generally included in an electronic imaging microscope, such as control of the zoom functions (the optical zoom function and electronic zoom function) and the AF function.
  • the processing unit 158 performs the processing related to the later described display control method according to the embodiment.
  • An example of a functional configuration of the processing unit 158 , and an example of the processing related to the display control method according to the embodiment will be described later.
  • the control unit 154 plays a role of proactively performing the processing related to the display control method according to the embodiment. Furthermore, by including, for example, the imaging control unit 156 and the processing unit 158 , the control unit 154 plays a role of controlling the whole medical observation apparatus 100 .
  • control unit 154 The functional configuration of the control unit 154 is not limited to the example illustrated in FIG. 5 .
  • control unit 154 may have any configuration according to a way in which functions that the medical observation apparatus 100 has are divided, such as a configuration according to a way in which the processing related to the display control method according to the embodiment is divided.
  • the control unit 154 may further have an arm control unit (not illustrated in the drawings) that controls driving of the arm 104 .
  • the control of the driving of the arm 104 may be “application of control signals that control driving, to the actuators (not illustrated in the drawings) respectively corresponding to the joints 110 a , 110 b , 110 c , 110 d , 110 e , and 110 f”.
  • the medical observation apparatus 100 performs the processing related to the later described display control method according to the embodiment by means of, for example, the functional configuration illustrated in FIG. 5 .
  • the functional configuration of the medical observation apparatus according to the embodiment is not limited to the configuration illustrated in FIG. 5 .
  • the medical observation apparatus may include one or both of the imaging control unit 156 and processing unit 158 both illustrated in FIG. 5 , individually from the control unit 154 (for example, by realization of one or both of them by means of another processing circuit).
  • the functional configuration that enables execution of the processing related to the display control method according to the embodiment in the medical observation apparatus according to the embodiment is not limited to the configuration illustrated in FIG. 5 , and for example, the medical observation apparatus according to the embodiment may have a functional configuration according to a way in which the processing related to the display control method according to the embodiment is divided.
  • the medical observation apparatus according to the embodiment has the configuration illustrated in FIG. 1
  • the medical observation apparatus according to the embodiment has an arm unit (not illustrated in the drawings) formed of the arm 104 .
  • the arm 104 forming the arm unit (not illustrated in the drawings) supports the imaging device 106 that forms the imaging unit 150 .
  • the medical observation apparatus may be provided without the communication unit 152 .
  • the medical observation apparatus according to the embodiment may be provided without the control unit 154 .
  • the medical control device (not illustrated in the drawings) performs the processing related to the later described display control method according to the embodiment by including a control unit having functions and a configuration similar to those of the control unit 154 , and controls operation in each component, such as the imaging unit 150 included in the medical observation apparatus according to the embodiment.
  • the medical control device controls operation in each component included in the medical observation apparatus according to the embodiment by performing communication with the medical observation apparatus according to the embodiment via a communication device included in the medical control device or an external communication device connected to the medical control device.
  • the medical observation apparatus according to the embodiment may be configured without a part of the functions of the control unit 154 .
  • the display control method according to the embodiment will be described next.
  • a case where the medical observation apparatus 100 performs the processing related to the display control method according to the embodiment will be described as an example.
  • the processing related to the display control method according to the embodiment may be performed by the medical control device (not illustrated in the drawings).
  • a surgical operation (a so-called microsurgery) performed by use of the medical observation apparatus 100 , for example, like in an example described below, measurement of a length of a lesion in the field of view for observation by a surgical operator may be needed.
  • a laparoscopic (abdominal incisional) hernia surgical operation performed by use of the medical observation apparatus 100 illustrated in FIG. 4 (for example, a rigid endoscope), for trimming in a mesh shape that closes a hernial orifice and provides a sufficient overlapping region, measurement of a length of the hernial orifice is needed.
  • a rotator cuff repair surgical operation performed by use of the medical observation apparatus 100 illustrated in FIG. 4 (for example, a rigid endoscope) and by use of a patching method for a rotator cuff tear, for determination of a size of a fascia patch necessary for repair of a torn region of a rotator cuff, measurement of a length of the torn region is needed.
  • the medical observation apparatus 100 illustrated in FIG. 4 for example, a rigid endoscope
  • a patching method for a rotator cuff tear for determination of a size of a fascia patch necessary for repair of a torn region of a rotator cuff, measurement of a length of the torn region is needed.
  • lengths of lesions may be measured by use of measuring tapes.
  • this method where measurement is performed by use of a measuring tape, for example, maneuvers are stopped by insertion of the measuring tape in a lesion region, and labor, such as disposal or cleaning of the measuring tape, is generated, and thus this method is hardly a convenient measuring method.
  • the length is sometimes estimated by a surgical operator relying on experience and feeling instead of performing measurement.
  • an estimate by experience and feeling doe not necessarily enable accurate measurement of a length of a lesion.
  • surgical clipping for a cerebral aneurysm if the estimation accuracy of the length is low, a process of trial and error is caused in the selection of a clip, for example, and thus undesired circumstances, such as waste of clips and increase in the risk of damage in the aneurysm, may be caused.
  • the medical observation apparatus 100 thus causes an annotation image to be displayed on the display screen of the display device 200 , the annotation image indicating a distance between positions of two points in an observation target.
  • a distance between positions of two points in an observation target may simply be referred to as a “distance between two points”
  • the positions of the two points in the observation target are each acquired based on predetermined operation on the observation target.
  • the predetermined operation on the observation target includes, for example, operation where a predetermined medical tool, such as forceps, or a position detecting probe is moved into an imaging range of the imaging device 106 that functions as the imaging unit 150 .
  • the positions in the observation target are, for example, acquired by “detection of the predetermined medical tool from one or both of a captured right eye medical image and a captured left eye medical image”, that is, “detection of an object corresponding to the predetermined operation, from one or both of the captured right eye medical image and the captured left eye medical image”.
  • the detection of the object corresponding to the predetermined operation from the captured medical image/images is performed by, for example, any image processing that enables an object to be detected from an image, such as detection by pattern matching.
  • a position of a specific portion of the object detected from the captured medical image/images corresponds to a position acquired based on the predetermined operation on the observation target.
  • a position in a plane corresponding to that captured medical image is determined as a position in an observation target.
  • the position in the observation target is represented by two-dimensional coordinates having an arbitrary position as the origin.
  • a spatial position corresponding to the captured medical images is determined as a position in an observation target.
  • the position in the observation target is represented by three-dimensional coordinates having an arbitrary position as the origin.
  • One or both of image processing related to the above described detection and image processing related to the determination of a position may be performed by the medical observation apparatus 100 , or may be performed in a device, such as the medical control device (not illustrated in the drawings), which is external to the medical observation apparatus 100 . Furthermore, the image processing related to the detection and the image processing related the determination of a position may be performed by the medical observation apparatus 100 and the external device in cooperation with each other.
  • a position in an observation target, the position being based on the position detecting probe, is acquired by, for example, “detection of a spatial position of the position detecting probe by the above described navigation device”. If a result of the detection in the navigation device is used, the medical observation apparatus 100 determines, as the position in the observation target, a spatial position indicated by positional information, by acquiring the positional information from the navigation device (an example of a detecting device that detects a position in an observation target). In this case, because the medical observation apparatus 100 does not need to perform the image processing related to the detection and the image processing related to the determination of the position, the processing load in the medical observation apparatus 100 is reduced, and the configuration of the medical observation apparatus 100 is able to be simplified.
  • a distance between two points corresponding to positions of two points determined as described above is acquired by, for example, calculation of a Euclidean distance between the acquired positions of the two points.
  • the distance between two points is not limited to the Euclidean distance, and may be a distance found by any method that enables representation of the distance between two points.
  • the processing for the above described calculation of a distance may be performed by the medical observation apparatus 100 , or may be performed in a device, such as the medical control device (not illustrated in the drawings), which is external to the medical observation apparatus 100 .
  • FIG. 6 is an explanatory diagram for explanation of an example of an annotation image displayed by the display control method according to the embodiment.
  • FIG. 7 is an explanatory diagram for explanation of another example of the annotation image displayed by the display control method according to the embodiment.
  • “A” represents an example of a captured medical image before the annotation image is displayed.
  • Examples of the captured medical image represented by “A” in FIG. 6 include one or both of a captured right eye medical image and a captured left eye medical image.
  • “B” represents an example of positions acquired for the captured medical image represented by “A” in FIG. 6 , and this example represented by “B” in FIG. 6 illustrates two positions P 1 and P 2 .
  • “C” represents an example of a captured medical image after an annotation image AO corresponding to the positions P 1 and P 2 illustrated in FIG. 6 has been displayed over the captured medical image represented by “A” in FIG. 6 .
  • FIG. 7 illustrate an example of a captured medical image after plural annotation images have been displayed over the captured medical image represented by “A” in FIG. 6 , and in FIG. 7 , an example where annotation images AO 1 and AO 2 have been displayed is illustrated.
  • the medical observation apparatus 100 displays the annotation image AO superimposed on one or both of the captured right eye medical image and the captured left eye medical image.
  • a medical worker such as a surgical operator, is able to visually recognize the length of a lesion in the field of view for observation.
  • the medical observation apparatus 100 is able to display the plural annotation images AO 1 and AO 2 superimposed on one or both of the captured right eye medical image and the captured left eye medical image.
  • the medical observation apparatus 100 displays the annotation images corresponding respectively to the acquired pairs of positions of two points, the annotation images having been superimposed on the captured medical image.
  • a medical worker such as a surgical operator, is able to visually recognize the lengths of plural lesions in the field of view for observation at once.
  • annotation image according to the embodiment are not limited to images displaying numerical values of distances between two points, as represented by “C” in FIG. 6 and illustrated in FIG. 7 .
  • the annotation image according to the embodiment may be “an image of a medical tool corresponding to a distance between two points, such as an image of a clip corresponding to a distance between two points”, “a model number of a medical tool corresponding to a distance between two points”, or “an image indicating a name of a medical tool corresponding to a distance between two points”.
  • an annotation image is displayed based on operation where a predetermined medical tool, such as forceps, is moved into the imaging range of the imaging device 106 , or operation where the position detecting probe is moved into the imaging range of the imaging device 106 . That is, when the display control method according to the embodiment is used, an annotation image is displayed by operation that is higher in relevance to ““medical intervention performed by a medical worker through use of the medical observation apparatus 100 ” than GUI manipulation is. Therefore, a possibility that medical intervention is interrupted by predetermined operation on an observation target according to the embodiment is lower than that in a case where GUI manipulation according to the technique described in Japanese Translation of PCT International Application, Publication No. 2007-521864 is performed.
  • FIG. 8 is a functional block diagram illustrating a first example of a configuration of the processing unit 158 included in the medical observation apparatus 100 according to the embodiment.
  • a “right eye image signal” illustrated in FIG. 8 is an image signal representing a captured right eye medical image captured by a first imaging device (the same applying hereinafter to the other drawings).
  • a “left eye image signal” illustrated in FIG. 8 is an image signal representing a captured left eye medical image captured by a second imaging device (the same applying hereinafter to the other drawings).
  • the processing unit 158 has, for example, an acquiring unit 160 , a distance calculating unit 162 , and a display control unit 164 .
  • the acquiring unit 160 acquires positions of at least two points in an observation target, the positions being determined based on predetermined operation on the observation target.
  • the acquiring unit 160 acquires the positions in the observation target by, for example, detecting an object corresponding to predetermined operation, from one or both of a captured right eye medical image and a captured left eye medical image, as described above. Furthermore, the acquiring unit 160 may acquire the positions in the observation target by, for example, acquiring positional information from a navigation device (an example of a detecting device, the same applying hereinafter) as described above.
  • a navigation device an example of a detecting device, the same applying hereinafter
  • the acquiring unit 160 transmits information indicating the acquired positions, to the distance calculating unit 162 .
  • the position indicating information may be, for example, data representing the two-dimensional coordinates, or data representing the three-dimensional space coordinates.
  • the acquiring unit 160 may transmit, as the position indicating information, the positional information acquired from the navigation device, to the distance calculating unit 162 , or may transmit, as the position indicating information, information resulting from some sort of processing on the positional information, to the distance calculating unit 162 .
  • FIG. 9 is a flow chart illustrating an example of processing in the acquiring unit 160 that the processing unit 158 included in the medical observation apparatus 100 according to the embodiment has.
  • FIG. 9 illustrates an example of processing “in a case where positions in an observation target are acquired by detection of an object from a captured medical image, the object corresponding to predetermined operation”.
  • the acquiring unit 160 determines whether or not a predetermined medical tool is in the field of view for observation (S 100 ). If the predetermined medical tool has been detected by any image processing that enables detection of an object from an image, such as detection by pattern matching, for example, the acquiring unit 160 determines that the predetermined medical tool is in the filed of view.
  • Step S 100 If it is not determined that the predetermined medical tool is in the filed of view at Step S 100 , the acquiring unit 160 repeats Step S 100 .
  • the acquiring unit 160 determines whether or not a position acquiring manipulation has been detected (S 102 ).
  • the position acquiring manipulation may be any manipulation, for example, “a manipulation of a manipulation device included in the medical observation apparatus 100 ”, “a manipulation of a manipulation device, such as a foot switch FS, which is external to the medical observation apparatus 100 ”, “a manipulation through a gesture”, or “a manipulation through voice”.
  • the acquiring unit 160 identifies the position acquiring manipulation, based on, for example, a manipulation signal corresponding to the manipulation.
  • the acquiring unit 160 identifies the position acquiring manipulation through gesture, based on, for example, “a gesture detection result of detection by arbitrary image processing on a captured image having a manipulation detection target captured therein”.
  • the image processing related to the gesture detection may be performed by the medical observation apparatus 100 , or may be performed in a device external to the medical observation apparatus 100 .
  • the acquiring unit 160 identifies the position acquiring manipulation through voice, based on, for example, “a predetermined voice detection result of detection by arbitrary signal processing on voice acquired by a voice input device, such as a microphone”.
  • the voice input device may be a voice input device included in the medical observation apparatus 100 , or a voice input device external to the medical observation apparatus 100 .
  • the signal processing related to the position acquiring manipulation for voice may be performed by the medical observation apparatus 100 , or may be performed in a device external to the medical observation apparatus 100 .
  • Step S 102 If it is not determined that a position acquiring manipulation has been detected at Step S 102 , the acquiring unit 160 repeats the processing from Step S 100 .
  • the acquiring unit 160 acquires positions corresponding to a predetermined medical tool, based on a captured medical image (S 104 ).
  • the acquiring unit 160 acquires positions in an observation target by performing, for example, the processing illustrated in FIG. 9 . As described above, the processing in the acquiring unit 160 is not limited to the example illustrated in FIG. 9 .
  • the distance calculating unit 162 calculates, based on position indicating information transmitted from the acquiring unit 160 , a distance corresponding to the acquired positions of two points. If plural pairs of positions of two points have been acquired by the acquiring unit 160 , the distance calculating unit 162 calculates a distance corresponding to each of the acquired pairs of positions of two points. The distance calculating unit 162 calculates, for example, a Euclidean distance between two points.
  • the distance calculating unit 162 transmits information indicating the calculated distance between two points, to the display control unit 164 .
  • the position indicating information may be, for example, numerical data representing the distance. If plural distances between two points have been calculated, the distance calculating unit 162 transmits information indicating distances respectively corresponding to the pairs of positions of two points, to the display control unit 164 .
  • the information indicating distances may be, for example, numerical data representing the distances.
  • FIG. 10 is a flow chart illustrating an example of processing in the distance calculating unit 162 that the processing unit 158 included in the medical observation apparatus according to the embodiment has.
  • the distance calculating unit 162 determines whether or not position indicating information transmitted from the acquiring unit 160 indicates positions for two points (S 200 ). For example, the distance calculating unit 162 sets a pair of positions of two points in the order that position indicating information is transmitted, and determines that the position indicating information indicates positions for two points when the pair of positions of two points has been set. Furthermore, if position indicating information includes identification information (for example, an ID) indicating a pair of positions of two points, the pair of positions of two points may be set according to the identification information, and the position indicating information may be determined to indicate positions for two points when the pair of positions of two points has been set.
  • identification information for example, an ID
  • Step S 200 If it is not determined that the position indicating information indicates positions for two points at Step S 200 , the distance calculating unit 162 repeats Step S 200 until it is determined that the position indicating information indicates positions for two points.
  • the distance calculating unit 162 calculates a distance between the two points (S 202 ).
  • the distance calculating unit 162 calculates a distance corresponding to the acquired positions of two points.
  • the processing in the distance calculating unit 162 is not limited to the example illustrated in FIG. 10 .
  • the display control unit 164 causes a captured right eye medical image, a captured left eye medical image, and an annotation image, to be displayed on the display screen of the display device 200 , the annotation image indicating a distance between two points having their positions acquired.
  • an image signal representing the captured right eye medical image output from the display control unit 164 , an image signal representing the captured left eye medical image, and an image signal representing the annotation image are written as “image signals” (the same applying to the figures related to the processing unit 158 according to another example described later).
  • the display control unit 164 reads an annotation image corresponding to information indicating a distance, from a recording medium (not illustrated in the drawings) that functions as a storage unit (not illustrated in the drawings), and causes the read annotation image to be displayed on the display screen of the display device 200 . If information indicating distances respectively corresponding to pairs of positions of two points has been transmitted, that is, if plural pairs of positions of two points have been acquired, the display control unit 164 causes annotation images respectively corresponding to the distances between two points to be displayed on the display screen.
  • An annotation image corresponding to information indicating a distance is identified by reference to a table (or a database) having distances and annotation images associated with each other.
  • the display control unit 164 causes an annotation image to be displayed superimposed on one or both of a captured right eye medical image and a captured left eye medical image, for example, as represented by “C” in FIG. 6 and illustrated in FIG. 7 .
  • the processing unit 158 that performs the processing related to the display control method according to the first example causes an annotation image as represented by “C” in FIG. 6 or illustrated in FIG. 7 to be displayed with a captured medical image, by means of, for example, the functional configuration illustrated in FIG. 8 .
  • the functional configuration of the processing unit according to the first example is not limited to the example illustrated in FIG. 8 .
  • the display control unit 164 may have a function of the distance calculating unit 162 .
  • the processing in the distance calculating unit 162 is performed in a device external to the medical observation apparatus 100
  • the processing unit 158 according to the first example may be provided without the distance calculating unit 162 .
  • the display control unit 164 causes an annotation image to be displayed on the display screen, the annotation image corresponding to information indicating a distance acquired from the external device.
  • the processing unit 158 according to the first example may have a configuration according to a way in which the processing related to the display control method according to the embodiment is divided.
  • FIG. 11 is a functional block diagram illustrating a second example of a configuration of the processing unit 158 included in the medical observation apparatus 100 according to the embodiment.
  • the processing unit 158 has, for example, the acquiring unit 160 , the distance calculating unit 162 , and a display control unit 166 .
  • the processing unit 158 according to the second example illustrated in FIG. 11 basically has the same functions as the processing unit 158 according to the first example illustrated in FIG. 8 , but functions that the display control unit 166 has are different from the functions that the display control unit 164 illustrated in FIG. 8 has. Parts of a functional configuration of the processing unit 158 according to the second example will be described below, the parts being different from those of the processing unit 158 according to the first example illustrated in FIG. 8 , and description of the same parts will be omitted.
  • the display control unit 166 causes a captured right eye medical image, a captured left eye medical image, and an annotation image, to be displayed on the display screen of the display device 200 .
  • the display control unit 166 changes the display of the annotation image correspondingly to any change in one or both of position and posture of the imaging unit 150 .
  • a change in one or both of the position and posture of the imaging unit 150 corresponds to a change in one or both of position and posture of the imaging device 106 that functions as the imaging unit 150 .
  • a change in the position and a change in the posture of the imaging device 106 are detected by, for example, angle sensors (not illustrated in the drawings) respectively provided in the above described joints 110 a , 110 b , 110 c , 110 d , 110 e , and 110 f .
  • a change in the position and a change in the posture of the imaging device 106 may be detected by any method that enables these changes to be detected.
  • FIG. 11 data representing a change in the position and a change in the posture of the imaging device 106 are illustrated as “imaging position and posture information”.
  • the captured right eye medical image and the captured left eye medical image are changed.
  • positions of two points in an observation target acquired based on predetermined operation on the observation target are not changed. Therefore, the distance between the two points acquired based on the predetermined operation on the observation target is not changed even if one or both of the position and posture of the imaging unit 150 is/are changed.
  • “Realization of display of an annotation image with a captured medical image that has been changed” is enabled by: the predetermined operation being performed on the observation target again; and the above described processing related to the first example being performed by the medical observation apparatus 100 again.
  • a medical worker such as a surgical operator, needs to perform the predetermined operation on the observation target every time one or both of the position and posture of the imaging unit 150 is changed, the medical worker may feel that this process is burdensome.
  • the display control unit 166 thus moves the positions of two points on the display screen (hereinafter, referred to as the “display positions of two points”, distinctively from the positions of two points in the observation target) correspondingly to the change in one or both of the position and posture of the imaging unit 150 .
  • the display positions of two points after this movement may be found by, for example, “calculation by use of the display positions of two points before the movement and an affine matrix corresponding to the change in one or both of the position and posture of the imaging unit 150 ”.
  • the method of determining the display positions of two points after the movement is not particularly limited.
  • the display control unit 166 then causes an annotation image to be displayed, the annotation image corresponding to the display positions of two points after the movement.
  • the annotation image corresponding to the display positions of two points after the movement may be, for example, “an image resulting from rotation, enlargement or reduction, parallel translation, or any combination thereof, of the annotation image before the movement, in a three-dimensional space”.
  • the display control unit 166 reads an annotation image corresponding to information indicating a distance and display positions of two points, from a recording medium (not illustrated in the drawings), and causes the read annotation image to be displayed on the display screen of the display device 200 .
  • the recording medium (not illustrated in the drawings) has, stored therein beforehand, for example, “an image resulting from rotation, enlargement or reduction, parallel translation, or any combination thereof, of an annotation image, in a three-dimensional space”, for each combination of display positions of two points before movement and display positions of two points after the movement”.
  • the display control unit 166 reads an image corresponding to a combination of display positions of two points before movement and display positions of two points after the movement as the annotation image corresponding to the display positions of two points after the movement, from the recording medium (not illustrated in the drawings), and causes the read annotation image to be displayed on the display screen of the display device 200 .
  • the annotation image corresponding to the display positions of two points after the movement is not limited to the example described above.
  • the display control unit 166 may perform affine transformation on a reference image that is an annotation image read from a recording medium similarly to the processing related to the display control method according to the first example, and thereby generate an annotation image corresponding to display positions of two points after movement.
  • the processing unit 158 that performs the processing related to the display control method according to the second examples basically has the same configuration as the processing unit 158 according to the first example illustrated in FIG. 8 . Therefore, similarly to the processing unit 158 according to the first example illustrated in FIG. 8 , the processing unit 158 according to the second example is able to cause an annotation image as represented by “C” in FIG. 6 or illustrated in FIG. 7 , to be displayed together with a captured medical image.
  • the processing unit 158 according to the second example automatically changes display of an annotation image correspondingly to any change in one or both of position and posture of the imaging unit 150 . That is, a medical worker who uses the medical observation apparatus 100 that performs the processing related to the display control method according to the second example does not need to perform predetermined operation on an observation target every time one or both of the position and posture of the imaging unit 150 is changed.
  • the medical observation apparatus 100 that performs the processing related to the display control method according to the second example enables improvement in the convenience for a user of the medical observation apparatus 100 , more than the medical observation apparatus 100 that performs the processing related to the display control method according to the first example.
  • the functional configuration of the processing unit 158 according to the second example is not limited to the example illustrated in FIG. 11 .
  • the display control unit 166 may have a function of the distance calculating unit 162 .
  • the processing in the distance calculating unit 162 is performed in a device external to the medical observation apparatus 100
  • the processing unit 158 according to the second example may be provided without the distance calculating unit 162 .
  • the display control unit 166 causes an annotation image to be displayed on the display screen, the annotation image corresponding to information indicating a distance acquired from the external device.
  • processing unit 158 according to the second example may have a configuration according to a way in which the processing related to the display control method according to the embodiment is divided.
  • FIG. 12 is a functional block diagram illustrating a third example of the configuration of the processing unit 158 included in the medical observation apparatus 100 according to the embodiment.
  • the processing unit 158 has, for example, the acquiring unit 160 , the distance calculating unit 162 , and a display control unit 168 .
  • the processing unit 158 according to the third example illustrated in FIG. 12 basically has the same functions as the processing unit 158 according to the first example illustrated in FIG. 8 , but functions that the display control unit 168 has are different from the functions that the display control unit 164 illustrated in FIG. 8 has. Parts of a functional configuration of the processing unit 158 according to the third example will be described below, the parts being different from those of the processing unit 158 according to the first example illustrated in FIG. 8 , and description of the same parts will be omitted.
  • the display control unit 168 causes a captured right eye medical image, a captured left eye medical image, and an annotation image, to be displayed on the display screen of the display device 200 .
  • the display control unit 168 causes an image to be displayed on the display screen, the image representing a medical tool corresponding to a distance between two points.
  • the display control unit 168 causes an image to be displayed on one or both of the display screen of the display device 200 where the captured right eye medical image and captured left eye medical image are displayed and a display screen of another display device, the image representing a medical tool.
  • Examples of the image representing a medical tool corresponding to a distance between two points include an image of a clip corresponding to a distance between two points, as described already as another example of the annotation image.
  • the image representing a medical tool corresponding to a distance between two points is not limited to an image of clip, and may be an image of any medical tool corresponding to an operative surgical procedure, such as a surgical needle or a surgical suture.
  • the display control unit 168 reads an image representing a medical tool corresponding to a distance between two points, from a recording medium (not illustrated in the drawings) that functions as a storage unit (not illustrated in the drawings), and causes the read image to be displayed on the display screen of the display device 200 .
  • Data representing an image representing a medical tool corresponding to a distance between two points are illustrated as “image information” in FIG. 12 .
  • FIG. 13 is an explanatory diagram for explanation of an example of an image displayed by the display control method according to the embodiment.
  • FIG. 13 illustrates “an example where images M 1 , M 2 , and M 3 (examples of images representing medical tools, the same applying hereinafter) of clips corresponding to a distance between two points are being displayed further, in addition to a captured medical image having the annotation image represented by “C” in FIG. 6 superimposed thereon”.
  • the image displayed by the display control method according to the third example is not limited to the example illustrated in FIG. 13 .
  • an image of a clip corresponding to a distance between two points may be displayed at the position of the annotation image AO illustrated in FIG. 13 . That is, an image representing a medical tool may be displayed superimposed on one or both of a captured right eye medical image and a captured left eye medical image.
  • the processing unit 158 that performs processing related to the display control method according to the third example basically has the same configuration as the processing unit 158 according to the first example illustrated in FIG. 8 . Therefore, similarly to the processing unit 158 according to the first example illustrated in FIG. 8 , the processing unit 158 according to the third example is able to cause an annotation image as represented by “C” in FIG. 6 or illustrated in FIG. 7 , to be displayed, together with a captured medical image.
  • the processing unit 158 causes an image to be displayed on the display screen, the image representing a medical tool corresponding to a distance between two points.
  • a medical worker such as a surgical operator, is able to select a medical tool corresponding to a distance between two points easily.
  • the medical observation apparatus 100 that performs the processing related to the display control method according to the third example enables improvement of the convenience for a user of the medical observation apparatus 100 , more than the medical observation apparatus 100 that performs the processing related to the display control method according to the first example.
  • the functional configuration of the processing unit 158 according to the third example is not limited to the example illustrated in FIG. 12 .
  • the display control unit 168 may have a function of the distance calculating unit 162 .
  • the processing in the distance calculating unit 162 is performed in a device external to the medical observation apparatus 100
  • the processing unit 158 according to the third example may be provided without the distance calculating unit 162 .
  • the display control unit 168 causes an annotation image to be displayed on the display screen, the annotation image corresponding to information indicating a distance acquired from the external device.
  • processing unit 158 according to the third example may have a configuration according to a way in which the processing related to the display control method according to the embodiment is divided.
  • the processing related to the display control method according to the embodiment is not limited to the above described first to third examples.
  • the processing unit 158 included in the medical observation apparatus 100 according to the embodiment may perform processing that is a combination of the processing according to the second example and the processing according to the third example.
  • a medical worker such as a surgical operator, is able to perceive a result of measurement of a distance between two points in the field of view for observation in a captured medical image in real time, and thus use of an additional tool for measurement, such as a measuring tape, is not needed and improvement in efficiency of the surgical operation is able to be expected.
  • Convenience for a user of the medical observation apparatus according to the embodiment is able to be improved by execution of a program (for example, a program that enables execution of the processing related to the display control method according to the embodiment) by a processor or the like in a computer system, the program being for causing the computer system to function as the medical observation apparatus.
  • the computer system according to the embodiment may be a single computer, or plural computers.
  • the processing related to the display control method according to the embodiment is executed by the computer system according to the embodiment.
  • the program (a computer program) for causing the computer system to function as the medical observation apparatus according to the embodiment is provided, but according to the embodiment, a recording medium storing therein the program may also be provided.
  • a medical observation apparatus comprising:
  • an imaging device configured to capture an observation target to obtain a captured right eye medical image and a captured left eye medical image
  • circuitry configured to:
  • the circuitry causes the annotation image to be displayed, the annotation image indicating the calculated distance.
  • the imaging device is supported by the arm.

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Surgery (AREA)
  • Physics & Mathematics (AREA)
  • Animal Behavior & Ethology (AREA)
  • Public Health (AREA)
  • Radiology & Medical Imaging (AREA)
  • Optics & Photonics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • General Health & Medical Sciences (AREA)
  • Pathology (AREA)
  • Veterinary Medicine (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Endoscopes (AREA)
  • Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
  • Studio Devices (AREA)
  • Processing Or Creating Images (AREA)
  • Closed-Circuit Television Systems (AREA)
US16/525,587 2018-08-06 2019-07-30 Medical observation apparatus Active US11638000B2 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
JPJP2018-147585 2018-08-06
JP2018-147585 2018-08-06
JP2018147585A JP2020022563A (ja) 2018-08-06 2018-08-06 医療用観察装置

Publications (2)

Publication Number Publication Date
US20200045293A1 US20200045293A1 (en) 2020-02-06
US11638000B2 true US11638000B2 (en) 2023-04-25

Family

ID=69229256

Family Applications (1)

Application Number Title Priority Date Filing Date
US16/525,587 Active US11638000B2 (en) 2018-08-06 2019-07-30 Medical observation apparatus

Country Status (2)

Country Link
US (1) US11638000B2 (ja)
JP (2) JP2020022563A (ja)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
USD914207S1 (en) * 2018-01-03 2021-03-23 Karl Storz Se & Co. Kg Instrument holding device for surgery
DE112019003447T5 (de) * 2018-07-06 2021-03-18 Sony Corporation Medizinisches Beobachtungssystem, medizinisches Beobachtungsgerät und Antriebsverfahren für das medizinische Beobachtungsgerät
JP2023069343A (ja) * 2021-11-05 2023-05-18 学校法人帝京大学 手術用デジタル顕微鏡システムおよび手術用デジタル顕微鏡システムの表示制御方法

Citations (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002253480A (ja) 2001-03-02 2002-09-10 Olympus Optical Co Ltd 医療処置補助装置
JP2007521864A (ja) 2004-01-19 2007-08-09 コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ 医療画像に対する柔軟な測定機能を提供する方法及び装置
US20090003671A1 (en) * 2006-03-08 2009-01-01 Olympus Medical Systems Corp. Medical image processing apparatus and medical image processing method
JP2009014711A (ja) 2007-06-04 2009-01-22 Olympus Corp 計測用内視鏡装置およびプログラム
US20090073257A1 (en) * 2006-03-14 2009-03-19 Olympus Medical Systems Corp. Medical image processing apparatus and medical image processing method
JP2009168499A (ja) 2008-01-11 2009-07-30 Olympus Corp 内視鏡装置およびプログラム
US20090220133A1 (en) * 2006-08-24 2009-09-03 Olympus Medical Systems Corp. Medical image processing apparatus and medical image processing method
US20090306514A1 (en) * 2008-06-10 2009-12-10 Kabushiki Kaisha Toshiba Ultrasound imaging apparatus and method for displaying ultrasound image
JP2011145527A (ja) 2010-01-15 2011-07-28 Olympus Corp 内視鏡装置およびプログラム
JP2012075507A (ja) 2010-09-30 2012-04-19 Panasonic Corp 手術用カメラ
JP2012147857A (ja) 2011-01-17 2012-08-09 Olympus Medical Systems Corp 画像処理装置
US20130009958A1 (en) * 2010-03-31 2013-01-10 Fujifilm Corporation Projection image generation apparatus, program and method
JP2013005830A (ja) 2011-06-22 2013-01-10 Fujifilm Corp 内視鏡システム、プロセッサ装置及び撮影距離測定方法
JP2013137466A (ja) 2011-12-28 2013-07-11 Olympus Corp 内視鏡画像処理装置及び内視鏡画像処理用プログラム
JP2014010098A (ja) 2012-07-02 2014-01-20 Olympus Corp 計測装置、計測方法、およびプログラム
US20140037177A1 (en) * 2011-04-06 2014-02-06 Canon Kabushiki Kaisha Information processing apparatus
JP2014066788A (ja) 2012-09-25 2014-04-17 Sony Corp 画面表示装置及び画面表示システム
JP2014094246A (ja) 2012-11-12 2014-05-22 Honda Electronic Co Ltd 医療画像測定装置及び医療画像測定方法
WO2014156378A1 (ja) 2013-03-27 2014-10-02 オリンパスメディカルシステムズ株式会社 内視鏡システム
US20150030229A1 (en) * 2013-07-24 2015-01-29 Anja Borsdorf Methods for Updating 2D/3D Registration on Movement and Computing Device
US20150085081A1 (en) * 2012-05-30 2015-03-26 Olympus Medical Systems Corp. Medical three-dimensional observation apparatus
JP2015104447A (ja) 2013-11-29 2015-06-08 株式会社アルム 顕微鏡用映像処理装置、および医療用顕微鏡システム
JP2015211824A (ja) 2014-04-17 2015-11-26 オリンパス株式会社 内視鏡画像表示装置、内視鏡画像表示方法及び内視鏡画像表示プログラム
US20160165222A1 (en) * 2014-12-08 2016-06-09 Sony Olympus Medical Solutions Inc. Medical stereoscopic observation apparatus, medical stereoscopic observation method, and program
JP2016131866A (ja) 2015-01-18 2016-07-25 鈴木　秀幸 鏡視下手術用超音波式距離計測装置
EP2358269B1 (en) * 2007-03-08 2019-04-10 Sync-RX, Ltd. Image processing and tool actuation for medical procedures

Patent Citations (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002253480A (ja) 2001-03-02 2002-09-10 Olympus Optical Co Ltd 医療処置補助装置
JP2007521864A (ja) 2004-01-19 2007-08-09 コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ 医療画像に対する柔軟な測定機能を提供する方法及び装置
US20090003671A1 (en) * 2006-03-08 2009-01-01 Olympus Medical Systems Corp. Medical image processing apparatus and medical image processing method
US20090073257A1 (en) * 2006-03-14 2009-03-19 Olympus Medical Systems Corp. Medical image processing apparatus and medical image processing method
US20090220133A1 (en) * 2006-08-24 2009-09-03 Olympus Medical Systems Corp. Medical image processing apparatus and medical image processing method
EP2358269B1 (en) * 2007-03-08 2019-04-10 Sync-RX, Ltd. Image processing and tool actuation for medical procedures
JP2009014711A (ja) 2007-06-04 2009-01-22 Olympus Corp 計測用内視鏡装置およびプログラム
JP2009168499A (ja) 2008-01-11 2009-07-30 Olympus Corp 内視鏡装置およびプログラム
US20090306514A1 (en) * 2008-06-10 2009-12-10 Kabushiki Kaisha Toshiba Ultrasound imaging apparatus and method for displaying ultrasound image
JP2011145527A (ja) 2010-01-15 2011-07-28 Olympus Corp 内視鏡装置およびプログラム
US20130009958A1 (en) * 2010-03-31 2013-01-10 Fujifilm Corporation Projection image generation apparatus, program and method
JP2012075507A (ja) 2010-09-30 2012-04-19 Panasonic Corp 手術用カメラ
JP2012147857A (ja) 2011-01-17 2012-08-09 Olympus Medical Systems Corp 画像処理装置
US20140037177A1 (en) * 2011-04-06 2014-02-06 Canon Kabushiki Kaisha Information processing apparatus
JP2013005830A (ja) 2011-06-22 2013-01-10 Fujifilm Corp 内視鏡システム、プロセッサ装置及び撮影距離測定方法
JP2013137466A (ja) 2011-12-28 2013-07-11 Olympus Corp 内視鏡画像処理装置及び内視鏡画像処理用プログラム
US20150085081A1 (en) * 2012-05-30 2015-03-26 Olympus Medical Systems Corp. Medical three-dimensional observation apparatus
JP2014010098A (ja) 2012-07-02 2014-01-20 Olympus Corp 計測装置、計測方法、およびプログラム
JP2014066788A (ja) 2012-09-25 2014-04-17 Sony Corp 画面表示装置及び画面表示システム
JP2014094246A (ja) 2012-11-12 2014-05-22 Honda Electronic Co Ltd 医療画像測定装置及び医療画像測定方法
WO2014156378A1 (ja) 2013-03-27 2014-10-02 オリンパスメディカルシステムズ株式会社 内視鏡システム
US20150030229A1 (en) * 2013-07-24 2015-01-29 Anja Borsdorf Methods for Updating 2D/3D Registration on Movement and Computing Device
JP2015104447A (ja) 2013-11-29 2015-06-08 株式会社アルム 顕微鏡用映像処理装置、および医療用顕微鏡システム
JP2015211824A (ja) 2014-04-17 2015-11-26 オリンパス株式会社 内視鏡画像表示装置、内視鏡画像表示方法及び内視鏡画像表示プログラム
US20160165222A1 (en) * 2014-12-08 2016-06-09 Sony Olympus Medical Solutions Inc. Medical stereoscopic observation apparatus, medical stereoscopic observation method, and program
JP2016131866A (ja) 2015-01-18 2016-07-25 鈴木　秀幸 鏡視下手術用超音波式距離計測装置

Also Published As

Publication number Publication date
US20200045293A1 (en) 2020-02-06
JP2023099552A (ja) 2023-07-13
JP2020022563A (ja) 2020-02-13

Similar Documents

Publication Publication Date Title
CN109715106B (zh) 控制装置、控制方法及医疗系统
US11638000B2 (en) Medical observation apparatus
JP2017113343A (ja) 医療用撮像装置及び手術ナビゲーションシステム
US11510751B2 (en) Medical observation apparatus
JP7392654B2 (ja) 医療用観察システム、医療用観察装置及び医療用観察方法
US20220401049A1 (en) Medical display control device, medical observation device, display control method, and medical observation system
JP7034636B2 (ja) 医療用観察装置、および医療用観察システム
JP7163282B2 (ja) 医療用観察装置、処理方法、および医療用観察システム
JP6976720B2 (ja) 医療用観察装置、およびズーム制御方法
CN110461205A (zh) 手术成像系统、手术用图像处理设备以及用于控制成像过程的方法
JP6965338B2 (ja) 医療用観察装置、制御装置、及び観察視野補正方法
JP2023095970A (ja) 医療用観察システム
WO2018139156A1 (ja) 医療用観察装置、および制御方法
US20190282063A1 (en) Medical image processing apparatus, medical observation apparatus, and image processing method
JP6957605B2 (ja) 医療用観察システム、および制御方法
US20190261841A1 (en) Medical control apparatus, medical observation apparatus, and control method
JP7134656B2 (ja) 医療用表示制御装置、および表示制御方法
US11648082B2 (en) Medical holding device, and medical observation device
US11464402B2 (en) Medical dimming control apparatus and dimming control method
JP7143092B2 (ja) 医療用画像処理装置、医療用観察装置、および画像処理方法

Legal Events

Date Code Title Description
FEPP Fee payment procedure

Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

AS Assignment

Owner name: SONY OLYMPUS MEDICAL SOLUTIONS INC., JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:HONDA, TAKESHI;REEL/FRAME:050044/0528

Effective date: 20190730

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: FINAL REJECTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE AFTER FINAL ACTION FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: ADVISORY ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: FINAL REJECTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE AFTER FINAL ACTION FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: ADVISORY ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

STCF Information on status: patent grant

Free format text: PATENTED CASE