US10856722B2 - Image processing apparatus and electronic endoscope system - Google Patents

Image processing apparatus and electronic endoscope system Download PDF

Info

Publication number
US10856722B2
US10856722B2 US15/556,425 US201715556425A US10856722B2 US 10856722 B2 US10856722 B2 US 10856722B2 US 201715556425 A US201715556425 A US 201715556425A US 10856722 B2 US10856722 B2 US 10856722B2
Authority
US
United States
Prior art keywords
observation image
narrow
component
image
band
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US15/556,425
Other versions
US20180184889A1 (en
Inventor
Takao Makino
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hoya Corp
Original Assignee
Hoya Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hoya Corp filed Critical Hoya Corp
Assigned to HOYA CORPORATION reassignment HOYA CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: MAKINO, TAKAO
Publication of US20180184889A1 publication Critical patent/US20180184889A1/en
Application granted granted Critical
Publication of US10856722B2 publication Critical patent/US10856722B2/en
Active legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
    • A61B1/00009—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
    • A61B1/045—Control thereof
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
    • A61B1/00009—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope
    • A61B1/000094—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope extracting biological structures
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00163—Optical arrangements
    • A61B1/00186—Optical arrangements with imaging filters
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
    • A61B1/05—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances characterised by the image sensor, e.g. camera, being in the distal end portion
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/06—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements
    • A61B1/0638—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements providing two or more wavelengths
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/06—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements
    • A61B1/0646—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements with illumination filters
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/06—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements
    • A61B1/0661—Endoscope light sources
    • A61B1/0669—Endoscope light sources at proximal end of an endoscope
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B23/00—Telescopes, e.g. binoculars; Periscopes; Instruments for viewing the inside of hollow bodies; Viewfinders; Optical aiming or sighting devices
    • G02B23/24—Instruments or systems for viewing the inside of hollow bodies, e.g. fibrescopes
    • G02B23/2407—Optical details
    • G02B23/2461—Illumination
    • G02B23/2469—Illumination using optical fibres
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection

Definitions

  • the present invention relates to an image processing apparatus and an electronic endoscope system.
  • Patent Documents 1 and 2 disclose an electronic endoscope system that acquires a normal observation image captured using normal light and a narrow-band observation image captured using narrow-band light that has a bandwidth narrower than that of normal light, and displays the images side by side on one screen.
  • Patent Document 1 JP 2004-321244A
  • Patent Document 2 JP 2015-223249A
  • the present invention is made based on awareness of the problems above, and one object thereof is to provide an image processing apparatus and an electronic endoscope system that can acquire an observation image that is natural overall and has excellent visibility, while highlighting characteristic parts such as affected parts and parts that are to be examined.
  • An image processing apparatus includes: an observation image input unit to which a normal observation image captured using normal light and a narrow-band observation image captured using narrow-band light that has a bandwidth narrower than that of normal light are input; a characteristic part extraction unit that extracts a characteristic part that is included in the narrow-band observation image; and a highlight display unit that displays a single image in which a part of the normal observation image, the part corresponding to a characteristic part that is included in the narrow-band observation image, is highlighted using the characteristic part.
  • An electronic endoscope system includes: an electronic endoscope that acquires a normal observation image captured using normal light and a narrow-band observation image captured using narrow-band light that has a bandwidth narrower than that of normal light; and an image processing apparatus that performs image processing on the normal observation image and the narrow-band observation image, wherein the image processing apparatus includes: a characteristic part extraction unit that extracts a characteristic part that is included in the narrow-band observation image; and a highlight display unit that displays a single image in which a part of the normal observation image, the part corresponding to a characteristic part that is included in the narrow-band observation image, is highlighted using the characteristic part.
  • the highlight display unit can display the single image in which the part of the normal observation image, the part corresponding to the characteristic part that is included in the narrow-band observation image, is combined with the characteristic part.
  • the characteristic part extraction unit can extract, as the characteristic part, a surface layer characteristic component that is obtained by subtracting a B component of the narrow-band observation image from an R component of the normal observation image, and the highlight display unit can subtract the surface layer characteristic component from a G component and a B component of the normal observation image.
  • the characteristic part extraction unit can extract, as the characteristic part, a deep layer characteristic component that is obtained by subtracting a G component of the narrow-band observation image from the R component of the normal observation image, and further subtracting the surface layer characteristic component therefrom, and the highlight display unit can subtract the deep layer characteristic component from the R component of the normal observation image.
  • the highlight display unit can display the single image in which the part of the normal observation image, the part corresponding to the characteristic part that is included in the narrow-band observation image, is replaced with the characteristic part.
  • modes in which the highlight display unit displays a highlighted single image includes a mode in which a part of a normal observation image, the part corresponding to a characteristic part that is included in a narrow-band observation image, is combined with the characteristic part, and a mode in which the part of the normal observation image, the part corresponding to the characteristic part that is included in the narrow-band observation image, is replaced with the characteristic part. That is, “to highlight” is used as a concept that includes “to combine” and “to replace”.
  • the present invention can provide an image processing apparatus and an electronic endoscope system that can acquire an observation image that is natural overall and has excellent visibility, while highlighting characteristic parts such as affected parts and parts that are to be examined.
  • FIG. 1 is a block diagram showing a configuration of an electronic endoscope system according to an embodiment of the present invention.
  • FIG. 2 is a diagram showing blood vessels, which are characteristic parts in a surface layer and a deep layer of a subject tissue in a narrow-band observation image.
  • FIGS. 3A and 3B are diagrams showing R, G, and B wavelength components corresponding to surface layer information and deep layer information when normal light and narrow-band light are used.
  • FIG. 4 is a diagram showing a method for computing a surface layer characteristic component and a deep layer characteristic component.
  • FIG. 5 is a flowchart showing image processing according to the embodiment of the present invention.
  • the electronic endoscope system 1 includes an electronic endoscope 10 , a processor (an image processing apparatus and a light source apparatus) 20 , and a monitor 30 .
  • the electronic endoscope 10 includes a grip and control portion that is to be gripped by an operator, an insertion portion that is flexible and extends from the grip and control portion, a universal tube that extends from the grip and control portion in a direction away from the insertion portion, and a connector that is provided at the leading end of the universal tube.
  • Light-guide fibers 11 are built into the electronic endoscope 10 .
  • the light-guide fibers 11 extend to the inside of the connector via the insertion portion, the grip and operation portion, and the universal tube of the electronic endoscope 10 .
  • the connector of the electronic endoscope 10 is connected to a connector of the processor 20 , and thus the electronic endoscope 10 and the processor 20 are optically connected to each other.
  • Illumination light normal light or narrow-band light described below
  • a filter-equipped light source 21 which is built into the processor 20
  • an illumination lens 12 which is provided at the leading end of the insertion portion of the electronic endoscope 10 , according to a predetermined light distribution pattern.
  • the filter-equipped light source 21 includes a high-intensity lamp (e.g. a xenon lamp, a halogen lamp, a mercury lamp, or a metal halide lamp) that emits white light that includes wavelength bands respectively corresponding to R (Red), G (Green), and B (blue).
  • the filter-equipped light source 21 also includes a filter unit that is located on the light path of the white light emitted from the high-intensity lamp.
  • the filter unit includes a rotary filter turret that is provided with a filter for the color white, which allows white light from the high-intensity lamp to pass, thereby generating normal light, and a narrow-band filter, which narrows the wavelength band of the white light emitted from the high-intensity lamp, thereby generating narrow-band light.
  • the narrow-band filter has a spectral transmittance with a narrow width at half maximum, for each of the R, G and B wavelength bands.
  • the rotary filter turret of the filter unit is driven to rotate, and thus white light from the high-intensity lamp alternatingly passes through the filter for the color white and the narrow-band filter, and normal light and narrow-band light that has a bandwidth narrower than that of normal light are alternatingly emitted from the filter-equipped light source 21 .
  • the wavelength band of narrow-band light can be set as desired as long as it is narrower than the wavelength band of normal light.
  • narrow-band light may have a wavelength band that matches the spectral properties of hemoglobin.
  • the filter-equipped light source 21 is well known as disclosed in Patent Document 2 above, for example, and therefore a further detailed description thereof is omitted.
  • An imaging unit 13 is provided at the leading end of the insertion portion of the electronic endoscope 10 .
  • the imaging unit 13 is composed of a plurality of constituent elements that include an objective lens 13 a and a CCD 13 b that captures a subject image that has passed through the objective lens 13 a , and are integrated into one piece using a resin material such as an epoxy resin.
  • the CCD 13 b alternatingly acquires a normal observation image signal and a narrow-band observation image signal that are respectively based on normal light and narrow-band light that are alternatingly emitted from the filter-equipped light source 21 via the light-guide fibers 11 and the illumination lens 12 .
  • the normal observation image signal and the narrow-band observation image signal are transmitted to the processor 20 via a signal transmission cable 14 .
  • the processor 20 includes a control unit 22 , an observation image input unit (an image input processing unit) 23 , an image processing unit (a computation unit) 24 , an image memory 25 , and a display processing unit 26 .
  • the control unit 22 totally controls all of the constituent elements of the processor 20 .
  • the observation image input unit 23 performs input processing on the normal observation image signal and the narrow-band observation image signal transmitted from the signal transmission cable 14 of the electronic endoscope 10 , to input the signals as a normal observation image and a narrow-band observation image.
  • the image processing unit 24 performs image processing on the normal observation image and the narrow-band observation image input to the observation image input unit 23 .
  • the image processing unit 24 includes a characteristic part extraction unit (an image feature calculation unit) 24 a and a highlight display unit (a display result creation unit) 24 b.
  • the characteristic part extraction unit 24 a extracts characteristic parts that are included in a narrow-band observation image input to the observation image input unit 23 . More specifically, the characteristic part extraction unit 24 a extracts, as a characteristic part of the narrow-band observation image, a surface layer characteristic component that is obtained by subtracting the B component of the narrow-band observation image from the R component of the normal observation image input to the observation image input unit 23 . Furthermore, the characteristic part extraction unit 24 a extracts, as a characteristic part of the narrow-band observation image, a deep layer characteristic component that is obtained by subtracting the G component of the narrow-band observation image from the R component of the normal observation image input to the observation image input unit 23 , and further subtracting the above-described surface layer characteristic component therefrom.
  • the highlight display unit 24 b combines the characteristic part of the narrow-band observation image extracted by the characteristic part extraction unit 24 a , with a part, which corresponds to the characteristic part, of the normal observation image input to the observation image input unit 23 , thereby generating a single combined observation image (a single image) in which the characteristic part is highlighted. More specifically, the highlight display unit 24 b subtracts the surface layer characteristic component extracted by the characteristic part extraction unit 24 a , from the G component and the B component of the normal observation image input to the observation image input unit 23 , and also subtracts the deep layer characteristic component extracted by the characteristic part extraction unit 24 a , from the R component of the normal observation image input to the observation image input unit 23 .
  • FIG. 2 is a diagram showing blood vessels, which are characteristic parts in the surface layer and the deep layer of subject tissue in a narrow-band observation image.
  • FIGS. 3A and 3B are diagrams showing R, G, and B wavelength components corresponding to surface layer information and deep layer information when normal light and narrow-band light are used.
  • the R component is a dominant color component and has the smallest feature value, and therefore it can be said that the G component and the B component are the color components that are suitable for the purpose of highlighting a feature.
  • a narrow-band observation image includes a large amount of information regarding the surface layer of tissue in the B component, and a large amount of information regarding the deep layer of tissue in the G component.
  • blood vessels in the surface layer of tissue are responsive to (absorb a large amount of) light that has the wavelength of the B component, and are likely to be highlighted by the B component
  • blood vessels in the deep layer of tissue are responsive to (absorb a large amount of) light that has the wavelength of the G component, and are likely to be highlighted by the G component. Therefore, by selectively using the above-described surface layer characteristic component and deep layer characteristic component, it is possible to highlight the vessels in the surface layer of tissue and the vessels in the deep layer of tissue such that they appear different from each other.
  • FIG. 4 is a diagram showing a method for computing a surface layer characteristic component and a deep layer characteristic component.
  • the surface layer characteristic component is obtained by subtracting the B component of the narrow-band observation image from the R component of the normal observation image, and removing color components that have a value no greater than a predetermined threshold value, from the result of subtraction.
  • the deep layer characteristic component is obtained by subtracting the G component of the narrow-band observation image from the R component of the normal observation image, removing color components that have a value no greater than a predetermined threshold value, from the result of subtraction, to obtain the sum of the surface layer characteristic component and the deep layer characteristic component, and subtracting the above-described surface layer characteristic component from the sum.
  • the predetermined threshold value may be a fixed value, and may be dynamically calculated and set based on, for example, the average of the values of the R, G, and B components. It is also possible to omit the processing performed to remove color components that have a value no greater than the predetermined threshold value (to set the threshold value to zero).
  • the image memory 25 stores a combined observation image (a single image) in which the characteristic parts of the narrow-band observation image have been highlighted by the image processing unit 24 .
  • the display processing unit 26 displays the combined observation image (the single image) stored in the image memory 25 on the monitor 30 .
  • step S 1 the image processing unit 24 acquires an observation image of the current frame.
  • step S 2 the image processing unit 24 determines whether the observation image of the current frame acquired in step S 1 is a normal observation image or a narrow-band observation image. This determination processing is performed by, for example, detecting the rotational phase of the filter unit (the rotary filter turret) of the filter-equipped light source 21 .
  • step S 3 the image processing unit 24 reads out the observation image of the previous frame, which is previous to the current frame from which the observation image was acquired in step S 1 . Since a normal observation image and a narrow-band observation image are alternatingly output one at each frame, the observation image of the current frame acquired in step S 1 and the observation image of the previous frame read out in step S 3 constitute one set composed of a normal observation image and a narrow-band observation image. Also, the normal observation image and the narrow-band observation image that constitute one set appear different only due to the wavelength bands of their irradiation light being different, and hence they can be regarded as substantially identical subject images. Furthermore, by using a normal observation image and a narrow-band observation image that constitute one set, it is possible to display a combined observation image (a single image) without lowering the frame rate.
  • step S 4 the characteristic part extraction unit 24 a of the image processing unit 24 extracts (calculates) the surface layer characteristic component based on the R component of the normal observation image and the B component of the narrow-band observation image.
  • step S 5 the characteristic part extraction unit 24 a of the image processing unit 24 extracts (calculates) the deep layer characteristic component based on the R component of the normal observation image and the G component of the narrow-band observation image as well as the surface layer characteristic component extracted in step S 4 .
  • step S 6 the highlight display unit 24 b of the image processing unit 24 subtracts the deep layer characteristic component extracted in step S 5 , from the R component of the normal observation image.
  • step S 7 the highlight display unit 24 b of the image processing unit 24 subtracts the surface layer characteristic component extracted in step S 4 , from the G component and the B component of the normal observation image.
  • step S 6 and the processing in step S 7 can be determined as desired, and it is possible to simultaneously perform the processing in these steps.
  • step S 8 the display processing unit 26 displays, on the monitor 30 , an observation image (a combined observation image, a single image) in which the characteristic parts of the narrow-band observation image have been highlighted by the image processing unit 24 .
  • step S 9 the combined observation image in which the characteristic parts of the narrow-band observation image have been highlighted by the image processing unit 24 is stored in the image memory 25 .
  • the combined observation image stored in the image memory 25 will be externally saved after observation, and will be used as a material for more detailed diagnosis or explanation, for example.
  • step S 10 the processor 20 determines whether or not image capturing by the electronic endoscope 10 is complete. Upon the processor 20 determining that image capturing performed by the electronic endoscope 10 is complete (step S 10 : Yes), processing from step S 1 to step S 9 is terminated. Upon the processor 20 determining that image capturing performed by the electronic endoscope 10 is not complete (step S 10 : No), the processing loop from step S 1 to step S 9 is repeatedly performed.
  • the characteristic part extraction unit 24 a extracts a characteristic part that is included in a narrow-band observation image
  • the highlight display unit 24 b displays a single image in which a part of a normal observation image, which corresponds to the characteristic part included in the narrow-band observation image, is highlighted using the characteristic part. Therefore, it is possible to highlight characteristic parts such as affected parts and parts that are to be examined, and to perform treatment (an operation) or examination with high accuracy.
  • parts other than the characteristic parts such as affected parts and parts that are to be examined are maintained so as to have the color and so on of the normal observation image, and the observation image with the highlighted parts is displayed in a large size on the entire screen.
  • the observation image can be natural and can have excellent visibility overall. Also, images of two frames that have been successively captured are used to suppress a decrease in the frame rate to the minimum, and also from this point of view, natural image display is realized.
  • the embodiment above describes an example in which the highlight display unit 24 b combines the characteristic part of the narrow-band observation image extracted by the characteristic part extraction unit 24 a , with a part, which corresponds to the characteristic part, of the normal observation image input to the observation image input unit 23 , thereby generating a combined observation image in which the characteristic part is highlighted.
  • the method for generating the combined observation image employed by the highlight display unit 24 b is not limited to the method described in the embodiment above, and various design changes are applicable. For example, it is possible to change the combination of the R, G, and B components and the combination of the addition and subtraction of the surface layer characteristic component and the deep layer characteristic component according to the purpose of the observation image, and it is possible to increase or reduce the surface layer characteristic component and the deep layer characteristic component by multiplying them by a given coefficient before performing addition or subtraction. Also, it is possible to employ various kinds of computation such as multiplication and division as appropriate.
  • the highlight display unit 24 b to replace a part of the normal observation image input to the observation image input unit 23 , the part corresponding to the characteristic part of the narrow-band observation image extracted by the characteristic part extraction unit 24 a , with the characteristic part, thereby generating a partially-replaced observation image (a single image) in which the characteristic part is highlighted.
  • this mode it is possible to further clarify the boundary between the characteristic part of the partially-replaced observation image and the other part.
  • the image processing apparatus and the electronic endoscope system according to the present invention can be preferably employed as an image processing apparatus and an electronic endoscope system in the field of medical endoscopes, for example.

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Surgery (AREA)
  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Optics & Photonics (AREA)
  • Medical Informatics (AREA)
  • General Health & Medical Sciences (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Pathology (AREA)
  • Veterinary Medicine (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Animal Behavior & Ethology (AREA)
  • Biophysics (AREA)
  • Public Health (AREA)
  • Heart & Thoracic Surgery (AREA)
  • General Physics & Mathematics (AREA)
  • Signal Processing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Astronomy & Astrophysics (AREA)
  • Quality & Reliability (AREA)
  • Endoscopes (AREA)
  • Instruments For Viewing The Inside Of Hollow Bodies (AREA)

Abstract

Provided are an image processing apparatus and an electronic endoscope system that can acquire an observation image that is natural overall and has excellent visibility, while highlighting characteristic parts such as affected parts and parts that are to be examined. A normal observation image captured using normal light and a narrow-band observation image captured using narrow-band light that has a bandwidth narrower than that of normal light are input to the observation image input unit. A characteristic part extraction unit extracts a characteristic part that is included in a narrow-band observation image. A highlight display unit displays a single image in which a part of the normal observation image, the part corresponding to a characteristic part that is included in the narrow-band observation image, is highlighted using the characteristic part.

Description

TECHNICAL FIELD
The present invention relates to an image processing apparatus and an electronic endoscope system.
BACKGROUND ART
Patent Documents 1 and 2 disclose an electronic endoscope system that acquires a normal observation image captured using normal light and a narrow-band observation image captured using narrow-band light that has a bandwidth narrower than that of normal light, and displays the images side by side on one screen.
PRIOR ART DOCUMENTS Patent Documents
Patent Document 1: JP 2004-321244A
Patent Document 2: JP 2015-223249A
SUMMARY OF INVENTION Problem to be Solved by Invention
However, since characteristic parts such as affected parts and parts that are to be examined are highlighted in a narrow-band observation image, it is inevitable that a narrow-band observation image has an unnatural appearance that is significantly different from the appearance of a normal observation image in terms of colors and so on. Also, since a normal observation image and a narrow-band observation image are divided in two on the screen, each image has a smaller size, which degrades visibility.
The present invention is made based on awareness of the problems above, and one object thereof is to provide an image processing apparatus and an electronic endoscope system that can acquire an observation image that is natural overall and has excellent visibility, while highlighting characteristic parts such as affected parts and parts that are to be examined.
Means for Solving Problem
An image processing apparatus according to one aspect of the present invention includes: an observation image input unit to which a normal observation image captured using normal light and a narrow-band observation image captured using narrow-band light that has a bandwidth narrower than that of normal light are input; a characteristic part extraction unit that extracts a characteristic part that is included in the narrow-band observation image; and a highlight display unit that displays a single image in which a part of the normal observation image, the part corresponding to a characteristic part that is included in the narrow-band observation image, is highlighted using the characteristic part.
An electronic endoscope system according to one aspect of the present invention includes: an electronic endoscope that acquires a normal observation image captured using normal light and a narrow-band observation image captured using narrow-band light that has a bandwidth narrower than that of normal light; and an image processing apparatus that performs image processing on the normal observation image and the narrow-band observation image, wherein the image processing apparatus includes: a characteristic part extraction unit that extracts a characteristic part that is included in the narrow-band observation image; and a highlight display unit that displays a single image in which a part of the normal observation image, the part corresponding to a characteristic part that is included in the narrow-band observation image, is highlighted using the characteristic part.
The highlight display unit can display the single image in which the part of the normal observation image, the part corresponding to the characteristic part that is included in the narrow-band observation image, is combined with the characteristic part.
The characteristic part extraction unit can extract, as the characteristic part, a surface layer characteristic component that is obtained by subtracting a B component of the narrow-band observation image from an R component of the normal observation image, and the highlight display unit can subtract the surface layer characteristic component from a G component and a B component of the normal observation image.
The characteristic part extraction unit can extract, as the characteristic part, a deep layer characteristic component that is obtained by subtracting a G component of the narrow-band observation image from the R component of the normal observation image, and further subtracting the surface layer characteristic component therefrom, and the highlight display unit can subtract the deep layer characteristic component from the R component of the normal observation image.
The highlight display unit can display the single image in which the part of the normal observation image, the part corresponding to the characteristic part that is included in the narrow-band observation image, is replaced with the characteristic part.
In the present description, modes in which the highlight display unit displays a highlighted single image includes a mode in which a part of a normal observation image, the part corresponding to a characteristic part that is included in a narrow-band observation image, is combined with the characteristic part, and a mode in which the part of the normal observation image, the part corresponding to the characteristic part that is included in the narrow-band observation image, is replaced with the characteristic part. That is, “to highlight” is used as a concept that includes “to combine” and “to replace”.
Advantageous Effects of Invention
The present invention can provide an image processing apparatus and an electronic endoscope system that can acquire an observation image that is natural overall and has excellent visibility, while highlighting characteristic parts such as affected parts and parts that are to be examined.
BRIEF DESCRIPTION OF DRAWINGS
FIG. 1 is a block diagram showing a configuration of an electronic endoscope system according to an embodiment of the present invention.
FIG. 2 is a diagram showing blood vessels, which are characteristic parts in a surface layer and a deep layer of a subject tissue in a narrow-band observation image.
FIGS. 3A and 3B are diagrams showing R, G, and B wavelength components corresponding to surface layer information and deep layer information when normal light and narrow-band light are used.
FIG. 4 is a diagram showing a method for computing a surface layer characteristic component and a deep layer characteristic component.
FIG. 5 is a flowchart showing image processing according to the embodiment of the present invention.
 DESCRIPTION OF EMBODIMENTS
The following describes an electronic endoscope system 1 according to an embodiment of the present invention with reference to FIGS. 1 to 5. The electronic endoscope system 1 includes an electronic endoscope 10, a processor (an image processing apparatus and a light source apparatus) 20, and a monitor 30.
Although details of the shape of the electronic endoscope 10 are omitted from FIG. 1, the electronic endoscope 10 includes a grip and control portion that is to be gripped by an operator, an insertion portion that is flexible and extends from the grip and control portion, a universal tube that extends from the grip and control portion in a direction away from the insertion portion, and a connector that is provided at the leading end of the universal tube.
Light-guide fibers 11 are built into the electronic endoscope 10. The light-guide fibers 11 extend to the inside of the connector via the insertion portion, the grip and operation portion, and the universal tube of the electronic endoscope 10. The connector of the electronic endoscope 10 is connected to a connector of the processor 20, and thus the electronic endoscope 10 and the processor 20 are optically connected to each other. Illumination light (normal light or narrow-band light described below) from a filter-equipped light source 21, which is built into the processor 20, is guided inside the light-guide fibers 11, and is emitted outward from an illumination lens 12, which is provided at the leading end of the insertion portion of the electronic endoscope 10, according to a predetermined light distribution pattern.
The filter-equipped light source 21 includes a high-intensity lamp (e.g. a xenon lamp, a halogen lamp, a mercury lamp, or a metal halide lamp) that emits white light that includes wavelength bands respectively corresponding to R (Red), G (Green), and B (blue). The filter-equipped light source 21 also includes a filter unit that is located on the light path of the white light emitted from the high-intensity lamp. The filter unit includes a rotary filter turret that is provided with a filter for the color white, which allows white light from the high-intensity lamp to pass, thereby generating normal light, and a narrow-band filter, which narrows the wavelength band of the white light emitted from the high-intensity lamp, thereby generating narrow-band light. The narrow-band filter has a spectral transmittance with a narrow width at half maximum, for each of the R, G and B wavelength bands. The rotary filter turret of the filter unit is driven to rotate, and thus white light from the high-intensity lamp alternatingly passes through the filter for the color white and the narrow-band filter, and normal light and narrow-band light that has a bandwidth narrower than that of normal light are alternatingly emitted from the filter-equipped light source 21. The wavelength band of narrow-band light can be set as desired as long as it is narrower than the wavelength band of normal light. For example, narrow-band light may have a wavelength band that matches the spectral properties of hemoglobin. The filter-equipped light source 21 is well known as disclosed in Patent Document 2 above, for example, and therefore a further detailed description thereof is omitted.
An imaging unit 13 is provided at the leading end of the insertion portion of the electronic endoscope 10. The imaging unit 13 is composed of a plurality of constituent elements that include an objective lens 13 a and a CCD 13 b that captures a subject image that has passed through the objective lens 13 a, and are integrated into one piece using a resin material such as an epoxy resin. The CCD 13 b alternatingly acquires a normal observation image signal and a narrow-band observation image signal that are respectively based on normal light and narrow-band light that are alternatingly emitted from the filter-equipped light source 21 via the light-guide fibers 11 and the illumination lens 12. The normal observation image signal and the narrow-band observation image signal are transmitted to the processor 20 via a signal transmission cable 14.
The processor 20 includes a control unit 22, an observation image input unit (an image input processing unit) 23, an image processing unit (a computation unit) 24, an image memory 25, and a display processing unit 26. The control unit 22 totally controls all of the constituent elements of the processor 20.
The observation image input unit 23 performs input processing on the normal observation image signal and the narrow-band observation image signal transmitted from the signal transmission cable 14 of the electronic endoscope 10, to input the signals as a normal observation image and a narrow-band observation image.
The image processing unit 24 performs image processing on the normal observation image and the narrow-band observation image input to the observation image input unit 23. The image processing unit 24 includes a characteristic part extraction unit (an image feature calculation unit) 24 a and a highlight display unit (a display result creation unit) 24 b.
The characteristic part extraction unit 24 a extracts characteristic parts that are included in a narrow-band observation image input to the observation image input unit 23. More specifically, the characteristic part extraction unit 24 a extracts, as a characteristic part of the narrow-band observation image, a surface layer characteristic component that is obtained by subtracting the B component of the narrow-band observation image from the R component of the normal observation image input to the observation image input unit 23. Furthermore, the characteristic part extraction unit 24 a extracts, as a characteristic part of the narrow-band observation image, a deep layer characteristic component that is obtained by subtracting the G component of the narrow-band observation image from the R component of the normal observation image input to the observation image input unit 23, and further subtracting the above-described surface layer characteristic component therefrom.
The highlight display unit 24 b combines the characteristic part of the narrow-band observation image extracted by the characteristic part extraction unit 24 a, with a part, which corresponds to the characteristic part, of the normal observation image input to the observation image input unit 23, thereby generating a single combined observation image (a single image) in which the characteristic part is highlighted. More specifically, the highlight display unit 24 b subtracts the surface layer characteristic component extracted by the characteristic part extraction unit 24 a, from the G component and the B component of the normal observation image input to the observation image input unit 23, and also subtracts the deep layer characteristic component extracted by the characteristic part extraction unit 24 a, from the R component of the normal observation image input to the observation image input unit 23.
FIG. 2 is a diagram showing blood vessels, which are characteristic parts in the surface layer and the deep layer of subject tissue in a narrow-band observation image. FIGS. 3A and 3B are diagrams showing R, G, and B wavelength components corresponding to surface layer information and deep layer information when normal light and narrow-band light are used. In an observation image obtained using an endoscope, the R component is a dominant color component and has the smallest feature value, and therefore it can be said that the G component and the B component are the color components that are suitable for the purpose of highlighting a feature. A narrow-band observation image includes a large amount of information regarding the surface layer of tissue in the B component, and a large amount of information regarding the deep layer of tissue in the G component. Therefore, blood vessels in the surface layer of tissue are responsive to (absorb a large amount of) light that has the wavelength of the B component, and are likely to be highlighted by the B component, and blood vessels in the deep layer of tissue are responsive to (absorb a large amount of) light that has the wavelength of the G component, and are likely to be highlighted by the G component. Therefore, by selectively using the above-described surface layer characteristic component and deep layer characteristic component, it is possible to highlight the vessels in the surface layer of tissue and the vessels in the deep layer of tissue such that they appear different from each other.
FIG. 4 is a diagram showing a method for computing a surface layer characteristic component and a deep layer characteristic component. As seen from this drawing, the surface layer characteristic component is obtained by subtracting the B component of the narrow-band observation image from the R component of the normal observation image, and removing color components that have a value no greater than a predetermined threshold value, from the result of subtraction. Also, the deep layer characteristic component is obtained by subtracting the G component of the narrow-band observation image from the R component of the normal observation image, removing color components that have a value no greater than a predetermined threshold value, from the result of subtraction, to obtain the sum of the surface layer characteristic component and the deep layer characteristic component, and subtracting the above-described surface layer characteristic component from the sum. Thus, only the surface layer characteristic component and the deep layer characteristic component of the narrow-band observation image are extracted. For example, by subtracting the deep layer characteristic component, which is unlikely to affect the overall color or the like, from the R component of the normal observation image, which has a small feature value, and further subtracting the original surface layer characteristic component from the G component and the B component, it is possible to highlight the characteristic parts by only changing the color and contrast of the characteristic parts without changing the color and contrast of the normal observation image. Note that the predetermined threshold value may be a fixed value, and may be dynamically calculated and set based on, for example, the average of the values of the R, G, and B components. It is also possible to omit the processing performed to remove color components that have a value no greater than the predetermined threshold value (to set the threshold value to zero).
The image memory 25 stores a combined observation image (a single image) in which the characteristic parts of the narrow-band observation image have been highlighted by the image processing unit 24. The display processing unit 26 displays the combined observation image (the single image) stored in the image memory 25 on the monitor 30.
The following describes image processing performed by the electronic endoscope system 1 and the processor 20 according to the present embodiment with reference to the flowchart shown in FIG. 5.
In step S1, the image processing unit 24 acquires an observation image of the current frame.
In step S2, the image processing unit 24 determines whether the observation image of the current frame acquired in step S1 is a normal observation image or a narrow-band observation image. This determination processing is performed by, for example, detecting the rotational phase of the filter unit (the rotary filter turret) of the filter-equipped light source 21.
In step S3, the image processing unit 24 reads out the observation image of the previous frame, which is previous to the current frame from which the observation image was acquired in step S1. Since a normal observation image and a narrow-band observation image are alternatingly output one at each frame, the observation image of the current frame acquired in step S1 and the observation image of the previous frame read out in step S3 constitute one set composed of a normal observation image and a narrow-band observation image. Also, the normal observation image and the narrow-band observation image that constitute one set appear different only due to the wavelength bands of their irradiation light being different, and hence they can be regarded as substantially identical subject images. Furthermore, by using a normal observation image and a narrow-band observation image that constitute one set, it is possible to display a combined observation image (a single image) without lowering the frame rate.
In step S4, the characteristic part extraction unit 24 a of the image processing unit 24 extracts (calculates) the surface layer characteristic component based on the R component of the normal observation image and the B component of the narrow-band observation image.
In step S5, the characteristic part extraction unit 24 a of the image processing unit 24 extracts (calculates) the deep layer characteristic component based on the R component of the normal observation image and the G component of the narrow-band observation image as well as the surface layer characteristic component extracted in step S4.
In step S6, the highlight display unit 24 b of the image processing unit 24 subtracts the deep layer characteristic component extracted in step S5, from the R component of the normal observation image.
In step S7, the highlight display unit 24 b of the image processing unit 24 subtracts the surface layer characteristic component extracted in step S4, from the G component and the B component of the normal observation image.
The order in which the processing in step S6 and the processing in step S7 are performed can be determined as desired, and it is possible to simultaneously perform the processing in these steps.
In step S8, the display processing unit 26 displays, on the monitor 30, an observation image (a combined observation image, a single image) in which the characteristic parts of the narrow-band observation image have been highlighted by the image processing unit 24.
In step S9, the combined observation image in which the characteristic parts of the narrow-band observation image have been highlighted by the image processing unit 24 is stored in the image memory 25. The combined observation image stored in the image memory 25 will be externally saved after observation, and will be used as a material for more detailed diagnosis or explanation, for example.
In step S10, the processor 20 determines whether or not image capturing by the electronic endoscope 10 is complete. Upon the processor 20 determining that image capturing performed by the electronic endoscope 10 is complete (step S10: Yes), processing from step S1 to step S9 is terminated. Upon the processor 20 determining that image capturing performed by the electronic endoscope 10 is not complete (step S10: No), the processing loop from step S1 to step S9 is repeatedly performed.
As described above, in the electronic endoscope system 1 and the processor 20 according to the present embodiment, the characteristic part extraction unit 24 a extracts a characteristic part that is included in a narrow-band observation image, and the highlight display unit 24 b displays a single image in which a part of a normal observation image, which corresponds to the characteristic part included in the narrow-band observation image, is highlighted using the characteristic part. Therefore, it is possible to highlight characteristic parts such as affected parts and parts that are to be examined, and to perform treatment (an operation) or examination with high accuracy. In addition, parts other than the characteristic parts such as affected parts and parts that are to be examined are maintained so as to have the color and so on of the normal observation image, and the observation image with the highlighted parts is displayed in a large size on the entire screen. Thus, the observation image can be natural and can have excellent visibility overall. Also, images of two frames that have been successively captured are used to suppress a decrease in the frame rate to the minimum, and also from this point of view, natural image display is realized.
The embodiment above describes an example in which the highlight display unit 24 b combines the characteristic part of the narrow-band observation image extracted by the characteristic part extraction unit 24 a, with a part, which corresponds to the characteristic part, of the normal observation image input to the observation image input unit 23, thereby generating a combined observation image in which the characteristic part is highlighted.
Here, the method for generating the combined observation image employed by the highlight display unit 24 b is not limited to the method described in the embodiment above, and various design changes are applicable. For example, it is possible to change the combination of the R, G, and B components and the combination of the addition and subtraction of the surface layer characteristic component and the deep layer characteristic component according to the purpose of the observation image, and it is possible to increase or reduce the surface layer characteristic component and the deep layer characteristic component by multiplying them by a given coefficient before performing addition or subtraction. Also, it is possible to employ various kinds of computation such as multiplication and division as appropriate.
Furthermore, it is also possible for the highlight display unit 24 b to replace a part of the normal observation image input to the observation image input unit 23, the part corresponding to the characteristic part of the narrow-band observation image extracted by the characteristic part extraction unit 24 a, with the characteristic part, thereby generating a partially-replaced observation image (a single image) in which the characteristic part is highlighted. In this mode, it is possible to further clarify the boundary between the characteristic part of the partially-replaced observation image and the other part.
INDUSTRIAL APPLICABILITY
The image processing apparatus and the electronic endoscope system according to the present invention can be preferably employed as an image processing apparatus and an electronic endoscope system in the field of medical endoscopes, for example.
DESCRIPTION OF REFERENCE SIGNS
    • 1 Electronic Endoscope System
    • 10 Electronic Endoscope
    • 11 Light Guide Fibers
    • 12 Illumination Lens
    • 13 Imaging Unit
    • 13 a Objective Lens
    • 13 b CCD
    • 14 Signal Transmission Cable
    • 20 Processor (Image Processing Apparatus, Light Source Apparatus)
    • 21 Filter-Equipped Light Source
    • 22 Control Unit
    • 23 Observation Image Input Unit (Image Input Processing Unit)
    • 24 Image Processing Unit (Computation Unit)
    • 24 a Characteristic Part Extraction Unit (Image Feature Calculation Unit)
    • 24 b Highlight Display Unit (Display Result Creation Unit)
    • 25 Image Memory
    • 26 Display Processing Unit
    • 30 Monitor

Claims (6)

The invention claimed is:
1. An image processing apparatus including a processor, the processor configured to:
capture an input normal observation image using normal light and an input narrow-band observation image using narrow-band light that has a bandwidth narrower than that of normal light;
extract a characteristic part that is included in the narrow-band observation image, wherein, as the characteristic part, a surface layer characteristic component that is obtained by subtracting a Blue component of the narrow-band observation image from an Red component of the normal observation image is extracted; and
display a single image in which a part of the normal observation image is highlighted, the part corresponding to a characteristic part of the normal observation image, wherein, in the single image in which the part of the normal observation image is highlighted, the part of the normal observation image corresponding to the characteristic part included in the narrow-band observation image is combined with the characteristic part of the normal observation image and the processor subtracts the surface layer characteristic component from a Green component and a Blue component of the normal observation image.
2. The image processing apparatus according to claim 1,
as the extracted characteristic part, a deep layer characteristic component is obtained by subtracting a Green component of the narrow-band observation image from the Red component of the normal observation image, and the surface layer characteristic component is therefrom, and
the single image display comprises subtracting the deep layer characteristic component from the Red component of the normal observation image.
3. The image processing apparatus according to claim 1,
wherein the display comprises displaying the single image in which the part of the normal observation image is highlighted, wherein, the part corresponding to the characteristic part that is included in the narrow-band observation image, is replaced with the characteristic part.
4. An electronic endoscope system comprising:
an electronic endoscope that acquires a normal observation image captured using normal light and a narrow-band observation image captured using narrow-band light that has a bandwidth narrower than that of normal light; and
an image processing apparatus that performs image processing on the normal observation image and the narrow-band observation image,
wherein the image processing apparatus includes a processor, the processor configured to:
extract a characteristic part that is included in the narrow-band observation image, wherein, as the characteristic part, a surface layer characteristic component that is obtained by subtracting a Blue component of the narrow-band observation image from a Red component of the normal observation image is extracted; and
display a single image in which a part of the normal observation image is highlighted, the part corresponding to a characteristic part of the normal observation image, wherein, in the single image in which the part of the normal observation image is highlighted, the part of the normal observation image corresponding to the characteristic part that is included in the narrow-band observation image is combined with the characteristic part of the normal observation image and the surface layer characteristic component is subtracted from a Green component and a Blue component of the normal observation image.
5. The electronic endoscope system according to claim 4,
as the extracted characteristic part, a deep layer characteristic component that is obtained by subtracting a Green component of the narrow-band observation image from the Red component of the normal observation image, and the surface layer characteristic component is subtracted therefrom, and
the single image display comprises subtracting the deep layer characteristic component from the Red component of the normal observation image.
6. The electronic endoscope system according to claim 4,
wherein the display comprises displaying the single image in which the part of the normal observation image is highlighted, wherein, the part corresponding to the characteristic part that is included in the narrow-band observation image, is replaced with the characteristic part.
US15/556,425 2016-06-02 2017-07-18 Image processing apparatus and electronic endoscope system Active 2038-07-05 US10856722B2 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
JP2016-110888 2016-06-02
JP2016110888A JP6626783B2 (en) 2016-06-02 2016-06-02 Image processing apparatus and electronic endoscope system
PCT/IB2017/054322 WO2017208216A1 (en) 2016-06-02 2017-07-18 Image processing device and electronic endoscope system

Publications (2)

Publication Number Publication Date
US20180184889A1 US20180184889A1 (en) 2018-07-05
US10856722B2 true US10856722B2 (en) 2020-12-08

Family

ID=60479219

Family Applications (1)

Application Number Title Priority Date Filing Date
US15/556,425 Active 2038-07-05 US10856722B2 (en) 2016-06-02 2017-07-18 Image processing apparatus and electronic endoscope system

Country Status (5)

Country Link
US (1) US10856722B2 (en)
JP (1) JP6626783B2 (en)
CN (1) CN109526203B (en)
DE (1) DE112017000023B4 (en)
WO (1) WO2017208216A1 (en)

Citations (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5833617A (en) * 1996-03-06 1998-11-10 Fuji Photo Film Co., Ltd. Fluorescence detecting apparatus
US20030197793A1 (en) * 2001-01-09 2003-10-23 Tomoo Mitsunaga Imaging device
US20040109068A1 (en) * 2001-01-09 2004-06-10 Tomoo Mitsunaga Image processing device
JP2004321244A (en) 2003-04-21 2004-11-18 Olympus Corp Electronic endoscope system
US20050134712A1 (en) * 2003-12-18 2005-06-23 Gruhlke Russell W. Color image sensor having imaging element array forming images on respective regions of sensor elements
US20050134698A1 (en) * 2003-12-18 2005-06-23 Schroeder Dale W. Color image sensor with imaging elements imaging on respective regions of sensor elements
US20090312629A1 (en) * 2008-06-13 2009-12-17 Inneroptic Technology Inc. Correction of relative tracking errors based on a fiducial
US20100030021A1 (en) * 2008-07-31 2010-02-04 Olympus Medical Systems Corp. Image Display Apparatus, Endoscope System Using the Same, and Image Display Method
US20100053213A1 (en) * 2008-09-01 2010-03-04 Fujifilm Corporation Medical image display apparatus and method, and program for displaying medical image
US20110063427A1 (en) * 2008-03-18 2011-03-17 Novadaq Technologies Inc. Imaging system for combined full-color reflectance and near-infrared imaging
US20110235877A1 (en) 2010-03-23 2011-09-29 Olympus Corporation Image processing device, image processing method, and program
US20110285879A1 (en) * 2010-03-31 2011-11-24 Canon Kabushiki Kaisha Image processing device and image pickup device using the same
US20120327205A1 (en) * 2009-12-28 2012-12-27 Olympus Corporation Image processing device, electronic apparatus, information storage device, and image processing method
US20130039562A1 (en) * 2010-04-28 2013-02-14 Olympus Corporation Image processing device and fluoroscopy device
JP2013150712A (en) 2012-01-25 2013-08-08 Fujifilm Corp Endoscope system, processor device for endoscope system, and image processing method
US20130293704A1 (en) * 2011-11-30 2013-11-07 Panasonic Corporation Imaging apparatus
US8643710B2 (en) 2010-09-30 2014-02-04 Olympus Medical Systems Corp. Image pickup apparatus
US9031294B2 (en) * 2010-07-23 2015-05-12 National Institute Of Advanced Industrial Science And Technology Region segmented image data creating system and feature extracting system for histopathological images
US9028396B2 (en) * 2011-01-27 2015-05-12 Fujifilm Corporation Endoscope system, processing unit therefor, and image processing method
JP2015223249A (en) 2014-05-27 2015-12-14 Hoya株式会社 Processor for endoscope
US20160089012A1 (en) * 2014-09-30 2016-03-31 Fujifilm Corporation Endoscope system and method for operating the same
US20160296106A1 (en) 2015-01-21 2016-10-13 Hoya Corporation Endoscope system
US20170209031A1 (en) * 2014-11-12 2017-07-27 Sony Corporation Image processing apparatus, image processing method, and program
US20170251912A1 (en) * 2015-01-21 2017-09-07 Olympus Corporation Endoscope device

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006261861A (en) * 2005-03-15 2006-09-28 Fuji Photo Film Co Ltd Imaging apparatus
JP4956694B2 (en) * 2010-09-29 2012-06-20 オリンパスメディカルシステムズ株式会社 Information processing apparatus and capsule endoscope system
JP5670264B2 (en) * 2011-06-13 2015-02-18 富士フイルム株式会社 Endoscope system and method for operating endoscope system
JP5362149B1 (en) * 2012-03-30 2013-12-11 オリンパスメディカルシステムズ株式会社 Endoscope device
JP6054806B2 (en) * 2013-04-25 2016-12-27 富士フイルム株式会社 Image processing apparatus and method for operating endoscope system
JP6412709B2 (en) * 2014-04-02 2018-10-24 オリンパス株式会社 Observation image acquisition system
JP2015231576A (en) * 2015-09-24 2015-12-24 富士フイルム株式会社 Endoscope system, processor device for endoscope system, and image processing method
JP6081622B2 (en) * 2016-01-07 2017-02-15 富士フイルム株式会社 ENDOSCOPE SYSTEM, PROCESSOR DEVICE, AND METHOD FOR OPERATING ENDOSCOPE SYSTEM

Patent Citations (29)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5833617A (en) * 1996-03-06 1998-11-10 Fuji Photo Film Co., Ltd. Fluorescence detecting apparatus
US20030197793A1 (en) * 2001-01-09 2003-10-23 Tomoo Mitsunaga Imaging device
US20040109068A1 (en) * 2001-01-09 2004-06-10 Tomoo Mitsunaga Image processing device
JP2004321244A (en) 2003-04-21 2004-11-18 Olympus Corp Electronic endoscope system
US20050134712A1 (en) * 2003-12-18 2005-06-23 Gruhlke Russell W. Color image sensor having imaging element array forming images on respective regions of sensor elements
US20050134698A1 (en) * 2003-12-18 2005-06-23 Schroeder Dale W. Color image sensor with imaging elements imaging on respective regions of sensor elements
US20110063427A1 (en) * 2008-03-18 2011-03-17 Novadaq Technologies Inc. Imaging system for combined full-color reflectance and near-infrared imaging
US20090312629A1 (en) * 2008-06-13 2009-12-17 Inneroptic Technology Inc. Correction of relative tracking errors based on a fiducial
US20100030021A1 (en) * 2008-07-31 2010-02-04 Olympus Medical Systems Corp. Image Display Apparatus, Endoscope System Using the Same, and Image Display Method
JP2010035637A (en) 2008-07-31 2010-02-18 Olympus Medical Systems Corp Image display apparatus and endoscope system using the same
US20100053213A1 (en) * 2008-09-01 2010-03-04 Fujifilm Corporation Medical image display apparatus and method, and program for displaying medical image
US20120327205A1 (en) * 2009-12-28 2012-12-27 Olympus Corporation Image processing device, electronic apparatus, information storage device, and image processing method
US8913111B2 (en) 2009-12-28 2014-12-16 Olympus Corporation Image processing device, electronic apparatus, information storage device, and image processing method
US20110235877A1 (en) 2010-03-23 2011-09-29 Olympus Corporation Image processing device, image processing method, and program
JP2011194111A (en) 2010-03-23 2011-10-06 Olympus Corp Image processing device, image processing method, and program
US20150305607A1 (en) 2010-03-23 2015-10-29 Olympus Corporation Image processing device, image processing method, and program
US20110285879A1 (en) * 2010-03-31 2011-11-24 Canon Kabushiki Kaisha Image processing device and image pickup device using the same
US20130039562A1 (en) * 2010-04-28 2013-02-14 Olympus Corporation Image processing device and fluoroscopy device
US9031294B2 (en) * 2010-07-23 2015-05-12 National Institute Of Advanced Industrial Science And Technology Region segmented image data creating system and feature extracting system for histopathological images
US8643710B2 (en) 2010-09-30 2014-02-04 Olympus Medical Systems Corp. Image pickup apparatus
US9028396B2 (en) * 2011-01-27 2015-05-12 Fujifilm Corporation Endoscope system, processing unit therefor, and image processing method
US20130293704A1 (en) * 2011-11-30 2013-11-07 Panasonic Corporation Imaging apparatus
JP2013150712A (en) 2012-01-25 2013-08-08 Fujifilm Corp Endoscope system, processor device for endoscope system, and image processing method
US20140316195A1 (en) * 2012-01-25 2014-10-23 Fujifilm Corporation Endoscope system, processor device of endoscope system, and image processing method
JP2015223249A (en) 2014-05-27 2015-12-14 Hoya株式会社 Processor for endoscope
US20160089012A1 (en) * 2014-09-30 2016-03-31 Fujifilm Corporation Endoscope system and method for operating the same
US20170209031A1 (en) * 2014-11-12 2017-07-27 Sony Corporation Image processing apparatus, image processing method, and program
US20160296106A1 (en) 2015-01-21 2016-10-13 Hoya Corporation Endoscope system
US20170251912A1 (en) * 2015-01-21 2017-09-07 Olympus Corporation Endoscope device

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
International Search Report (ISR) and Written Opinion from International Searching Authority (Japan Patent Office) in International Pat. Appl. No. PCT/IB2017/054322, dated Sep. 26, 2017, together with an English language copy of the ISR.
Office Action from Japan Patent Office (JPO) in Japanese Patent Appl. No. 2016-110888, dated Aug. 21, 2018.
Office Action issued in Germany Counterpart Patent Appl. No. 112017000023.8, dated Aug. 26, 2020.
U.S. Appl. No. 15/556,420 to Takao Makino, filed Sep. 7, 2017.

Also Published As

Publication number Publication date
CN109526203B (en) 2021-06-08
CN109526203A (en) 2019-03-26
DE112017000023B4 (en) 2021-03-25
JP6626783B2 (en) 2019-12-25
JP2017213307A (en) 2017-12-07
DE112017000023T5 (en) 2018-02-15
WO2017208216A1 (en) 2017-12-07
US20180184889A1 (en) 2018-07-05

Similar Documents

Publication Publication Date Title
JP7135082B2 (en) Endoscope device, method of operating endoscope device, and program
JP5968944B2 (en) Endoscope system, processor device, light source device, operation method of endoscope system, operation method of processor device, operation method of light source device
US9345392B2 (en) Endoscope system and method for imaging exposure control
JP5623469B2 (en) ENDOSCOPE SYSTEM, ENDOSCOPE SYSTEM PROCESSOR DEVICE, AND ENDOSCOPE CONTROL PROGRAM
JP5690790B2 (en) Endoscope system and method for operating endoscope system
EP2926713A1 (en) Observation device
JP2016052391A (en) Imaging system
WO2013035450A1 (en) Fluorescence observation device
JP6467562B2 (en) Endoscope system
JP6473222B2 (en) Image processing apparatus, living body observation apparatus, and control method for image processing apparatus
CN106659360A (en) Diagnostic support device and diagnostic support information display method
JPWO2018142658A1 (en) Endoscope system
JP6615369B2 (en) Endoscope system
CN110769738A (en) Image processing apparatus, endoscope apparatus, working method of image processing apparatus, and image processing program
WO2018235179A1 (en) Image processing apparatus, endoscope apparatus, operation method of image processing apparatus, and image processing program
JP2021035549A (en) Endoscope system
WO2015025672A1 (en) Endoscope system, processor device, operation method, and table creation method
CN108289599B (en) Endoscope system and imaging method
US10856722B2 (en) Image processing apparatus and electronic endoscope system
CN109526202B (en) Image processing device and electronic endoscope system
WO2015025620A1 (en) Endoscope system, processor device, and operation method
CN106455948B (en) camera system
JP5948191B2 (en) Endoscope probe device and endoscope system
JP2014094175A (en) Image processing system for electronic endoscope
CN112739250B (en) Medical image processing device, processor device and medical image processing method

Legal Events

Date Code Title Description
AS Assignment

Owner name: HOYA CORPORATION, JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MAKINO, TAKAO;REEL/FRAME:043522/0943

Effective date: 20170830

FEPP Fee payment procedure

Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: FINAL REJECTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE AFTER FINAL ACTION FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: AWAITING TC RESP., ISSUE FEE NOT PAID

STPP Information on status: patent application and granting procedure in general

Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED

STCF Information on status: patent grant

Free format text: PATENTED CASE

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 4