WO2020235502A1 - 画像処理装置 - Google Patents

画像処理装置 Download PDF

Info

Publication number
WO2020235502A1
WO2020235502A1 PCT/JP2020/019511 JP2020019511W WO2020235502A1 WO 2020235502 A1 WO2020235502 A1 WO 2020235502A1 JP 2020019511 W JP2020019511 W JP 2020019511W WO 2020235502 A1 WO2020235502 A1 WO 2020235502A1
Authority
WO
WIPO (PCT)
Prior art keywords
texture
image
biological information
subject
display
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2020/019511
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
典雅 繁田
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fujifilm Corp
Original Assignee
Fujifilm Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fujifilm Corp filed Critical Fujifilm Corp
Priority to JP2021520774A priority Critical patent/JP7284814B2/ja
Publication of WO2020235502A1 publication Critical patent/WO2020235502A1/ja
Priority to US17/455,117 priority patent/US12020350B2/en
Anticipated expiration legal-status Critical
Priority to US18/669,035 priority patent/US12430820B2/en
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—Two-dimensional [2D] image generation
    • G06T11/10—Texturing; Colouring; Generation of textures or colours
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
    • A61B1/00009—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope
    • A61B1/000094—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope extracting biological structures
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
    • A61B1/045—Control thereof
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/40—Analysis of texture
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00043—Operational features of endoscopes provided with output arrangements
    • A61B1/00045—Display arrangement
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10068—Endoscopic image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10081—Computed x-ray tomography [CT]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10088—Magnetic resonance imaging [MRI]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10116—X-ray image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10132—Ultrasound image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30096—Tumor; Lesion
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30101—Blood vessel; Artery; Vein; Vascular
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2210/00—Indexing scheme for image generation or computer graphics
    • G06T2210/41—Medical

Definitions

  • the present invention relates to an image processing device that performs image processing using a medical image such as an endoscopic image.
  • the endoscope system is, for example, an endoscope (so-called scope) inserted into a subject, a light source device that generates illumination light for illuminating the subject, and a processor device that acquires an endoscope image of the subject using the endoscope. , And a monitor or the like for displaying an endoscopic image or the like.
  • an endoscopic system has been known that not only photographs a subject naturally but also calculates and displays biological information of the subject using an endoscopic image.
  • an endoscope system in which a pseudo-color image showing a hemoglobin index value is superimposed on a so-called region of interest (Patent Document 1).
  • endoscopic systems that display a part of a region such as a region where the oxygen saturation is below a certain value in pseudo color (Patent Documents 2 and 3).
  • an endoscope system in which the saturation is set low in a region where a large displacement is large when the oxygen saturation is displayed in pseudo color (Patent Document 4).
  • An endoscope system that changes the display mode according to the reliability indicating the certainty of setting the region of interest is also known (Patent Document 5).
  • An object of the present invention is to provide an image processing device that shows biological information in a manner that clearly shows that the biological information is shown when the biological information is shown on an image.
  • the image processing device of the present invention is an image processing device including a processor, and the processor acquires an endoscopic image of a subject and generates an endoscopic image or an endoscopic image for display. Using an image, biological information related to a subject is calculated, a plurality of textures representing the biological information are superimposed on an endoscopic image or a display image, and boundaries of adjacent textures are shown.
  • the processor preferably provides a gap between adjacent textures.
  • the texture preferably has a size of 2 pixels or more.
  • the processor superimposes a texture having a color corresponding to biological information on the endoscopic image or the display image.
  • the processor superimposes a texture having a size corresponding to the biological information on the endoscopic image or the display image.
  • the processor preferably superimposes a texture having a shape corresponding to biological information on an endoscopic image or a display image.
  • the processor superimposes the texture on the part that satisfies the specific condition and does not superimpose the texture on the part that does not satisfy the specific condition.
  • the processor superimposes the texture on the portion where the difference or ratio between the reference value defined for the biometric information and the calculated biometric information value satisfies a specific condition.
  • the processor preferably superimposes the texture on the portion where the rate of change of biological information is equal to or higher than the reference value.
  • the processor sets the position where the texture can be superimposed at a fixed position with respect to the endoscopic image or the display image.
  • the processor preferably sets an upper limit value for the area on which the texture can be superimposed, and superimposes the texture within a range in which the area of the area on which the texture is superimposed is equal to or less than the upper limit value.
  • the processor When the movement of the subject of the endoscope image or the display image stops, or when the movement of the endoscope that captures the endoscope image or the display image stops, the processor superimposes the texture. It is preferable to stop.
  • the biological information when the biological information is shown on the image, the biological information can be shown in a manner that clearly shows that the biological information is shown.
  • the endoscope system 10 includes an endoscope 12 (scope), a light source device 14, a processor device 16, a monitor 18, and a console 19.
  • the endoscope 12 photographs a subject.
  • the light source device 14 generates illumination light.
  • the processor device 16 controls the system of the endoscope system 10. Further, the processor device 16 generates an endoscopic image and performs image processing on the endoscopic image as needed. That is, the processor device 16 functions as an image processing device.
  • the monitor 18 is a display unit that displays an endoscopic image or the like.
  • the console 19 is an input device for inputting settings to the processor device 16 and the like.
  • the endoscope 12 includes an insertion portion 12a to be inserted into the subject, an operation portion 12b provided at the base end portion of the insertion portion 12a, a curved portion 12c provided on the tip end side of the insertion portion 12a, and a tip portion 12d. ,have.
  • the angle knob 12e of the operation unit 12b By operating the angle knob 12e of the operation unit 12b, the curved portion 12c is curved. As a result, the tip portion 12d faces in a desired direction.
  • the operation unit 12b is provided with an angle knob 12e, a treatment tool insertion port 12f, and a zoom operation unit 13.
  • the treatment tool insertion port 12f is an entrance for inserting a treatment tool such as a biopsy forceps, a snare, or an electric knife.
  • the treatment tool inserted into the treatment tool insertion port 12f protrudes from the tip portion 12d.
  • the zoom operation unit 13 By operating the zoom operation unit 13, the subject can be enlarged or reduced for shooting.
  • the light source device 14 includes a light source unit 20 that emits illumination light and a light source control unit 22 that controls the operation of the light source unit 20.
  • the light source unit 20 emits illumination light that illuminates the subject.
  • the emission of the illumination light includes the emission of excitation light or the like used to emit the illumination light.
  • the light source unit 20 includes, for example, a light source of a laser diode, an LED (Light Emitting Diode), a xenon lamp, or a halogen lamp, and is excited to be used to emit at least white illumination light or white illumination light. It emits light.
  • White includes so-called pseudo-white, which is substantially equivalent to white in photographing a subject using the endoscope 12.
  • the light source unit 20 includes, if necessary, a phosphor that emits light when irradiated with excitation light, an optical filter that adjusts the wavelength band, spectral spectrum, light amount, etc. of the illumination light or excitation light.
  • the light source unit 20 can emit light having a specific wavelength band necessary for capturing an image used for calculating biological information such as oxygen saturation of hemoglobin contained in the subject.
  • the light source unit 20 has four-color LEDs of V-LED20a, B-LED20b, G-LED20c, and R-LED20d.
  • the V-LED 20a emits purple light VL having a center wavelength of 405 nm and a wavelength band of 380 to 420 nm.
  • the B-LED 20b emits blue light BL having a center wavelength of 460 nm and a wavelength band of 420 to 500 nm.
  • the G-LED 20c emits green light GL having a wavelength band of 480 to 600 nm.
  • the R-LED20d emits red light RL having a center wavelength of 620 to 630 nm and a wavelength band of 600 to 650 nm.
  • the center wavelengths of the V-LED 20a and the B-LED 20b have a width of about ⁇ 20 nm, preferably about ⁇ 5 nm to about ⁇ 10 nm.
  • the light source control unit 22 controls the timing of turning on, off, or shielding each light source constituting the light source unit 20, the amount of light emitted, and the like.
  • the light source unit 20 can emit a plurality of types of illumination light having different spectral spectra.
  • the light source control unit 22 disperses the illumination light by inputting independent control signals for turning on / off the LEDs 20a to 20d, the amount of light emitted at the time of lighting, the insertion / removal of the optical filter, and the like. Adjust the spectrum.
  • the light source unit 20 emits white light.
  • the light source unit 20 can emit illumination light composed of at least narrow band light (hereinafter referred to as narrow band light).
  • the “narrow band” means a substantially single wavelength band in relation to the characteristics of the subject and / or the spectral characteristics of the color filter of the image sensor 48.
  • the wavelength band is, for example, about ⁇ 20 nm or less (preferably about ⁇ 10 nm or less)
  • this light is a narrow band.
  • the wide band means having a relatively wide wavelength band as compared with a narrow band light in relation to the characteristics of the subject and / or the spectral characteristics of the color filter of the image sensor 48. Therefore, when the wavelength band is, for example, ⁇ 20 nm or more, the light is a wide band light.
  • the tip portion 12d of the endoscope 12 is provided with an illumination optical system 30a and a photographing optical system 30b.
  • the illumination optical system 30a has an illumination lens 45, and the illumination light is emitted toward the subject through the illumination lens 45.
  • the photographing optical system 30b includes an objective lens 46, a zoom lens 47, and an image sensor 48.
  • the image sensor 48 uses the objective lens 46 and the zoom lens 47 to administer the reflected light of the illumination light returning from the subject (in addition to the reflected light, scattered light, fluorescence emitted by the subject, or to the subject).
  • the subject is photographed using (including fluorescence caused by the same chemicals).
  • the zoom lens 47 moves by operating the zoom operation unit 13 to enlarge or reduce the subject image.
  • the image sensor 48 has a color filter of one of a plurality of color filters for each pixel.
  • the image sensor 48 is a color sensor having a primary color system color filter.
  • the image sensor 48 includes an R pixel having a red color filter (R filter), a G pixel having a green color filter (G filter), and a B pixel having a blue color filter (B filter).
  • R filter red color filter
  • G filter green color filter
  • B filter blue color filter
  • CMOS Complementary Metal Oxide Semiconductor
  • the image sensor 48 of the present embodiment is a primary color system color sensor
  • a complementary color system color sensor can also be used.
  • Complementary color sensors include, for example, a cyan pixel provided with a cyan color filter, a magenta pixel provided with a magenta color filter, a yellow pixel provided with a yellow color filter, and a green pixel provided with a green color filter.
  • the complementary color sensor is used, the image obtained from the pixels of each of the above colors can be converted into an image similar to the image obtained by the primary color sensor by performing complementary color-primary color conversion.
  • the primary color system or complementary color system sensor has one or a plurality of types of pixels having characteristics other than the above, such as W pixels (white pixels that receive light in almost all wavelength bands).
  • W pixels white pixels that receive light in almost all wavelength bands.
  • the image sensor 48 of the present embodiment is a color sensor, a monochrome sensor having no color filter may be used.
  • the processor device 16 includes a control unit 52 (processor).
  • the control unit 52 is a hardware resource for executing a program instruction stored in a memory (not shown), and displays the processor device 16 as an image acquisition unit 54 and an image processing unit 61 in accordance with the execution of the program instruction. It functions as a control unit 66.
  • the control unit 52 also causes the processor device 16 to function as an image generation unit 71, a biological information calculation unit 72, and a texture processing unit 73, which will be described later, in accordance with the execution of the program command.
  • the control unit 52 comprehensively controls the endoscope system 10 such as synchronous control of the irradiation timing of the illumination light and the shooting timing.
  • the control unit 52 sets the settings in each part of the endoscope system 10 such as the light source control unit 22, the image sensor 48, or the image processing unit 61. Enter in.
  • the image acquisition unit 54 acquires an endoscopic image of the subject. More specifically, the image acquisition unit 54 acquires an image of a subject photographed using pixels of each color, that is, a RAW image, from the image sensor 48.
  • the RAW image is an image (endoscopic image) before the demosaic process is performed. If it is an image before the demosaic processing is performed, the RAW image also includes an image obtained by performing arbitrary processing such as noise reduction processing on the image acquired from the image sensor 48.
  • the image acquisition unit 54 includes a DSP (Digital Signal Processor) 56, a noise reduction unit 58, and a conversion unit 59 in order to perform various processing on the acquired RAW image as necessary to generate an endoscopic image. To be equipped.
  • DSP Digital Signal Processor
  • the DSP 56 includes, for example, an offset processing unit, a defect correction processing unit, a demosaic processing unit, a linear matrix processing unit, a YC conversion processing unit, and the like (none of which are shown).
  • the DSP 56 performs various processing on a RAW image or an image generated by using the RAW image using these.
  • the offset processing unit performs offset processing on the RAW image.
  • the offset process is a process of reducing the dark current component from the RAW image and setting an accurate zero level.
  • the offset process may be referred to as a clamp process.
  • the defect correction processing unit performs defect correction processing on the RAW image.
  • the defect correction process is a process of correcting or generating a pixel value of a RAW pixel corresponding to a defective pixel of the image sensor 48 when the image sensor 48 includes a pixel (defective pixel) having a defect due to a manufacturing process or a change with time. Is.
  • the demosaic processing unit performs demosaic processing on the RAW image of each color corresponding to the color filter of each color.
  • the demosaic process is a process of generating pixel values that are missing due to the arrangement of color filters in a RAW image by interpolation.
  • the linear matrix processing unit performs linear matrix processing on the endoscopic image generated by assigning one or a plurality of RAW images to channels of each RGB color.
  • the linear matrix processing is a processing for enhancing the color reproducibility of an endoscopic image.
  • an endoscope image generated by assigning one or a plurality of RAW images to each RGB color channel is an endoscope having a brightness channel Y, a color difference channel Cb, and a color difference channel Cr. This is the process of converting to an image.
  • the noise reduction unit 58 performs noise reduction processing on an endoscopic image having a luminance channel Y, a color difference channel Cb, and a color difference channel Cr by using, for example, a moving average method or a median filter method.
  • the conversion unit 59 reconverts the luminance channel Y, the color difference channel Cb, and the color difference channel Cr after the noise reduction processing into an endoscopic image having channels of each color of BGR.
  • the image processing unit 61 performs necessary image processing on the endoscopic image output by the image acquisition unit 54. Further, the image processing unit 61 performs an calculation using an endoscopic image output by the image acquisition unit 54 or another endoscopic image generated by using the endoscopic image output by the image acquisition unit 54. To do. Specifically, as shown in FIG. 3, the image processing unit 61 includes an image generation unit 71, a biological information calculation unit 72, a texture processing unit 73, and the like.
  • the image generation unit 71 acquires an endoscope image from the image acquisition unit 54 and generates an endoscope image (hereinafter referred to as a display image) to be used for display on a monitor 18 or the like.
  • the image generation unit 71 may use the image acquisition unit 54 to capture a B image of the subject using the B pixel, a G image of the subject photographed using the G pixel, and an R image of the subject using the R pixel. Images and images are acquired, and all or part of them are used to generate an image for display.
  • the image acquisition unit 54 This includes performing and outputting the necessary image processing on one endoscopic image acquired from the image acquisition unit 54, and outputting the one endoscopic image acquired from the image acquisition unit 54 as it is.
  • the image generation unit 71 generates a display image by using the endoscopic image acquired from the image acquisition unit 54 or the endoscopic image acquired from the image acquisition unit 54 when generating the display image. Is subjected to necessary image processing.
  • the image processing performed by the image generation unit 71 is, for example, an enhancement process for emphasizing the subject or a part of the subject. Emphasis means making it possible to obtain information on a specific part by distinguishing it from other organizations or structures. For example, a process of surrounding a part having a specific feature with a frame to show an outline, or changing the color or brightness relative to another part (for example, normal mucous membrane) is an emphasis process. ..
  • the biological information calculation unit 72 is an endoscopic image acquired from the image acquisition unit 54 and / or an endoscopic image generated by the image generation unit 71 using the endoscopic image acquired from the image acquisition unit 54.
  • the biological information related to the subject is calculated using the display image.
  • the biological information is a numerical value or the like representing the whole or partial characteristics of the subject, for example, oxygen saturation, blood concentration, blood vessel density, or a lesion or a candidate for a lesion (including a target for biological tissue examination) and the like. The certainty of having a specific form of.
  • the biological information calculation unit 72 calculates the oxygen saturation of the subject for each pixel using the endoscopic image acquired from the image acquisition unit 54.
  • oxygen saturation for example, a B image in which the subject is photographed using narrow-band light having a wavelength of about 470 nm, a G image in which the subject is photographed using wideband green light, and a wideband red light are used. It can be calculated using an R image obtained by photographing the subject. More specifically, the ratio of the B image to the G image (hereinafter referred to as B / G) and the ratio of the R image to the G image (hereinafter referred to as R / G) are obtained for each pixel.
  • B / G the ratio of the B image to the G image
  • R / G the ratio of the R image to the G image
  • a table or the like for associating B / G and R / G with the oxygen saturation value by an experiment or a simulation is prepared in advance. Then, the oxygen saturation is calculated from the calculated B / G and R / G values using the above table.
  • the biological information calculation unit 72 calculates the oxygen saturation
  • the image acquisition unit 54 acquires each of these images and provides them to the biological information calculation unit 72.
  • the texture processing unit 73 superimposes a plurality of textures representing biological information on the endoscopic image obtained from the image acquisition unit 54 or the display image generated by the image generation unit 71 (hereinafter referred to as a display image or the like).
  • the texture processing unit 73 superimposes the texture on the endoscopic image, for example, when the endoscopic image is used for display or the like. That is, the object on which the texture processing unit 73 superimposes the texture is an image displayed on the monitor 18 or the like.
  • the display image or the like on which the texture processing unit 73 superimposes the texture is referred to as a biological information image.
  • the biological information image is an image (endoscopic image) that displays biological information by texture.
  • Representing biological information is a mode of display directly based on the value of biological information calculated by the biological information calculation unit 72, or indirectly based on a numerical value calculated using biological information or the like.
  • a texture is a figure or the like (including a case where it has the form of a character or a symbol) superimposed on a part of a display image or the like and has a specific pattern and / or shape (outline).
  • the texture can be composed of lines (so-called hollow figures) representing the shapes of figures and the like. Further, since the texture has a gradation pattern whose color differs depending on the position, the shape may not be clearly recognized in the display image or the like.
  • the texture used by the texture processing unit 73 is a quadrangular figure, and the tile texture 85 (see FIG. 4 and the like) in which the inside is colored.
  • “Superimposing" a texture on a display image or the like means that at least when the display image or the like is displayed on a monitor 18 or the like, a subject whose texture features such as pattern, shape, and / or color are reflected in the display image or the like. It means to display the texture on the display image, etc. in a manner that can be distinguished and recognized. In addition to displaying the texture on the display image, etc. in a separable manner, the texture is combined (integrated) with the display image, etc. ) Including. In the present embodiment, the texture processing unit 73 superimposes the texture on the display image or the like in a separable manner.
  • the texture processing unit 73 has a plurality of types of textures having different shapes, colors, patterns, or sizes, and one type of texture can be selected and used based on the settings and the like. Further, the texture processing unit 73 can use a plurality of types of textures in combination based on the settings and the like.
  • the texture processing unit 73 determines the size, color (hue, lightness, saturation, etc.), brightness (luminance), and / or transparency of some or all elements of one texture. It can be changed and superimposed on a display image or the like. However, the texture processing unit 73 sets the size of each texture to at least two pixels or more of a display image or the like on which the texture is superimposed. This is to clarify the difference from the pixels of the display image or the like so that the texture can be recognized as the texture. Further, the texture processing unit 73 can superimpose and display at least two or more textures in the subject display area 83 (see FIG. 4) while making the size of one texture smaller than the display image or the like on which the textures are superimposed. Make it the size.
  • the texture processing unit 73 changes the size, color, brightness, and / or transparency of each texture for display. It can be superimposed on an image or the like.
  • the texture processing unit 73 when a plurality of textures are superposed on the display image or the like, the texture is superimposed on the display image or the like at adjacent positions (hereinafter, adjacent). It is an aspect showing the boundary of the texture).
  • "Indicating a boundary” means displaying a pattern or shape of each texture in a display image or the like in a form that can be distinguished and perceived. For example, a mode in which a gap is provided between adjacent textures is a mode in which a boundary is provided.
  • the texture processing unit 73 provides a gap between adjacent textures (see FIG. 4 and the like).
  • the texture processing unit 73 determines a position where the texture can be superimposed at a fixed position with respect to the display image or the like. That is, the texture processing unit 73 predetermines the position on which the texture is superimposed according to the display size of the display image or the like, regardless of the shape of the subject or the like. Further, the texture processing unit 73 superimposes the texture on the portion that satisfies the specific condition, and does not superimpose the texture on the portion that does not satisfy the specific condition.
  • the texture processing unit 73 determines a plurality of grid points that specify the position where the texture is superimposed on the display image or the like by setting or the like. Then, based on the biological information to be displayed, the texture is superimposed on a part or all of the required positions of the lattice points on which the texture can be superimposed. Further, the grid points defined by the texture processing unit 73 are determined independently of the shape of the subject. Then, the texture processing unit 73 does not change the position and distribution (density, etc.) of the grid points in the display image or the like depending on the change of the subject reflected in the display image or the like. That is, the position where the texture can be superimposed does not follow the change in the shape of the subject and is constant. However, whether or not the texture processing unit 73 superimposes the texture on each grid point depends on the change of the subject. This is because the value of the biological information calculated by the biological information calculation unit 72 changes according to the change of the subject.
  • the texture processing unit 73 sets grid points 82 in a square grid pattern with respect to the display image 81. Further, among these grid points 82, the grid point 82 in the subject display area 83 for displaying the subject is set as the texture superimposition position.
  • the grid points 82 inside the subject display area 83 are indicated by filled circles ( ⁇ ), and the grid points 82 outside the subject display area 83 are indicated by hollow circles ( ⁇ ). ing.
  • the arrangement (grid type) and density (or spacing) of the grid points 82 can be arbitrarily changed by setting or the like.
  • the texture processing unit 73 may automatically change the size and / or shape of the texture according to the arrangement and density of the grid points 82. This is to properly show the boundaries between textures.
  • the texture processing unit 73 aligns the center (center position) with the grid points 82 and superimposes the tile texture 85 on the display image 81. Further, the texture processing unit 73 provides a gap of ⁇ X in the X direction and a gap of ⁇ Y in the Y direction by adjusting the size of the tile texture 85.
  • the gap in the X direction ( ⁇ X) and the gap in the Y direction ( ⁇ Y) may be equal or different.
  • the gap in the X direction ( ⁇ X) and the gap in the Y direction ( ⁇ Y) are equal. This is to reduce the directionality of the gap between the tile textures 85 and prevent an unintended shape or pattern from appearing on the display image 81 due to the directionality of the gap between the tile textures 85.
  • the texture processing unit 73 acquires biological information from the biological information calculation unit 72 and superimposes the texture on each grid point 82 by using the value of the acquired biological information or the numerical value calculated using the biological information. Whether or not it is determined for each grid point 82.
  • the biological information calculation unit 72 calculates the oxygen saturation as biological information. Therefore, the texture processing unit 73 superimposes the texture on the portion of the region where the oxygen saturation is equal to or lower than the reference value (hereinafter referred to as the hypoxic region).
  • the reference value is predetermined for each biological information according to the type of biological information. When the biological information is oxygen saturation, the reference value is, for example, 70%.
  • the texture processing unit 73 uses the oxygen saturation value calculated by the biological information calculation unit 72 to form a predetermined region including each grid point 82 (for example, a so-called Wigner-Seitz cell of each grid point 82).
  • the average value of oxygen saturation is calculated for the pixels belonging to, and the calculated average value is compared with a predetermined reference value. Then, when the average value of the oxygen saturation is equal to or less than the reference value, it is determined that the lattice point 82 belongs to the low oxygen region, and the texture is superimposed on the lattice point 82.
  • the average value of oxygen saturation is larger than the reference value, it is determined that the grid point 82 belongs to the high oxygen region (the region where the oxygen saturation value is normal), and the grid point 82 is textured. Decide not to superimpose.
  • the texture processing unit 73 calculates the average value of oxygen saturation, but instead of the average value, other statistics such as total value, median value, minimum value, variance, or standard deviation.
  • the amount can be calculated and used for determining whether or not to superimpose the texture (hereinafter referred to as superimposition determination).
  • a plurality of statistics can be calculated and combined to be used for superimposition determination.
  • the texture processing unit 73 obtains, for example, the rate of change in oxygen saturation (for example, the gradient (differential value) of the distribution of oxygen saturation), and is a statistic about the rate of change in oxygen saturation or the rate of change in oxygen saturation. Can be used for superposition determination.
  • the reference value is set in advance to a value that can be compared with the statistic used for determining whether or not to superimpose the texture. Further, when a plurality of statistics and / or calculated values such as a differential value are used for the superposition determination, a plurality of reference values may be set according to each statistic and the like.
  • the display control unit 66 converts the display image or the like output by the image processing unit 61 into a format suitable for display and outputs it to the monitor 18. As a result, the monitor 18 displays the display image.
  • the texture processing unit 73 superimposes a texture on a display image or the like
  • the display control unit 66 acquires a biological information image from the texture processing unit 73 and outputs the biometric information image to the monitor 18.
  • the image generation unit 71 acquires an endoscope image from the image acquisition unit 54 and uses the endoscope image.
  • a display image 81 is generated (step S102).
  • the endoscopic image acquired by the image generation unit 71 includes an endoscopic image used by the biological information calculation unit 72 for calculating biological information, in addition to the one used for generating the display image 81.
  • the biological information calculation unit 72 acquires an endoscopic image from the image acquisition unit 54, and uses this to calculate the oxygen saturation, which is biological information, for each pixel (step S103).
  • the texture processing unit 73 makes a superposition determination for each grid point 82 using the oxygen saturation calculated by the biological information calculation unit 72.
  • the texture processing unit 73 superimposes the tile texture 85, which is one aspect of the texture, on the display image 81, for example, on the lattice points 82 belonging to the low oxygen region (step).
  • the biological information image 91 is generated.
  • the monitor 18 displays the biological information image 91 (step S105).
  • the generation and display of the biological information image 91 uses the endoscope 12 to capture the subject while the subject is being photographed by the endoscope 12 unless the observation mode in which the biological information is calculated and displayed is switched to. Repeat each time you take a picture or intermittently.
  • the endoscope system 10 calculates the biological information in the processor device 16 that functions as an image processing device, and displays the biological information using the biological information image 91 on which the texture is superimposed.
  • the mode in which the textures are superimposed is a mode in which the boundaries of adjacent textures are indicated. Therefore, the endoscope system 10 can show the biological information in a manner that clearly shows that the biological information is shown. For example, when partially or all the colors of a display image or the like are modulated to colors according to the value of biological information, the original color of the living body whose modulated portion is the subject depends on the color after modulation and the color of the subject. It may be difficult to tell if it is a color.
  • the endoscopic system 10 or the processor device 16 uses a texture and shows the boundary between adjacent textures to give the impression that the texture is clearly an artificial element. Therefore, according to the biological information image 91, it is possible to correctly recognize that the texture is an additional element superimposed by image processing (texture superimposition processing, etc.), and as a result, it is clear that the biological information is shown. Can be known to.
  • the biological information image 91 can display the biological information in a manner that clearly indicates that the biological information is shown.
  • the position of the grid point 82 on which the texture is superimposed does not depend on the shape of the subject and its change, and is fixed on the display image or the like. Therefore, the overlapping position of the texture moves according to the change of the subject, but the movement mode can give a discrete and artificial impression.
  • the biological information can be displayed in a manner that clearly indicates that the biological information is shown. Specifically, when the subject of the biological information image 91 (see FIG. 6) changes as in the biological information image 92 shown in FIG. 7, the superimposed position of the tile texture 85 moves following the change of the subject as a whole. To do.
  • the texture processing unit 73 superimposes the texture on the portion that satisfies the specific condition, and does not superimpose the texture on the portion that does not satisfy the specific condition. Therefore, since the superimposition of the texture is partial, it is possible to accurately display the notable part as compared with the case of superimposing the texture on the entire display image or the like.
  • the textured display of the above embodiment is particularly effective for identifying the infiltration site, presenting a candidate to be examined for biological tissue, and / or identifying the region of interest (region of interest).
  • the texture processing unit 73 superimposes the same tile texture 85 on a plurality of grid points 82 to generate a biological information image 91 or the like (see FIG. 6 or the like), but the texture processing unit 73 , When the tile texture 85 is superimposed on each grid point 82, its characteristics can be changed. Therefore, it is preferable that the texture processing unit 73 adjusts the characteristics of the texture according to the biological information. When adjusting the characteristics of the texture according to the biological information, not only the position and / or range of interest but also the content of the biological information (value of the biological information, etc.) can be notified by the display form of the texture by superimposing the texture. Is.
  • the texture processing unit 73 superimposes a texture having a color corresponding to biological information on a display image or the like.
  • "Having a color according to biological information” means that the degree of biological information, that is, the value of biological information calculated by the biological information calculation unit 72, the numerical value calculated using the biological information, etc. is displayed by color.
  • the texture processing unit 73 is either the first color texture 101 or the second color texture 102 based on the average value of the oxygen saturation in a predetermined region including the superposed grid points 82. Can be superimposed on the grid point 82 to generate a biological information image 93.
  • the original texture of the first color texture 101 and the second color texture 102 is the tile texture 85.
  • the first color texture 101 and the second color texture 102 have the same features as the tile texture 85 except for the color. Further, the color of the first color texture 101 is, for example, light blue, and the color of the second color texture 102 is, for example, dark blue.
  • the texture processing unit 73 when displaying the oxygen saturation value or the like by two kinds of colors, the texture processing unit 73 has, in addition to the reference value for determining whether or not to superimpose the texture for each grid point 82, A threshold is set to determine the color. Then, the color of the texture to be superposed is determined by further comparing the average value of oxygen saturation and the like with the threshold value of the lattice points 82 determined to superimpose the texture. For example, the second color texture 102 is superimposed on the lattice points 82 in which the average value of oxygen saturation is equal to or less than the threshold value, and the first color texture 101 is superimposed on the lattice points 82 in which the average value of oxygen saturation is larger than the threshold value. To superimpose. As a result, the texture processing unit 73 generates the biological information image 93 from the display image 81.
  • the texture processing unit 73 can display the oxygen saturation value and the like in three or more colors by using a texture of a color different from these in addition to the first color texture 101 and the second color texture 102. it can. In this case, a plurality of threshold values for determining the color may be prepared.
  • the texture processing unit 73 superimposes a texture having a size corresponding to biological information on a display image or the like. This is because not only the position and / or range of interest but also the content of biological information (value of biological information, etc.) can be notified by the display form of the texture.
  • the texture processing unit 73 is either the first size texture 111 or the second size texture 112 based on the average value of oxygen saturation in a predetermined region including the superposed grid points 82. Can be superimposed on the grid point 82 to generate a biological information image 110.
  • the original texture of the first size texture 111 and the second size texture 112 is the tile texture 85. Therefore, the first size texture 111 and the second size texture 112 have the same features as the tile texture 85 except for the size. Further, the first size texture 111 is relatively smaller than the second size texture 112.
  • the process for displaying the oxygen saturation value and the like according to the two sizes is the same as the case where the color of the texture is changed to the color according to the biological information. That is, the texture processing unit 73 sets a threshold value for determining the size of the texture in addition to the reference value for determining whether or not to superimpose the texture for each grid point 82. Then, the size of the texture to be superimposed is determined by further comparing the average value of oxygen saturation and the like with the threshold value of the lattice points 82 determined to be superimposed.
  • the second size texture 112 is superimposed on the grid points 82 in which the average value of oxygen saturation is equal to or less than the threshold value
  • the first size texture 111 is superposed on the grid points 82 in which the average value of oxygen saturation is larger than the threshold value.
  • the texture processing unit 73 generates the biological information image 110 from the display image 81.
  • the texture processing unit 73 uses textures of different sizes in addition to the first size texture 111 and the second size texture 112 to obtain oxygen saturation values and the like with a texture size of 3 or more. Can be displayed. In this case, a plurality of threshold values for determining the size of the texture may be prepared. Further, when superimposing the maximum size texture on the adjacent lattice points 82, it is preferable that the size of these textures excluding the contour and not overlapping is set as the maximum size of the texture. This is to make it easier to show the boundaries of the texture and to make it clearer that the texture shows biometric information.
  • the texture processing unit 73 superimposes a texture having a shape corresponding to biological information on a display image or the like. This is because not only the position and / or range of interest but also the content of biological information (value of biological information, etc.) can be notified by the display form of the texture.
  • the texture processing unit 73 is either the first shape texture 121 or the second shape texture 122 based on the average value of oxygen saturation in a predetermined region including the superposed grid points 82. Can be superimposed on the grid point 82 to generate a biological information image 120.
  • the first shape texture 121 is, for example, a texture having a triangular shape, and other features such as color are the same as those of the tile texture 85.
  • the second shape texture 122 is, for example, a texture having a star shape, and other features such as color are the same as those of the tile texture 85.
  • the process of displaying the oxygen saturation value and the like by the shapes of the two types of textures is the same as the case of changing the color of the texture according to the biological information. That is, the texture processing unit 73 sets a threshold value for determining the size of the texture in addition to the reference value for determining whether or not to superimpose the texture for each grid point 82. Then, the shape of the texture to be superposed is determined by further comparing the average value of oxygen saturation and the like with the threshold value of the lattice points 82 determined to superimpose the texture.
  • the second shape texture 122 is superimposed on the lattice point 82 in which the average value of oxygen saturation is equal to or less than the threshold value
  • the first shape texture 121 is superimposed on the lattice point 82 in which the average value of oxygen saturation is larger than the threshold value.
  • the texture processing unit 73 generates the biological information image 120 from the display image 81.
  • the oxygen saturation value is displayed according to the shapes of the two types of textures using the first shape texture 121 and the second shape texture 122, but three or more types of textures having different shapes are prepared in advance. Can be done. Therefore, the texture processing unit 73 uses a texture having a different shape (for example, tile texture 85) in addition to the first shape texture 121 and the second shape texture 122, so that the oxygen saturation degree depends on the shape of three or more textures.
  • the value of can be displayed. In this case, a plurality of threshold values for determining the shape of the textures to be superimposed may be prepared.
  • the shape of the texture is determined by the degree of biometric information, but the shape of the texture may be determined by the type of biometric information.
  • the biological information calculation unit 72 calculates the blood concentration in addition to the oxygen saturation
  • the first shape texture 121 is used for displaying the oxygen saturation
  • the second shape texture 122 is used for displaying the blood concentration.
  • one biometric information image can be used to obtain information related to a plurality of biometric information without being confused with each other.
  • the textured display forms of the second embodiment, the third embodiment, and the fourth embodiment can be arbitrarily combined. That is, the texture processing unit 73 can superimpose a texture having a color, size, and / or shape according to biological information on a display image or the like. In this case, the biological information can be displayed in a manner that clearly indicates that the biological information is shown, and the biological information is displayed in a manner that has the advantages of the second embodiment, the third embodiment, and the fourth embodiment. it can.
  • a boundary is shown by providing a gap between adjacent textures, but between adjacent textures. Even when there is no gap in, the boundary can be shown.
  • the texture processing unit 73 when the texture processing unit 73 superimposes a texture on a display image or the like in a manner in which adjacent textures share a boundary with each other without providing a gap between adjacent textures, the texture processing unit 73 depends on the characteristics such as the color or pattern of each texture. , Or, by adding a boundary line in addition to superimposing the adjacent textures, the boundary of the adjacent textures can be shown.
  • the texture processing unit 73 can generate the biological information image 130 using, for example, the tile texture 85.
  • the biological information image 130 is a biological information image in which a plurality of tile textures 85 are superimposed on the display image 81.
  • the texture processing unit 73 changes the size of the tile texture 85 to generate the biological information image 130. Therefore, in the biological information image 130, there is no gap between the adjacent tile textures 85, and the adjacent tile textures 85 share the contours with each other.
  • the contour lines of the adjacent tile texture 85 are integrally formed with the vertical line 131 and the horizontal line 132. Form. As a result, vertical lines 131 and horizontal lines 132 indicate boundaries between adjacent tile textures 85.
  • the texture processing unit 73 generates a biometric information image 141 by using, for example, a texture (hereinafter, referred to as a gradation texture) 140 having a gradation pattern in which the color changes from the central portion to the peripheral portion. be able to.
  • the biological information image 141 is a biological information image in which a plurality of gradation textures 140 are superimposed on the display image 81. Further, in the biological information image 141, there is no gap between the adjacent gradation textures 140, and the adjacent gradation textures 140 share the contour with each other.
  • the peripheral portion of the adjacent gradation texture 140 is integrated in the biological information image 141 to distinguish it from the central portion of each gradation texture 140.
  • the boundary portion 142 and the boundary portion 143 in a possible manner are formed.
  • the boundary portion 142 and the boundary portion 143 substantially indicate the boundary between the adjacent gradation textures 140.
  • the texture processing unit 73 uses the texture to obtain biological information (oxygen saturation). Although the value of is displayed, the texture processing unit 73 can display a numerical value or the like calculated using biometric information depending on the texture. For example, as shown in FIG. 13, the texture processing unit 73 can generate a biological information image 151 that displays the contour of a lesion or a portion having a specific morphology such as a lesion candidate (hereinafter, referred to as a lesion or the like) 150. it can.
  • the biological information image 151 is a biological information image in which the tile texture 85 is superimposed on a portion where the rate of change in oxygen saturation is steep.
  • the texture processing unit 73 acquires the oxygen saturation from the biological information calculation unit 72, and then differentiates the distribution to obtain the change rate of the oxygen saturation (hereinafter, simply changed). The rate) is calculated, and the calculated change rate is determined to be superimposed in the same manner as in the first embodiment. However, as the reference value for the superimposition determination, a predetermined value for the rate of change is used. Further, as a result of the superposition determination, the texture is superposed on the grid points 82 belonging to the region having the rate of change equal to or higher than the reference value, and the texture is not superposed on the grid points 82 belonging to the region having the rate of change lower than the reference value. As a result, the texture processing unit 73 generates the above-mentioned biological information image 151.
  • the contour of the lesion or the like 150 is shown by the texture as in the biological information image 151, the existence and position of the lesion or the like 150 for which it is important to pay attention to the contour can be directly notified.
  • whether or not the contour of the lesion or the like 150 is clear may be a guideline for diagnosis, but in the biological information image 151, if the tile texture 85 is present over the entire circumference of the lesion or the like 150, the lesion or the like 150 has a contour. Can be clearly seen that is clear.
  • the texture processing unit 73 superimposes the texture on the portion where the rate of change of the biological information is equal to or higher than the reference value, but the texture processing unit 73 uses "biological information" other than the rate of change. "Numerical values, etc. calculated by the above” can be used for the superposition determination.
  • the texture processing unit 73 can superimpose the texture on a portion where the difference or ratio between the reference value determined for the biological information and the value of the biological information calculated by the biological information calculation unit 72 satisfies a specific condition.
  • the difference between the reference value and the biometric information value is larger than that when the biometric information value is used as it is for the superposition determination. Be emphasized.
  • the texture may be repeatedly displayed and hidden in the subject portion where the biological information takes a value near the reference value.
  • the reference value and the biological information When the difference or ratio with the value of is used for the superposition determination, the blinking of such a texture can be reduced, and the biological information may be stably displayed by the texture.
  • the grid points 82 which are the positions where the texture processing unit 73 superimposes the texture, form a square grid, but the arrangement of the grid points 82 is arbitrary.
  • the texture processing unit 73 can set the grid points 82 in the shape of a triangular grid with respect to the display image 81 according to the settings and the like.
  • the hexagonal tile texture 165 can be spread over the triangular grid-shaped grid points 82 without any extra gap, except for the gap intentionally provided between the adjacent textures.
  • the tile texture 165 may be adjacent to each other in the three directions of the Y direction, the V1 direction, and the V2 direction depending on the arrangement of the grid points 82. Therefore, when a gap is provided between the adjacent tile textures 165, the size of the gap between the adjacent tile textures 165 can be set in each of the Y direction, the V1 direction, and the V2 direction. However, it is preferable that the gaps between the tile textures 165 adjacent to each of these directions are all equal. This is to prevent an unintended shape or pattern from appearing on the display image 81.
  • the texture processing unit 73 defines in advance a reference value used for determining the superimposition of the texture, but the reference value used for determining the superimposition of the texture uses the subject to be observed. It can be decided fluidly. For example, when oxygen saturation is used as biological information, for example, the oxygen saturation calculated for a portion designated by a doctor or the like who is an operator of the endoscope system 10 can be adopted as a reference value for superimposition determination.
  • the normal value of oxygen saturation varies depending on the subject, but as described above, by setting the reference value fluidly using the subject, it is possible to make a superposition determination suitable for the subject.
  • the texture processing unit 73 may superimpose the texture on all the grid points 82 in the subject display area 83 depending on the result of the superimposition determination.
  • the texture processing unit 73 sets an upper limit value (maximum value of the area occupied by the texture on the display image or the like) on the area where the texture can be superimposed on the display image or the like, and the area on which the texture is superimposed.
  • the texture processing unit 73 superimposes the texture of the grid point 82 whose biological information value or the like is close to the reference value.
  • the superimposition of the texture is stopped in order from the portion having the oxygen saturation close to the normal value. This is because it is easier to contribute to diagnosis and the like by superimposing the texture on the grid points 82 whose biometric information values and the like are far from the reference values than by the grid points 82 whose biometric information values and the like are close to the reference values.
  • the biological information is continuously displayed by the texture, but the endoscope 12 that captures the display image or the like when the movement of the subject such as the display image stops.
  • the texture processing unit 73 preferably stops the texture superimposition processing.
  • the doctor or the like who is the operator of the endoscope system 10 may want to closely examine the part being observed at that time. This is to prevent the display of the texture from interfering with the diagnosis or the like in some cases.
  • stop means that the amount or speed of these movements is equal to or less than a predetermined threshold value, and is substantially stopped in addition to the case where the movement is completely stopped. Includes being in a state close to doing (a state with little movement). Further, unless the function of the texture processing unit 73 is stopped due to a setting or the like, the texture processing unit 73 causes the subject or the endoscope 12 to move again when the movement of the subject or the endoscope 12 is stopped. , Resume the texture superimposition process.
  • the image processing unit 61 is provided with the motion recognition unit 201 as shown in FIG.
  • the motion recognition unit 201 acquires a display image or the like from the image acquisition unit 54 and / or the image generation unit 71. Then, the movement of the subject or the endoscope 12 is recognized by using the display image or the like.
  • any known method such as calculation of a motion vector can be used.
  • the motion recognition unit 201 inputs the recognition result such as the presence / absence, amount, and / or speed of the movement of the subject or the endoscope 12 to the texture processing unit 73. Then, the texture processing unit 73 stops or starts (restarts) the texture superimposition processing on the display image or the like depending on the recognition result of the motion recognition unit 201.
  • the processor device 16 functions as an image processing device, but as shown in FIG. 16, apart from the processor device 16, the image generation unit 71, the biological information calculation unit 72, and / or , An image processing device 251 including a texture processing unit 73 can be provided.
  • the image generation unit 71, the biological information calculation unit 72, and / or the texture processing unit 73 may be, for example, an endoscope system 10 (including a system having no texture processing unit 73 or the like). It can be provided in the diagnostic support device 911 that acquires a RAW image taken by the endoscope 12 directly from the endoscope or indirectly from the PACS (Picture Archiving and Communication Systems) 910. Further, as shown in FIG. 18, various inspection devices such as the first inspection device 921, the second inspection device 922, ..., The K inspection device 923, and the like including the endoscope system 10 are connected via the network 926.
  • the medical service support device 930 may be provided with an image generation unit 71, a biological information calculation unit 72, and / or a texture processing unit 73.
  • the endoscope 12 uses a so-called flexible endoscope having a flexible insertion portion 12a, but is a capsule-type endoscope used by the subject swallowing.
  • the present invention is also suitable when a rigid endoscope (laparoscope) used for surgery or the like is used.
  • the image acquisition unit acquires an endoscopic image of the subject
  • the biological information calculation unit generates the endoscopic image or the endoscopic image.
  • the step of calculating the biological information related to the subject using the display image and the texture processing unit superimpose a plurality of textures representing the biological information on the endoscopic image or the display image, and the boundary between adjacent textures. Including a method of processing an image and a method of operating an endoscopic system.
  • the hardware structure of the processing unit that executes various processes such as the image generation unit 71, the biological information calculation unit 72, and the texture processing unit 73 is as follows.
  • Various processors as shown.
  • Various processors include CPU (Central Processing Unit), GPU (Graphical Processing Unit), FPGA (Field Programmable Gate Array), which are general-purpose processors that execute software (programs) and function as various processing units.
  • PLD Programmable Logic Device
  • One processing unit may be composed of one of these various processors, or a combination of two or more processors of the same type or different types (for example, a plurality of FPGAs, a combination of a CPU and an FPGA, or a CPU and a CPU. It may be composed of a combination of GPUs, etc.). Further, a plurality of processing units may be configured by one processor. As an example of configuring a plurality of processing units with one processor, first, as represented by a computer such as a client or a server, one processor is configured by a combination of one or more CPUs and software. There is a form in which this processor functions as a plurality of processing units.
  • SoC System On Chip
  • the various processing units are configured by using one or more of the above-mentioned various processors as a hardware-like structure.
  • the hardware structure of these various processors is, more specifically, an electric circuit in the form of a combination of circuit elements such as semiconductor elements.
  • the present invention is a system or device for acquiring medical images (including moving images) other than endoscopic images. It can also be used in such cases.
  • the present invention can be applied to an ultrasonic inspection device, an X-ray imaging device (including a CT (Computed Tomography) inspection device, a mammography device, etc.), an MRI (magnetic resonance imaging) device, and the like.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Surgery (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Biophysics (AREA)
  • Optics & Photonics (AREA)
  • Pathology (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Molecular Biology (AREA)
  • Animal Behavior & Ethology (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Quality & Reliability (AREA)
  • Signal Processing (AREA)
  • Endoscopes (AREA)
PCT/JP2020/019511 2019-05-23 2020-05-15 画像処理装置 Ceased WO2020235502A1 (ja)

Priority Applications (3)

Application Number Priority Date Filing Date Title
JP2021520774A JP7284814B2 (ja) 2019-05-23 2020-05-15 画像処理装置
US17/455,117 US12020350B2 (en) 2019-05-23 2021-11-16 Image processing apparatus
US18/669,035 US12430820B2 (en) 2019-05-23 2024-05-20 Image processing apparatus

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2019096512 2019-05-23
JP2019-096512 2019-05-23

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/455,117 Continuation US12020350B2 (en) 2019-05-23 2021-11-16 Image processing apparatus

Publications (1)

Publication Number Publication Date
WO2020235502A1 true WO2020235502A1 (ja) 2020-11-26

Family

ID=73458492

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/019511 Ceased WO2020235502A1 (ja) 2019-05-23 2020-05-15 画像処理装置

Country Status (3)

Country Link
US (2) US12020350B2 (https=)
JP (1) JP7284814B2 (https=)
WO (1) WO2020235502A1 (https=)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023132138A1 (ja) * 2022-01-07 2023-07-13 富士フイルム株式会社 プロセッサ装置及びその作動方法並びに内視鏡システム
CN117500426A (zh) * 2021-06-16 2024-02-02 富士胶片株式会社 医疗图像处理装置、内窥镜系统及医疗图像处理装置的工作方法

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021144951A1 (ja) * 2020-01-17 2021-07-22 オリンパス株式会社 画像処理装置、画像処理方法、及び、画像処理プログラム

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS5695030A (en) * 1979-12-27 1981-08-01 Olympus Optical Co Endoscope monitor apparatus
JP2006116153A (ja) * 2004-10-22 2006-05-11 Olympus Corp 内視鏡用画像処理装置及び内視鏡装置
WO2012132790A1 (ja) * 2011-03-31 2012-10-04 オリンパス株式会社 蛍光観察装置
JP2016154588A (ja) * 2015-02-23 2016-09-01 Hoya株式会社 画像処理装置
WO2016175084A1 (ja) * 2015-04-30 2016-11-03 富士フイルム株式会社 画像処理装置、方法及びプログラム
WO2017057574A1 (ja) * 2015-09-29 2017-04-06 富士フイルム株式会社 画像処理装置、内視鏡システム、及び画像処理方法
JP2018126632A (ja) * 2018-05-25 2018-08-16 Hoya株式会社 画像処理装置

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1938274A2 (en) * 2005-09-12 2008-07-02 D.V.P. Technologies Ltd. Medical image processing
JP5658873B2 (ja) 2009-11-13 2015-01-28 オリンパス株式会社 画像処理装置、電子機器、内視鏡システム及びプログラム
JP5303012B2 (ja) 2010-12-15 2013-10-02 富士フイルム株式会社 内視鏡システム、内視鏡システムのプロセッサ装置及び内視鏡システムの作動方法
JP2013099464A (ja) 2011-11-09 2013-05-23 Fujifilm Corp 内視鏡システム、内視鏡システムのプロセッサ装置、及び画像表示方法
JP6034668B2 (ja) 2012-11-08 2016-11-30 富士フイルム株式会社 内視鏡システム
WO2021176708A1 (ja) * 2020-03-06 2021-09-10 オリンパス株式会社 アンテナシステム、カプセル型内視鏡システム及びアンテナシステムの作動方法
EP4137035B1 (en) * 2020-04-17 2025-09-24 FUJIFILM Corporation Image processing device and operating method therefor
US20220354356A1 (en) * 2021-02-17 2022-11-10 PacificMD Biotech, LLC System and method for endoscopic imaging and analyses
WO2022251814A2 (en) * 2021-05-24 2022-12-01 Stryker Corporation Systems and methods for generating three-dimensional measurements using endoscopic video data

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS5695030A (en) * 1979-12-27 1981-08-01 Olympus Optical Co Endoscope monitor apparatus
JP2006116153A (ja) * 2004-10-22 2006-05-11 Olympus Corp 内視鏡用画像処理装置及び内視鏡装置
WO2012132790A1 (ja) * 2011-03-31 2012-10-04 オリンパス株式会社 蛍光観察装置
JP2016154588A (ja) * 2015-02-23 2016-09-01 Hoya株式会社 画像処理装置
WO2016175084A1 (ja) * 2015-04-30 2016-11-03 富士フイルム株式会社 画像処理装置、方法及びプログラム
WO2017057574A1 (ja) * 2015-09-29 2017-04-06 富士フイルム株式会社 画像処理装置、内視鏡システム、及び画像処理方法
JP2018126632A (ja) * 2018-05-25 2018-08-16 Hoya株式会社 画像処理装置

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117500426A (zh) * 2021-06-16 2024-02-02 富士胶片株式会社 医疗图像处理装置、内窥镜系统及医疗图像处理装置的工作方法
WO2023132138A1 (ja) * 2022-01-07 2023-07-13 富士フイルム株式会社 プロセッサ装置及びその作動方法並びに内視鏡システム

Also Published As

Publication number Publication date
US20240320877A1 (en) 2024-09-26
US12430820B2 (en) 2025-09-30
JP7284814B2 (ja) 2023-05-31
US12020350B2 (en) 2024-06-25
US20220076458A1 (en) 2022-03-10
JPWO2020235502A1 (https=) 2020-11-26

Similar Documents

Publication Publication Date Title
CN110325100B (zh) 内窥镜系统及其操作方法
US11439297B2 (en) Medical image processing system, endoscope system, diagnosis support apparatus, and medical service support apparatus
US12430820B2 (en) Image processing apparatus
US12564307B2 (en) Image analysis processing apparatus, endoscope system, operation method of image analysis processing apparatus, and non-transitory computer readable medium
US11627864B2 (en) Medical image processing apparatus, endoscope system, and method for emphasizing region of interest
JP7335399B2 (ja) 医用画像処理装置及び内視鏡システム並びに医用画像処理装置の作動方法
CN115361898B (zh) 医疗图像处理装置、内窥镜系统及医疗图像处理装置的工作方法、非暂时性计算机可读介质
US20230237659A1 (en) Image processing apparatus, endoscope system, operation method of image processing apparatus, and non-transitory computer readable medium
US11998165B2 (en) Image processing device and method of operating same
WO2019198576A1 (ja) 医療画像処理装置
JP7196196B2 (ja) 内視鏡システム
JP7556961B2 (ja) 内視鏡システム、内視鏡システムの作動方法、及び内視鏡システム用プログラム
JP7750938B2 (ja) プロセッサ装置、医療画像処理装置、及び医療画像処理システム、並びに内視鏡システム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20809903

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021520774

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20809903

Country of ref document: EP

Kind code of ref document: A1