JP2013116353A - Image processing apparatus and endoscope system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing apparatus that can process an image captured by an endoscope system to facilitate visual understanding of a desired portion in the image, and to provide an endoscope system.SOLUTION: The image processing apparatus includes: a color image acquiring unit for acquiring an image captured with colors expressed by a combination of RGB three colors; a component designation unit for receiving a designation of a light wave-length component; and an emphasizing process unit for processing the image captured by the color image acquisition unit to emphasize or de-emphasize the wave-length component designated by the component section. The emphasizing process unit is configured to decompose the captured image into a plurality of spectral images, extract a designated wave-length spectral images having the wave-length component designated by the component designation unit, and emphasize or de-emphasize the designated spectral images, thereby generating, by integral treatment, a processed image in which each pixel is expressed by pixel values of RGB three colors.

Description

  The present invention relates to an image processing apparatus and an endoscope system that perform image processing on a color photographed image.

  Conventionally, in the medical field, an endoscope system in which a long and narrow tube having a CCD attached at the tip is inserted into the body of a subject and the inside of the subject is photographed to observe a tumor or a thrombus is widely used. Has been. According to such an endoscope system, by directly photographing the inside of the subject, it is possible to grasp the color and shape of a lesion that is difficult to understand with a radiographic image without causing external damage to the subject. It is possible to easily obtain information necessary for determining a treatment policy.

  Here, since the captured image obtained by the endoscope system is usually a copy of the mucous membrane in the body, the image is often similar in color and shape, so the lesion is visually observed. There are many cases where it is difficult to grasp by this, which is a burden on the doctor.

  In the general field of photography, as an example of a technique for grasping a specific part in a photographed image, for example, a technique for automatically extracting a person's face based on a color of a photographed image, pattern matching, or the like is known. (For example, refer to Patent Document 1). However, as described above, the photographed image obtained by the endoscope system is different from a general photographed image in which a person, a building, or the like can be easily distinguished from each other, and the mucous membrane is reflected on one side. As you can see, there are many things that look similar on the whole. Therefore, even if the technique described in Patent Document 1 is used, it is difficult to automatically extract a lesion that often has a subtle difference from the surroundings from an image of a single mucous membrane. Reducing the burden is not realistic.

  Incidentally, in recent years, in the field of endoscope systems, a technique of performing spectrum analysis on a captured image obtained by an endoscope system and extracting a component image (spectral image) of a desired spectral component has attracted attention. (For example, refer to Patent Document 2). For example, lesions are often reddened by hyperemia. Although hemoglobin, which is a major component of blood, is concentrated in such locations (redness locations), it is known that this hemoglobin has significant absorption and reflection for light of a specific wavelength. . In addition, when photographing using an endoscope system, a pigment for emphasizing the unevenness of a lesion, for example, may be sprayed on the photographing location for the purpose of assisting visual observation. The absorption and reflection of light is remarkable. If the technique described in Patent Document 2 is used, a spectral image corresponding to a spectral component that exhibits significant absorption and reflection in hemoglobin, and a spectral component that corresponds to a spectral component that exhibits significant absorption and reflection from the dispersed pigment. An image can be obtained. In such a spectroscopic image, the contrast or the like of a portion where hemoglobin is concentrated or a portion where a pigment is dispersed is emphasized more than other portions, so that it becomes easier to visually grasp the shape of the lesion.

  In addition, as another example of the technology for obtaining such a spectral image, the above-mentioned spectral light is used by illuminating a subject in an endoscope system by using illumination having a wavelength that is remarkably absorbed or reflected by hemoglobin or pigment. A technique has also been proposed in which an image similar to an image is obtained directly by photographing (see, for example, Patent Document 3).

JP-A-9-101579 JP 2003-93336 A JP 2006-346358 A

  Here, the spectral image is a monochromatic image corresponding to a spectral component that is predicted to be prominent in a lesion. Therefore, in the spectroscopic image, the lesion is clearly visible, but most of the other portions are not reflected, and there is a problem that it is difficult to grasp the positional relationship between these other portions and the lesion. In addition, since this spectral image is a monochromatic image, there is often a problem that it is difficult to distinguish such a different part from the lesion when a different part that looks misleading with the lesion is shown. For this reason, among users of an endoscope system, an image processing apparatus capable of performing image processing that makes it easy to visually grasp a spot to be found on a captured image obtained by the endoscope system, An endoscope system incorporating such an image processing apparatus is desired.

  In addition, so far, the example of the captured image obtained by the endoscope system has been described, and the problem of difficulty in visually grasping the location to be found in the captured image has been described. Although it is likely to occur in a captured image obtained by an endoscope system, it is a problem that may occur in a general captured image.

  In view of the above circumstances, the present invention incorporates an image processing apparatus capable of performing image processing that makes it easy to visually grasp a spot to be found on a photographed image, and such an image processing apparatus. An object is to provide an endoscope system.

The image processing apparatus of the present invention that achieves the above object provides:
A color image acquisition unit that acquires a captured image of a color in which the color of each pixel is expressed by a combination of RGB three colors obtained by capturing an object using an input device having a predetermined spectral sensitivity distribution for each of the three RGB colors When,
A component designating unit that receives designation of the wavelength component of light;
An enhancement processing unit that performs image processing for emphasizing or de-emphasizing the wavelength component specified by the component designation unit on the captured image acquired by the color image acquisition unit;
Emphasis processing section
The photographed image is decomposed into a plurality of spectral images having different wavelengths over a predetermined wavelength range,
Extract a specified wavelength spectral image having a wavelength component specified by the component specifying unit from a plurality of spectral images,
Apply emphasis or anti-emphasis processing to the specified wavelength spectral image,
A spectral distribution represented by a set of pixel values of pixels corresponding to each other in the plurality of spectral images is obtained by using the spectral images after the enhancement or de-emphasis processing for the plurality of spectral images and the specified wavelength spectral image. By weighting and integrating with spectral sensitivity distribution for each color, each pixel generates a processed image expressed by pixel values of RGB three colors.

Here, the image processing apparatus of the present invention is
A component index indicating the wavelength component of light, a plurality of component indexes indicating different wavelength components, and a plurality of selection items that are selected by a predetermined selection operation in a one-to-one correspondence A correspondence storage unit for storing the table;
An operation accepting unit that accepts a selection operation and designates the wavelength component indicated by the component index associated with the selection item selected by the selection operation in the component designation unit. An endoscope system of the present invention that achieves the above-described object provides:
A light source that emits light;
An optical probe equipped with an imager that captures a color image by photographing a subject;
A color image acquisition unit that acquires a captured image of a color in which the color of each pixel is expressed by a combination of RGB three colors obtained by capturing an object using an input device having a predetermined spectral sensitivity distribution for each of the three RGB colors When,
A component designating unit that receives designation of the wavelength component of light;
An enhancement processing unit that performs image processing for emphasizing or de-emphasizing the wavelength component specified by the component designation unit on the captured image acquired by the color image acquisition unit;
Emphasis processing section
The photographed image is decomposed into a plurality of spectral images having different wavelengths over a predetermined wavelength range,
Extract a specified wavelength spectral image having a wavelength component specified by the component specifying unit from a plurality of spectral images,
Apply emphasis or anti-emphasis processing to the specified wavelength spectral image,
A spectral distribution represented by a set of pixel values of pixels corresponding to each other in the plurality of spectral images is obtained by using the spectral images after the enhancement or de-emphasis processing for the plurality of spectral images and the specified wavelength spectral image. An image processing apparatus that generates a processed image in which each pixel is expressed by a pixel value of RGB three colors by weighting and integrating the spectral sensitivity distribution for each color; and image processing is performed by the image processing apparatus And a display device for displaying the processed image after the operation.

  The endoscope system of the present invention is only shown here in its basic form, but this is merely to avoid duplication, and the endoscope system of the present invention has only the above basic form. Instead, various forms corresponding to the respective forms of the image processing apparatus described above are included.

  As described above, according to the present invention, an image processing apparatus capable of performing image processing that makes it easy to visually recognize a spot to be found on a captured image, and such an image processing apparatus. Can be obtained.

1 is a diagram showing an endoscope system to which an embodiment of the present invention is applied. FIG. It is a figure which shows an image enhancement parameter set screen. It is a figure which shows an example of the light absorption characteristic and reflection characteristic of hemoglobin. FIG. 10 is a schematic diagram showing a flow of processing until an emphasized image is displayed on the monitor 300 after setting parameters. FIG. 5 is a schematic diagram showing a color photographed image G1 subjected to the spectral estimation process (step S101) in FIG. It is a schematic diagram which shows the spectral image GLi extracted from the color picked-up image G1 of FIG. It is a schematic diagram which shows the spectral image GLi after a binarization process. It is a schematic diagram which shows a mode that the image area was specified by template matching. It is a schematic diagram which shows an example of an emphasized image. It is a figure which shows a 2nd endoscope system. It is a figure which shows a pigment name set screen. FIG. 6 is a schematic diagram showing a flow of processing until a pseudo color image is displayed on a monitor after setting a pigment name. It is a figure which shows an example of the color picked-up image G1 which copied the state by which the pigment | dye was spread | dispersed to the protrusion-like lesion. It is a figure which shows an example of the pseudo color image G3 produced | generated from the color picked-up image G1 of FIG. It is a figure which shows a 2nd endoscope system. It is a figure which shows an image enhancement parameter set screen. FIG. 10 is a schematic diagram showing a flow of processing until an emphasized image is displayed on the monitor 300 after setting parameters. It is a schematic diagram which shows a mode that the spectral characteristic of each pixel in the enhancement image G4 is acquired. It is a figure which shows the spectral sensitivity characteristic in CCD120. It is a schematic diagram which shows a mode that the pixel value (R, G, B) of each pixel is calculated from the spectral characteristic S of each pixel in the enhancement image G4 and the spectral characteristic of the light receiving element. It is a figure which shows a 4th endoscope system. FIG. 6 is a schematic diagram showing a flow of processing until a peak color image is displayed on a monitor 300 after an operation of a switching button 602.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a diagram showing an endoscope system according to an embodiment of the present invention.

  An endoscope system 10 shown in FIG. 1 includes a scope 100 that is inserted into a subject, illuminates a subject inside the subject with white light, and obtains a color photographed image by one photographing under the illumination. 100 includes a processor 200 that performs various kinds of image processing and the like on the color photographed image obtained in 100, and a monitor 300 that displays a color photographed image that has undergone image processing and the like in the processor 200. Here, the scope 100 corresponds to an example of an optical probe according to the present invention, and the monitor 300 corresponds to an example of a display device according to the present invention. The processor 200 corresponds to an embodiment of the image processing apparatus of the present invention.

  The scope 100 is detachably attached to the processor 200 and includes an optical fiber light guide 110 having an illumination window 111 at the tip and a CCD 120 for photographing a subject. The light guide 110 guides the illumination light supplied from the light source 201 mounted on the processor 200 to the subject inside the subject and irradiates it from the illumination window 111. The CCD 120 is illuminated under the illumination by the irradiated light. Analog image data representing the color image of the subject by expressing the color reflected by the combination of three colors of R (red), G (green), and B (blue) by receiving the light reflected by the subject Generate. The light source 201 corresponds to an example of the light source referred to in the present invention.

  The scope 100 further performs correlated double sampling (CDS) with a scope-side CPU 130 that controls the operation of each component in the scope 100, a CCD drive circuit 140 that drives the CCD 120 based on the control of the scope-side CPU 130, and the like. A CDS / AGC circuit 150 that performs automatic gain control (AGC) for appropriately amplifying analog data and an A / D conversion circuit 160 that converts analog data into digital data are provided. Analog image data generated by the CCD 120 driven by the CCD drive circuit 140 is passed to the A / D conversion circuit 160 via the CDS and AGC, and converted into digital image data. The digital image data is passed to the processor 200. A combination of the CCD 120, the scope side CPU 130, the CCD drive circuit 140, the CDS / AGC circuit 150, and the A / D conversion circuit 160 corresponds to an example of the image pickup device according to the present invention.

  Here, in the scope 100, basically, a moving image is shot, digital image data representing a color moving image is obtained and passed to the processor 200. When the user operates an operation button (not shown) or the like to instruct to shoot a still image, the still image at that time is captured, and digital image data representing a color still image is obtained and passed to the processor 200. It is.

  In the following embodiments, processing for this color still image will be described. However, the image processing of the present invention can be applied to moving images, and is not limited to still images.

  The processor 200 includes a light source 201 that supplies light to the light guide 110 and a processor-side CPU 202 that controls each component of the processor 200.

  In this embodiment, the processor 200 performs basic image processing such as gamma correction and saturation correction on the color photographed image represented by the digital image data delivered from the scope 100 and displays the image on the monitor 300. Two emphasis displays: a basic display function and a function of displaying on the monitor 300 by performing emphasis processing for emphasizing a portion designated by the user as will be described later in addition to basic image processing on the captured image of the color It has a function.

  The processor 200 switches the image displayed on the monitor 300 between a color photographed image (normal image) displayed with the basic display function and a color photographed image (emphasized image) displayed with the highlight display function. A switching button 203 that accepts a switching operation from the camera 100, digital image data is acquired from the scope 100, and the digital image data is either a basic display function or a highlight display function under the control of the processor-side CPU 202 according to the switching operation. And a switching circuit 204 to be used. The switching circuit 204 corresponds to an example of a color image acquisition unit referred to in the present invention.

  Furthermore, the processor 200 includes a first image processing unit 205 that performs image processing in the basic display function, and a second image processing unit 206 that performs image processing and enhancement processing in the highlight display function. Further, when executing display by the highlighting function, the processor 200 performs spectrum analysis on the color photographed image, so that a component image (spectral image) of a spectrum component corresponding to a designation operation described later is obtained from the photographed image. And a spectral estimation processing unit 207 that performs spectral estimation processing to obtain an image region that satisfies the condition that the spectral image obtained by the spectral estimation processing unit 207 matches the shape according to the designated operation, A specific processing unit 208 that transmits the position of the specified image region to the second image processing unit 206. The spectral estimation process is a known technique disclosed in, for example, Japanese Patent Application Laid-Open No. 2006-239206, and the description thereof is omitted here. The second image processing unit 206 performs enhancement processing for emphasizing the image area transmitted from the specific processing unit 208 on the color photographed image. In addition, the processor 200 includes a display control unit 209 that displays on the monitor 300 a color photographed image that has undergone the processing in the first image processing unit 205 or the processing in the second image processing unit 206, and for image processing and enhancement processing. And a memory 210 for storing various control programs, image data, and the like. A combination of the spectral estimation processing unit 207 and the specific processing unit 208 corresponds to an example of the region specifying unit in the present invention. The second image processing unit 206 corresponds to an example of the image processing unit referred to in the present invention.

  In the processor 200, the digital image data transferred from the scope 100 is transferred to the first image processing unit 205 or the second image processing unit 206 by the switching circuit 204.

  Here, in this embodiment, every time the user presses the switching button 203, the switching circuit 204 is moved under the control of the processor side CPU 202, and the destination of the digital image data is sent from the first image processing unit 205 to the second image processing. Switching to the unit 206 or from the second image processing unit 206 to the first image processing unit 205. Further, when the digital image data is transferred to the second image processing unit 206, the digital image data is also transferred to the spectral estimation processing unit 207. The digital image data is stored in the memory 210 via the switching circuit 204 and the processor side CPU 202.

  When the digital image data is transferred to the first image processing unit 205 by the switching circuit 204, basic image processing such as gamma correction and saturation correction is performed on the color photographed image represented by the digital image data, and the display is performed. A normal image is displayed on the monitor 300 by the control unit 209. On the other hand, when the digital image data is transferred to the second image processing unit 206, the image region to be emphasized is specified through the processing in the spectral estimation processing unit 207 and the specifying processing unit 208, and the second image is processed. In the processing unit 206, basic image processing is performed on the color photographed image represented by the digital image data, the specified image area in the photographed image is emphasized, and the display control unit 209 creates the emphasized image. Displayed on the monitor 300.

  Hereinafter, a series of processing from when the display on the monitor 300 is instructed to be switched from the normal image display to the highlighted image display by the operation of the switching button 203 until the highlighted image is actually displayed on the monitor 300 will be described. explain.

  When the switching button 203 is pressed while a moving image is displayed on the monitor 300, an image captured by the CCD 120 at that time is shot as a still image, and digital image data representing the still image is stored in the memory 210. Remembered. Thereafter, an image enhancement parameter set screen described later is displayed on the monitor 300 in order to set various parameters in the enhancement processing for the still image.

  Further, when the switch button 203 is pressed while a still image obtained by operating an operation button (not shown) and having already stored digital image data in the memory 210 is displayed on the monitor 300, In order to set various parameters in the enhancement process for the currently displayed still image, an image enhancement parameter set screen described later is displayed on the monitor 300.

  FIG. 2 is a diagram showing an image enhancement parameter set screen.

  In the present embodiment, a parameter / auto set mode for automatically setting various parameters in the emphasis processing and a parameter / manual set mode for manually setting various parameters individually are prepared. In the image enhancement parameter set screen 220 of FIG. 2, the parameter / auto set mode is enabled and the parameter / manual set mode is disabled. Conversely, the parameter / manual set mode is enabled and the parameter / manual set mode is enabled. A second radio button 222 for disabling the auto set mode is provided.

  First, the parameter / manual set mode will be described.

  In this parameter / manual set mode, first, the wavelength of the spectral component in the spectral image to be extracted from the color photographed image is set. The image enhancement parameter set screen 220 includes a wavelength setting unit 223, and the user clicks an increase / decrease button 223a in the wavelength setting unit 223 with a mouse (not shown) or a keyboard (not shown) on the display unit 223b. The desired wavelength can be set by directly inputting a numerical value by.

  Here, the wavelength set in the wavelength setting unit 223 is, for example, the wavelength of a spectral component that is remarkably absorbed or reflected by hemoglobin in the case where it is desired to emphasize a red spot that is often found in a lesion, When a dye having a different degree of coloring is used in other places, the wavelength of the spectral component or the like where the absorption or reflection by the dye is remarkable.

  In this embodiment, when a spectral image is extracted from a color photographed image, a region darker than a certain reference brightness with respect to the spectral image, in order to identify a reddish portion or a colored portion, A binarization process is performed to binarize the area brighter than the brightness. In the present embodiment, the spectral image is handled as image data including pixel values indicating the brightness of each pixel constituting the spectral image. This pixel value is a value within the range of “0” to “255”. Then, any value within the range of “0” to “255” is set on the image enhancement parameter set screen 220 as a threshold value indicating the reference brightness for the above binarization. . The image enhancement parameter set screen 220 is provided with a threshold value setting unit 224 for setting this threshold value. Similar to the wavelength setting unit 223, in this threshold setting unit 224, the user sets a desired threshold by clicking the increase / decrease button 224a with a mouse or directly inputting a numerical value to the display unit 224b with a keyboard. can do.

  Further, in this embodiment, in order to identify a red spot or a colored lesion spot, the region specifying unit generally predicts that the shape of the image such as a red spot or a lesion spot in the spectroscopic image. The image area in which the shape is shown is identified. The image enhancement parameter set screen 220 is provided with a shape setting unit 225 for setting a desired shape. In the present embodiment, a plurality of types of shapes such as a circle corresponding to the shape of an image such as a red spot or a polyp, a continuous curve corresponding to the shape of a blood vessel, and the like are prepared as candidates. It is possible to set the shape. When the menu button 225a of the shape setting unit 225 is clicked, a pull-down menu listing the shapes of images that can be set is displayed, and when the user moves the cursor to a desired shape, the shape is set. The set shape is displayed on the display unit 225b.

  Further, on this image enhancement parameter setting screen 220, after the image region is specified, the degree of enhancement in each of the three types of enhancement processing, sharpness enhancement, contrast enhancement, and color enhancement, that can be executed in the present embodiment is set. be able to. Here, each of these three types of enhancement processing corresponds to an example of the enhancement processing referred to in the present invention. The image enhancement parameter set screen 220 includes a sharpness enhancement setting unit 226 that sets the degree of sharpness enhancement, a contrast enhancement setting unit 227 that sets the degree of contrast enhancement, and a color enhancement setting unit 228 that sets the degree of color enhancement. Is provided.

  The degree of sharpness enhancement is set to a value greater than or equal to “0” by a click operation on the increase / decrease button 226a of the sharpness enhancement setting unit 226 or a keyboard input to the display unit 226b. Similarly, the degree of contrast enhancement is set by a click operation on the increase / decrease button 227a of the contrast enhancement setting unit 227 or a keyboard input to the display unit 227b, and the degree of color enhancement is a click operation on the increase / decrease button 228a of the color enhancement setting unit 228. And is set by keyboard input to the display unit 228b. Here, when the value “0” is set, the enhancement processing is not executed. FIG. 2 is an example in which only color enhancement is performed.

  As described above, in the parameter / manual set mode, the wavelength of the spectral component in the spectral image to be extracted from the color photographed image, the threshold value in the binarization process, the shape of the image, and the degree of enhancement in each enhancement process are individually described. Set to These parameters are input from the user by a setting operation via the image enhancement parameter setting screen 220 and acquired by the processor side CPU. The processor-side CPU corresponds to an example that serves as both the component designation unit and the shape designation unit according to the present invention.

  On the other hand, in the parameter / auto set mode, all of these parameters are automatically set by setting a specific name of a location desired to be emphasized, such as a red spot.

  The parameter / auto set mode will be described below.

  In this embodiment, the parameters required for identification and enhancement of each candidate have a one-to-one correspondence with the names of multiple types of enhancement candidates such as red spots, polyps colored with a certain pigment (for example, indicocarmine), blood vessels, and the like. The attached correspondence table is stored in the memory 210 of FIG. The name of the enhancement candidate corresponds to an example of a selection item referred to in the present invention. The memory 210 corresponds to an example of a correspondence storage unit referred to in the present invention.

  In the present embodiment, for example, at the red spot, the wavelength is “540 nm”, the threshold is “64”, the shape is “circular”, the sharpness enhancement degree is “0”, the contrast enhancement degree is “0”, and the color enhancement. “2” is associated as the degree of.

  Among these parameters, the fact that the wavelength of “540 nm” is associated with the red spot is based on the absorption and reflection characteristics of hemoglobin concentrated at the red spot.

  FIG. 3 is a diagram showing an example of light absorption characteristics and reflection characteristics of hemoglobin.

Part (a) of FIG. 3 shows a graph E1 showing an example of the light absorption characteristic of hemoglobin, and Part (b) shows a graph E2 showing an example of the reflection characteristic of hemoglobin. In graph E1 of part (a), the horizontal axis indicates the wavelength and the vertical axis indicates the molar extinction coefficient, and the light absorption characteristics measured for each of reduced hemoglobin Hb and oxidized hemoglobin HbO ₂ are described. . From this graph E1, it can be seen that there are absorption peaks in the vicinity of “415 nm” and in the vicinity of “540 nm”. Further, in the graph E2 of Part (b), the horizontal axis indicates the wavelength and the vertical axis indicates the spectral reflectance, which is obtained by measuring the reflection characteristics of the mucous membrane of the digestive organ for a plurality of subjects. In addition, the reflection characteristics of hemoglobin are described. In this graph E2, there is a drop in reflection in the vicinity of “415 nm” and in the vicinity of “540 nm”, and it can be seen from this graph E2 that there are absorption peaks at these two locations.

  In the present embodiment, “540 nm” is employed as the wavelength corresponding to the spectral image to be extracted from the color photographed image in order to identify the red spot based on these characteristics. In the present embodiment, “415 nm” is adopted as the wavelength corresponding to the spectral image for the blood vessel.

  The image enhancement parameter set screen 220 shown in FIG. 2 includes a name setting unit 229 for setting names of enhancement candidates such as red spots in the parameter / auto set mode. When the menu button 229a of the name setting unit 229 is clicked, a pull-down menu in which names of emphasis candidates that can be set are listed is displayed. When the user moves the cursor to a desired name, the name is set. The set name is displayed on the display unit 229b. When the name of the emphasis candidate is set by the name setting unit 229, each parameter associated with the name in the correspondence table is read from the memory 210 by the processor side CPU 202 in FIG. Here, the processor side CPU 202 also serves as an example of an operation receiving unit according to the present invention.

  The image enhancement parameter set screen 220 is provided with a determination button 230 for approving the set contents in the parameter / manual set mode or the parameter / auto set mode. When the determination button 230 is clicked after the parameter setting is completed, processing using the set parameters is started in the spectral estimation processing unit 207 and the specific processing unit 208.

  Next, the flow of processing using these parameters will be described.

  In the following description, it is assumed that parameters necessary for specifying and emphasizing redness are set as parameters.

  FIG. 4 is a schematic diagram showing a flow of processing until the emphasized image is displayed on the monitor 300 after the parameter is set.

  A color photographed image G1 represented by digital image data expressing colors by a combination of three RGB colors is subjected to spectral estimation processing (step S101) in the spectral estimation processing unit 207.

  The color photographed image G1 includes a plurality of spectral images GL1, GL2,... GLn corresponding to different wavelengths. In the spectral estimation process, a spectral image GLi having a wavelength (“540 nm” in this example) set on the image enhancement parameter set screen 220 is extracted from the color captured image G1.

  FIG. 5 is a schematic diagram showing the color captured image G1 subjected to the spectral estimation process (step S101) in FIG. 4, and FIG. 6 shows the spectral image GLi extracted from the color captured image G1 in FIG. It is a schematic diagram shown.

  FIG. 5 shows a reddening spot A1 for which enhancement (color enhancement of “+2” in this example) is desired by the user, and a blood vessel A2 around the reddening spot A1. Since hemoglobin is concentrated in the reddening portion A1 and the blood vessel A2, light having a wavelength of “540 nm” is intensively absorbed at these portions. Therefore, in the spectroscopic image GLi having a wavelength of “540 nm”, as shown by hatching in FIG. 6, the reddening portion A1 and the blood vessel A2_1 that is close to the surface of the mucosa and has a large absorbance are darker than the other portions.

  When such a spectral image GLi is obtained by the spectral estimation process (step S101) in FIG. 4, the area specifying process described below is performed in the specifying process unit 208 (step S102).

  In this area specifying process (step S102), first, a binarization process for binarizing the spectral image GLi with the threshold (“64” in this example) set on the image enhancement parameter set screen 220 as a boundary. Done.

  FIG. 7 is a schematic diagram showing a spectral image GLi after binarization processing.

  As shown in FIG. 7, by the binarization processing, in the spectral image GLi, only the reddening spot A1 and a part of the blood vessel A2_1, which are darkly reflected by the hemoglobin, remain as images, and the other parts are uniform. It is leveled.

  When the binarization process is completed, in the region specifying process (step S102) in FIG. 4, the shape set on the image enhancement parameter set screen 220 in the spectral image GLi after the binarization process (in this example, “ A template corresponding to “circle”) is used, and an image region is specified by template matching, which is a well-known technique.

  FIG. 8 is a schematic diagram showing a state in which an image region is specified by template matching.

  As shown in FIG. 8, in the region specifying process (step S102), after the binarization process, a certain degree of match with respect to the circle is obtained for each part in the spectral image GLi from which unnecessary portions are removed. Template matching is performed. Usually, the red spot in the lesion often has a circular shape, and as shown in FIG. 8, the red spot A1 matches the circle more than a certain degree. In the present embodiment, the term “circular” is used in a broad sense including not only a perfect circle but also an ellipse, an ellipse, and the like. Then, when a certain degree of matching is obtained by this template matching, the matched circular area A3 is an image area as a result of processing in the area specifying process (step S102). Then, as shown in FIG. 4, mask information J1 representing the location of the circular area A3 and the like is generated and passed from the specific processing unit 208 to the second image processing unit 206 in FIG.

  In the second image processing unit 206, as shown in FIG. 4, the image enhancement parameter set screen 220 sets the image portion of the circular area A4 corresponding to the place represented by the mask information J1 in the color photographed image G1. The enhanced processing (in this example, “+2” color enhancement) is performed (step S103). As a result, an enhanced image G2 in which the image portion of the circular area A4 is enhanced is obtained. The emphasized image G2 is displayed on the monitor 300 by the display control unit 209.

  FIG. 9 is a schematic diagram illustrating an example of an emphasized image.

  In FIG. 9, in the color photographed image G1, the emphasized image G2 in which the color enhancement is applied to the image portion in the circular area A4 in which the mask information J1 indicates the place or the like, and the emphasized portion is hatched. It is shown in In FIG. 9, a circular region A4 that is not originally displayed is indicated by a dotted line in order to make the drawing easier to understand.

  According to the emphasized image G2, the reddening portion is displayed with the color emphasized more than the surroundings, so that the doctor can easily grasp the reddening portion visually in the emphasized image G2. In addition, since other portions in the periphery are reflected as an original color image, it is possible to easily grasp the positional relationship between the other portions and the redness portion. Further, in the enhanced image G2, the red spot is emphasized, so that the color characteristics are similar to the red spot, and the red color also appears in another part that looks confusingly with the original red photographed image G1. It is easy to distinguish from the location.

  As described above, according to the endoscope system 10 of the present embodiment, it is possible to perform image processing that makes it easy to visually grasp a spot to be discovered, such as a lesion, on a captured image.

  In the above description, the color photographed image obtained by one photographing under illumination with white light is exemplified as the color photographed image according to the present invention, but the present invention is not limited to this. The color photographed image referred to in the invention may be obtained by combining a plurality of monochrome photographed images obtained by photographing a plurality of times under the illumination of each of a plurality of types of monochromatic light having different wavelengths. good.

  In the above, as an example of the region specifying unit according to the present invention, the spectral estimation unit 207 that obtains a spectral image corresponding to a wavelength set by the user by performing spectral analysis on a color photographed image. Although illustrated, the present invention is not limited to this. The area specifying unit of the present invention is set by the user among the plurality of monochrome photographed images when the color photographed image is obtained by combining the plurality of monochrome photographed images as described above. A captured image obtained under monochromatic light having a different wavelength may be obtained as a spectral image.

  In the above, as an example of the region specifying unit according to the present invention, the spectral image corresponding to the wavelength set by the user is binarized with the threshold set by the user, and the spectral after the binarization processing is performed. Although the image having the specific processing unit 208 that identifies the image area that matches the shape set by the user is exemplified in the image, the present invention is not limited to this. The area specifying unit of the present invention may specify an image area by omitting the binarization process, or may specify an image area only by comparison with a threshold value.

  In the above, three types of enhancement processing, sharpness enhancement, contrast enhancement, and color enhancement, have been illustrated as an example of the enhancement processing according to the present invention. However, the present invention is not limited to these, and the enhancement according to the present invention. The processing may be enhancement processing other than the above three types such as hypertone processing.

  Further, in the above, as an example of the specified image region, an image region in which an image of a spot to be discovered such as a lesion is illustrated, and as an example of image processing to be performed on the specified image region, the image region is Although the emphasis process for emphasizing has been illustrated, the present invention is not limited to this, and the specified image area may be an image area in which an image of a place that hinders the detection of a lesion or the like is captured. May be a process of de-emphasizing the image region thus identified.

  In the above, a color still image is illustrated as an example of a color photographed image according to the present invention. However, the present invention is not limited to this, and a color photographed image according to the present invention is a color moving image. There may be.

  This is the end of the description of the endoscope system according to the embodiment of the present invention. Hereinafter, the other three types of endoscope systems will be described. In the following, these three types of endoscope systems are referred to as a second endoscope system, a third endoscope system, and a fourth endoscope system, respectively.

  First, the second endoscope system will be described.

  For example, in the technique described in Patent Document 2 described above, a process of performing spectrum analysis on a captured image obtained by an endoscope system and extracting a desired spectral image is performed. Which spectral image is to be extracted? Is determined by designation of the wavelength by the user operation.

  By the way, photographing with an endoscope system may be performed by spraying a pigment for emphasizing unevenness or the like often found in a lesion on a region to be photographed. And if the technique of the above-mentioned patent document 2 is utilized with respect to the color picked-up image obtained under the dispersion | distribution of such a pigment | dye, the wavelength of the wavelength corresponding to the disperse | distributed pigment | dye is used from the color picked-up image. Image processing for extracting a spectral image can be performed. In the spectral image extracted in this way, the spot where the pigment is dispersed is shown more emphasized than the surrounding area, so that it is easy to grasp the unevenness of the lesion.

  However, the technique described in Patent Document 2 has a problem that image processing cannot be performed in the first place unless the wavelength corresponding to the dye is known. The second endoscope system, which will be described later, is equipped with an image processing apparatus that can solve such problems and can easily perform image processing that makes it easy to grasp the unevenness of a lesion from a color photographed image. Is.

  FIG. 10 is a diagram showing a second endoscope system.

  In FIG. 10, components equivalent to those of the endoscope system 10 shown in FIG. 1 are denoted by the same reference numerals as those in FIG. 1, and in the following, redundant description of these components will be given. Omitted.

  The second endoscope system 20 shown in FIG. 10 takes out a spectral image having a wavelength corresponding to the dispersed pigment instead of the above-described highlighting function, and generates a pseudo color image to be described later based on the spectral image. 1 is different from the endoscope system 10 shown in FIG. 1 in having a pseudo color display function for generating and displaying. Hereinafter, the description will be made with attention paid to the pseudo color display function.

  The processor 400 of the second endoscope system 20 in FIG. 10 displays an image to be displayed on the monitor 300 as a color captured image (normal image) displayed with the basic display function and a pseudo image displayed with the pseudo color display function. A switch button 402 that accepts a switching operation from a user that switches between color images, digital image data is acquired from the scope 100, and the digital image data is displayed as a basic display under the control of the processor-side CPU 401 according to the switching operation. And a switching circuit 403 used for either the function or the pseudo color display function.

  Further, the processor 400 performs spectral analysis on the color captured image, thereby performing spectral estimation processing for obtaining one or more and three or less spectral images corresponding to a designation operation described later from the captured image. Each of the three colors RGB is assigned to each of the spectral images obtained by the unit 404 and the spectral estimation processing unit 404 in accordance with an assignment rule described later, so that a pseudo image composed of the spectral images to which those colors are assigned. And a pseudo color image generation unit 405 that obtains a color image. The pseudo color image generated by the pseudo color image generation unit 405 is displayed on the monitor 300 by the display control unit 209.

  Hereinafter, a series of operations from when the display on the monitor 300 is instructed to be switched from the normal image display to the pseudo color image display by the operation of the switching button 402 until the pseudo color image is actually displayed on the monitor 300 will be described. Processing will be described.

  In the following embodiment, processing for a color still image will be described. However, the image processing here can be applied to a moving image, and is not limited to a still image.

  The switch button 402 is pressed while a normal moving image is displayed on the monitor 300, and the switch button 402 is pressed after a still image is captured at that time or when a still image is displayed on the monitor 300. After that, a dye name setting screen for setting the dye name of the dispersed dye is displayed on the monitor 300.

  FIG. 11 is a diagram showing a pigment name set screen.

  In the dye name set screen 410 shown in FIG. 11, it is possible to set three desired dye names from the first name to the third name from among a plurality of predetermined dye names. This dye name set screen 410 includes a first name set unit 411, a second name set unit 412, and a third name set unit 413. When the menu buttons 411a, 412a, and 413a of the name setting units 411, 412 and 413 are clicked, a pull-down menu in which settable dye names are listed is displayed, and the user moves the cursor to a desired dye name. The dye name is set. The set pigment names are displayed on the display portions 411b, 412b, and 413b. The pigment name set screen 410 is provided with a determination button 414 for approving the content of the pigment name set. When the determination button 414 is clicked after the dye name is set, processing in the spectral estimation processing unit 404 and the pseudo color image generation unit 405 is started.

  Next, the flow of processing in the spectral estimation processing unit 404 and the pseudo color image generation unit 405 will be described.

  FIG. 12 is a schematic diagram showing the flow of processing until the pseudo color image is displayed on the monitor 300 after the dye name is set.

  In the second endoscope system 20, a plurality of types of dye names that can be set on the dye name set screen 410 in the memory 210 of FIG. 10, and a wavelength at which an absorption peak appears in the light absorption characteristics of each dye name. Are stored in a correspondence table in which one-to-one correspondence is made. Below, an example of the combination of a pigment name and a wavelength is shown.

  A dye name of indicocarmine is associated with a wavelength of 612 nm, a dye name of methylene blue is associated with a wavelength of 660 nm, a dye name of Evans blue is associated with a wavelength of 620 nm, and a dye name of crystal violet Is associated with a wavelength of 590 nm.

  When one or more and three or less dye names are set on the dye name set screen 410, first, the wavelength associated with the set dye name is read from the memory 210 by the processor side CPU 401, and spectral estimation is performed. The information is transmitted to the processing unit 207 and the pseudo color image generation unit 405 (step S201).

  Next, a color photographed image G1 represented by digital image data representing colors by a combination of three RGB colors is subjected to spectral estimation processing (step S202) in the spectral estimation processing unit 207. In the spectral estimation processing, from the color photographed image G1 that includes a plurality of spectral images GL1, GL2,..., GLn that are different in wavelength from each other, in step S201, it is transmitted from the processor side CPU 401 to the spectral estimation processing unit 207. Spectral images GLi and GLj of the selected wavelengths (two types of wavelengths in this example) are extracted.

  Then, in the pseudo color image generation unit 405, any one of the three RGB colors is assigned to the extracted spectral images GLi and GLj in accordance with an assignment rule to be described later, and the pseudo color composed of the spectral image to which these colors are assigned. An image G3 is generated (step S203). Here, in the process of step S <b> 203, R color is assigned to the spectral image of the wavelength corresponding to the dye name set in the first name setting unit 411 of the dye name setting screen 410 and set in the second name setting unit 412. G color is assigned to the spectral image of the wavelength corresponding to the dye name, and B color is assigned to the spectral image of the wavelength corresponding to the dye name set by the third name setting unit 413.

  The pseudo color image G3 generated by the process of step S203 is displayed on the monitor 300 by the display control unit 209.

  FIG. 13 is a diagram illustrating an example of a color photographed image G1 in which a pigment is dispersed on a protruding lesion, and FIG. 14 is a pseudo color image generated from the color photographed image G1 in FIG. It is a figure which shows an example of G3.

  In the pseudo color image G3 in FIG. 14, unnecessary information other than the pigment distribution is discarded from the color photographed image G1 in FIG. 13, and the pigment distribution state is captured in a simplified manner. In other words, the part of the dent where the pigment is attached is darkly displayed in one of the colors of RGB, and the part where the pigment is less attached such as the protrusion is thinly displayed, so that the unevenness of the projection lesion can be grasped. It is easy to do.

  As described above, according to the second endoscope system 20 described above, in order to obtain an image in which the unevenness of a lesion can be easily grasped from a color photographed image, only the name of the dispersed pigment that is surely grasped by the user is set. Is enough. That is, according to the second endoscope system 20, it is possible to easily perform image processing for obtaining an image that makes it easy to grasp the unevenness of a lesion from a color captured image.

  In the above description, the color photographed image obtained by one photographing under illumination with white light is exemplified as the color photographed image. However, the color photographed image is not limited to this. May be obtained by synthesizing a plurality of monochrome photographed images obtained by photographing a plurality of times under illumination of each of a plurality of types of monochromatic light having different colors.

  In the above description, the spectral estimation unit 502 that performs spectral analysis on a color captured image and obtains a plurality of spectral images included in the color captured image has been exemplified. It is not limited. In this acquisition method, when a color photographed image is obtained by combining a plurality of monochrome photographed images as described above, a wavelength set by the user among the plurality of monochrome photographed images is obtained. A captured image obtained under monochromatic light may be obtained as a spectral image.

  Moreover, although the example in which the number of pigment names is set to 1 or more and 3 or less has been described above, the number of pigment names set is not limited to this, for example, 1 to 2 or less, or The upper limit number may be a number other than three, such as one or more and four or less.

  In the above, an example in which any one of the three RGB colors is assigned to the spectral image is shown. However, the color assigned to the spectral image is not limited to this. For example, C (cyan), M (magenta) , G (green) may be any color other than RGB, such as any one of the three colors.

  In the above description, a color still image is illustrated as an image processing target. However, the image processing target is not limited to this, and a color moving image may be used.

In the second endoscope system 20 described above, an embodiment of an image processing apparatus in which the processor 400 can easily perform image processing for obtaining an image that easily grasps unevenness of a lesion from a color captured image. And conceptually speaking,
“A color image acquisition unit that acquires a color image obtained by shooting a subject,
A component designation unit that receives one or more designations of spectral components of light;
A component image acquisition unit that obtains a component image of one or more spectral components specified by the component specification unit from the captured image acquired by the color image acquisition unit;
One of the plurality of reference colors in the display device that displays a color with a combination of a plurality of reference colors is predetermined for each of the component images of one or more spectral components acquired by the component image acquisition unit. The image processing apparatus includes a pseudo color image generation unit that obtains a pseudo color photographed image composed of component images of one or more spectral components to which the reference color is assigned. .

Here, the processor 400
“The color image acquisition unit acquires a color captured image obtained by one imaging,
An apparatus that also satisfies the concept that the component image acquisition unit obtains a component image of one or more spectral components specified by the component specifying unit by performing spectral analysis on the captured image. is there.

In addition, the processor 400
“A component index indicating a spectral component of light, and a plurality of component indexes indicating different spectral components and a plurality of selection items selected by a predetermined selection operation are associated with each other on a one-to-one basis. A correspondence storage unit for storing a correspondence table;
An operation receiving unit that receives the selection operation and specifies, in the component specifying unit, a spectrum component indicated by a component index associated with the selection item selected by the selection operation; It is a device that also fulfills the concept of “some”.

Furthermore, the processor 400
“The correspondence storage unit associates the plurality of component indicators with a plurality of pigment names indicating pigments for coloring the subject, and a plurality of pigment names indicating different pigments one to one. It also satisfies the concept of “memorizing correspondence table”.

In addition, the second endoscope system 20 is conceptually speaking:
"Light source that emits light;
An optical probe equipped with an imager that captures a color image by photographing a subject;
A color image acquisition unit for acquiring a color captured image obtained by the imaging device;
A component designation unit that receives one or more designations of spectral components of light;
A component image acquisition unit that obtains a component image of one or more spectral components specified by the component specification unit from the captured image acquired by the color image acquisition unit;
One of the plurality of reference colors in the display device that displays a color with a combination of a plurality of reference colors is predetermined for each of the component images of one or more spectral components acquired by the component image acquisition unit. An image processing apparatus comprising: a pseudo color image generation unit that obtains a pseudo color photographed image including component images of one or more spectral components to which the reference color is assigned by assigning according to the assignment rule; The endoscope system includes a display device that displays a photographed image that has been subjected to image processing by the device.

  Needless to say, the second endoscope system 20 is a system that satisfies the concepts that the processor 400 satisfies.

  Next, a third endoscope system will be described.

  The third endoscope system is an image processing that is realized in the endoscope system 10 shown in FIG. 1 and that makes it easy to visually grasp a desired location in a captured image obtained by the endoscope system. This is achieved by a method different from that of the endoscope system 10 shown in FIG.

  FIG. 15 is a diagram showing a third endoscope system.

  In FIG. 15, the same components as those of the endoscope system 10 shown in FIG. 1 are denoted by the same reference numerals as those in FIG. Omitted.

  The third endoscope system 20 shown in FIG. 15 has an emphasis display function different from the emphasis display function of the endoscope system 10 shown in FIG. Different from the system 10. Hereinafter, the description will be made by paying attention to the highlighting function of the third endoscope system 20.

  The processor 500 of the third endoscope system 30 in FIG. 15 performs spectrum analysis on the color photographed image, thereby obtaining a plurality of spectral images included in the photographed image over a predetermined wavelength range. A spectral estimation processing unit 502 that performs spectral estimation processing obtained uniformly is provided. Further, the processor 500 extracts one spectral image corresponding to a designation operation described later from the plurality of spectral images obtained by the spectral estimation processing unit 502, performs enhancement processing on the spectral image, and performs processing after the processing. A second image processing unit 503 for synthesizing the spectral image with another spectral image is provided. By the processing in the second image processing unit 503, an enhanced image in which a desired spectral component included in the color photographed image is enhanced is obtained. This enhanced image is displayed on the monitor 300 by the display control unit 209.

  Hereinafter, a series of processing from when the display on the monitor 300 is instructed to be switched from the normal image display to the highlighted image display by the operation of the switching button 203 until the highlighted image is actually displayed on the monitor 300 will be described. explain.

  In the following embodiment, processing for a color still image will be described. However, the image processing here can be applied to a moving image, and is not limited to a still image.

  The switch button 203 is pressed while a normal moving image is displayed on the monitor 300, and the switch button 203 is pressed after a still image is captured at that time or when a still image is displayed on the monitor 300. After that, an image enhancement parameter set screen for setting various parameters in enhancement processing for the still image is displayed on the monitor 300. In the third endoscope system 30, as in the endoscope system 10 of FIG. 1, a parameter / autoset mode for automatically setting various parameters in the emphasis process and various parameters are manually set individually. There are parameters to set and manual set mode.

  FIG. 16 is a diagram showing an image enhancement parameter set screen.

  In the image enhancement parameter set screen 510 of FIG. 16, the parameter / auto set mode is enabled and the parameter / manual set mode is disabled. On the contrary, the parameter / manual set mode is enabled, A second radio button 512 for disabling the parameter / auto set mode is provided.

  First, the parameter / manual set mode will be described.

  In this parameter / manual set mode, the wavelength corresponding to the spectral image to be extracted from the plurality of spectral images included in the color photographed image and the degree of enhancement in each of the three types of enhancement processing applied to the spectral image. Set. Note that the parameter / manual set mode on the image enhancement parameter set screen 510 in FIG. 16 differs from the parameter / manual set mode on the image enhancement parameter set screen 220 in FIG. Only the wavelength.

  The image enhancement parameter set screen 510 in FIG. 16 includes a wavelength setting unit 513, and the user clicks the increase / decrease button 513a in the wavelength setting unit 513 with a mouse (not shown), or the display unit 513b does not display it. A desired wavelength can be set by directly inputting a numerical value using the illustrated keyboard.

  The image enhancement parameter set screen 510 includes a sharpness enhancement setting unit 514 that sets the degree of sharpness enhancement, a contrast enhancement setting unit 515 that sets the degree of contrast enhancement, and a color enhancement setting that sets the degree of color enhancement. Part 516. The degree of sharpness enhancement is set to a value of “0” or more by a click operation on the increase / decrease button 514a of the sharpness enhancement setting unit 514 or a keyboard input to the display unit 514b. Similarly, the degree of contrast enhancement is set by a click operation on the increase / decrease button 515a of the contrast enhancement setting unit 515 or a keyboard input on the display unit 515b, and the degree of color enhancement is clicked on the increase / decrease button 516a of the color enhancement setting unit 516. And is set by keyboard input to the display unit 516b. Here, when the value “0” is set, the enhancement processing is not executed. FIG. 16 is an example in which only color enhancement is executed.

  Next, the parameter autoset mode will be described.

  In the third endoscope system 30, a spectral image having a wavelength corresponding to each candidate is extracted to the names of a plurality of types of enhancement candidates such as a red spot, a pigment name (for example, indigo carmine), a blood vessel, and the like. A correspondence table in which parameters required for spectral image enhancement are associated one-to-one is stored in the memory 210 of FIG.

  The image enhancement parameter set screen 510 shown in FIG. 16 is provided with a name set unit 517 in which names of enhancement candidates such as red spots are set in the parameter / auto set mode. When the menu button 517a of the name setting unit 517 is clicked, a pull-down menu in which names of emphasis candidates that can be set are listed is displayed. When the user moves the cursor to a desired name, the name is set. The set name is displayed on the display unit 517b. When the name of the emphasis candidate is set by the name setting unit 517, each parameter associated with the name is read from the memory 210 by the processor side CPU 501 in FIG.

  Further, the image enhancement parameter set screen 510 of FIG. 16 is provided with a determination button 518 for approving the set contents in the parameter / manual set mode or the parameter / auto set mode. When the determination button 518 is clicked after the parameter setting is completed, processing using the set parameters is started in the spectral estimation processing unit 502 and the specific processing unit 504 in FIG.

  Next, the flow of processing using these parameters will be described.

  FIG. 17 is a schematic diagram showing the flow of processing until the emphasized image is displayed on the monitor 300 after setting the parameters.

  A color photographed image G1 represented by digital image data representing a color with a combination of RGB three colors is subjected to spectral estimation processing (step S301) in the spectral estimation processing unit 502. In the spectral estimation process, a plurality of spectral images GL1, GL2,... With different wavelengths from each other over a predetermined wavelength range from a color photographed image G1 that includes a plurality of spectral images with different wavelengths. GLn is obtained evenly. All of the plurality of spectral images GL1, GL2,... GLn are passed from the spectral estimation processing unit 502 to the second image processing unit 503.

  Next, in the second image processing unit 503, the spectral image GLi having the wavelength set on the image enhancement parameter set screen 510 in FIG. 16 is extracted from the plurality of spectral images GL1, GL2,. The spectral image GLi is subjected to the enhancement process set on the image enhancement parameter set screen 510 (step S302). Further, integration processing is performed to synthesize the spectral image GLi 'after the enhancement processing with the other spectral images GL1,..., GLi-1, GLi + 1,.

  Hereinafter, this integration process will be described.

  In this integration process, finally, RGB3 representing an enhanced image G4 obtained by synthesizing all the spectral images GL1,..., GLi-1, GLi ′, GLi + 1,. Digital image data in which colors are expressed by colors is obtained.

  Here, in the integration process, first, component values of pixels whose positions correspond to each other in all the spectral images GL1,..., GLi−1, GLi ′, GLi + 1,. Based on the above, the spectral characteristic of each pixel in the enhanced image G4 is acquired.

  FIG. 18 is a schematic diagram illustrating how the spectral characteristics of each pixel in the enhanced image G4 are acquired.

  In FIG. 18, from a spectral image GL1,..., GLi−1, GLi ′, GLi + 1,... GLn including a spectral image after enhancement processing schematically shown on the left side, certain coordinates ( A state in which the spectral characteristic S of the pixel of x, y) is acquired is shown.

  The spectral characteristic S of the pixel at the coordinate (x, y) is obtained by plotting the pixel value of the pixel at the coordinate (x, y) of each spectral image in association with the wavelength corresponding to each spectral image.

  When the spectral characteristic S shown in FIG. 18 is obtained, next, enhancement is performed based on the spectral characteristic S and the spectral sensitivity characteristic in the CCD 120 included in the scope 100 of the third endoscope system 30 in FIG. A pixel value (R, G, B) of each pixel of the image G4 is calculated.

  FIG. 19 is a diagram showing spectral sensitivity characteristics in the CCD 120.

  The CCD 120 has a plurality of light receiving elements that output RGB pixel values in order to express colors in RGB three colors. The three curves shown in FIG. 19 indicate the spectral sensitivity characteristic Lr of the R light receiving element of the CCD 120, the spectral sensitivity characteristic Lg of the G light receiving element, and the spectral sensitivity characteristic Lb of the B light receiving element, respectively. These spectral sensitivity characteristics are stored in advance in the memory 210 of FIG. 15 at the manufacturer.

  FIG. 20 is a schematic diagram showing how pixel values (R, G, B) of each pixel are calculated from the spectral characteristic S of each pixel in the enhanced image G4 and the spectral sensitivity characteristic of the light receiving element.

  In FIG. 20, as an example of the calculation of the pixel value (R, G, B), a state in which the B value is calculated among the pixel values of the pixel at a certain coordinate (x, y) is shown.

  The B value is calculated by performing integral calculation on the product of the spectral characteristic S for the pixel at the coordinates (x, y) and the spectral sensitivity characteristic Lb of the B light receiving element. Here, the enhancement processing in step S302 of FIG. 17 is reflected in the spectral characteristics S of the pixels. For this reason, this B value is different from the B value at the pixel of the coordinates (x, y) in the original color photographed image G1, and the spectral component for the wavelength set on the image enhancement parameter set screen 510 in FIG. Is an emphasized value. By performing the same integration calculation for the R value and the G value, the pixel value (R, G, B) in the pixel at this coordinate (x, y) is obtained.

  In the integration process in step S303 of FIG. 17, the integration calculation described with reference to FIGS. 18 to 20 is performed for all the pixels, and digital image data representing the enhanced image G4 is obtained. Then, the enhanced image G4 represented by the digital image data obtained by the integration process is displayed on the monitor 300 by the display control unit 209. According to the emphasized image G4, for example, a spectral component having a wavelength corresponding to a spot to be discovered, such as a red spot, is displayed with its color etc. emphasized. As a result, the desired part is emphasized more than the surroundings, and the doctor can easily grasp the part to be found in the emphasized image G4 visually.

  As described above, according to the third endoscope system 30 shown in FIG. 15, it is possible to perform image processing that makes it easy to visually grasp a spot to be found on a captured image.

  In the above description, the color photographed image obtained by one photographing under illumination with white light is exemplified as the color photographed image. However, the color photographed image is not limited to this. May be obtained by synthesizing a plurality of monochrome photographed images obtained by photographing a plurality of times under illumination of each of a plurality of types of monochromatic light having different colors.

  In the above description, the spectral estimation unit 502 that performs spectral analysis on a color captured image and obtains a plurality of spectral images included in the color captured image has been exemplified. It is not limited. In this acquisition method, when a color photographed image is obtained by combining a plurality of monochrome photographed images as described above, a wavelength set by the user among the plurality of monochrome photographed images is obtained. A captured image obtained under monochromatic light may be obtained as a spectral image.

  In the above description, three types of enhancement processing such as sharpness enhancement, contrast enhancement, and color enhancement are illustrated as an example of the enhancement processing. Emphasis processing other than types may be used.

  In the above description, a color still image is illustrated as an image processing target. However, the image processing target is not limited to this, and a color moving image may be used.

The third endoscope system 30 described above corresponds to an embodiment of an image processing apparatus in which the processor 500 can perform image processing that makes it easy to visually grasp a desired location. In a nutshell,
“A color image acquisition unit that acquires a color image obtained by shooting a subject,
A component designating unit for designating a spectral component of light;
The image processing apparatus includes an enhancement processing unit that performs image processing that emphasizes or de-emphasizes the component image of the spectral component specified by the component specifying unit from the captured image acquired by the color image acquisition unit. Yes.

Here, the processor 500
“The color image acquisition unit acquires a color captured image obtained by one imaging,
An apparatus that also satisfies the concept that the enhancement processing unit extracts a component image of one or more spectral components specified by the component specifying unit by performing spectral analysis on the captured image. is there.

Further, the processor 500
“A component index indicating a spectral component of light, and a plurality of component indexes indicating different spectral components and a plurality of selection items selected by a predetermined selection operation are associated with each other on a one-to-one basis. A correspondence storage unit for storing a correspondence table;
An operation receiving unit that receives the selection operation and specifies a spectrum component indicated by a component index associated with the selection item selected by the selection operation in the component specifying unit; It is a device that also fulfills the concept of “some”.

In addition, the above-described third endoscope system 30 is conceptually speaking:
"Light source that emits light;
An optical probe equipped with an imager that captures a color image by photographing a subject;
A color image acquisition unit for acquiring a color captured image obtained by the imaging device;
A component designating unit for designating a spectral component of light;
An image processing apparatus comprising: an enhancement processing unit that performs image processing for enhancing or de-emphasizing a component image of a spectral component designated by the component designation unit from a captured image obtained by the color image obtaining unit; The endoscope system includes a display device that displays a captured image that has been subjected to image processing by the processing device.

  Needless to say, the third endoscope system 30 is a system that satisfies the concepts that the processor 500 satisfies.

  Next, a fourth endoscope system will be described.

  For example, in the technique described in Patent Document 2 described above, a process of performing spectrum analysis on a captured image obtained by an endoscope system and extracting a desired spectral image is performed. Which spectral image is to be extracted? Is determined by designation of the wavelength by the user operation.

  However, for inexperienced users, it may not be clear what wavelength is specified to obtain an image suitable for diagnosis. In such a case, the technique described in Patent Document 2 uses the wavelength. There is a problem that the image processing cannot be performed in the first place. The fourth endoscope system described below solves such a problem, and even an inexperienced user can easily perform image processing for obtaining an image suitable for diagnosis from a color photographed image. It is equipped with a device.

  FIG. 21 is a diagram showing a fourth endoscope system.

  In FIG. 21, the same components as those of the endoscope system 10 shown in FIG. 1 are denoted by the same reference numerals as those in FIG. Omitted.

  The fourth endoscope system 40 shown in FIG. 21 detects the peak in the spectrum of light for each pixel in the color photographed image, instead of the highlighting function in the endoscope system 10 of FIG. 1 is different from the endoscope system 10 shown in FIG. 1 in that it has a peak color display function for generating and displaying a peak color image in which the pixel color is replaced with a color corresponding to the detected peak. Yes. Hereinafter, the description will be made with attention paid to the peak color display function.

  The processor 600 of the second endoscope system 20 in FIG. 21 displays an image to be displayed on the monitor 300 as a color captured image (normal image) displayed with the basic display function and a peak displayed with the peak color display function. A switch button 602 that accepts a switching operation from a user that switches between color images, digital image data is acquired from the scope 100, and the digital image data is displayed in basic display under the control of the processor-side CPU 601 according to the switching operation. And a switching circuit 603 used for either the function or the peak color display function.

  Further, the processor 600 performs spectral analysis on the color captured image, thereby performing spectral estimation processing for uniformly obtaining a plurality of spectral images included in the captured image over a predetermined wavelength range. Part 604 is provided. Further, the processor 600 acquires the spectral characteristics of each pixel in the color photographed image based on the component values of the pixels whose positions correspond to each other among the plurality of spectral images, and detects a peak in the spectral characteristics of each pixel. And a peak color image generation unit 606 that obtains a peak color image by replacing the color of each pixel in the color photographed image with a color corresponding to the peak detected by the peak wavelength detection unit 605. ing. The peak color image generated by the peak color image generation unit 405 is displayed on the monitor 300 by the display control unit 209.

  Hereinafter, a series of operations from when the display on the monitor 300 is instructed to be switched from the normal image display to the peak color image display by the operation of the switch button 602 until the pseudo color image is actually displayed on the monitor 300 will be described. Processing will be described.

  In the following embodiment, processing for a color still image will be described. However, the image processing here can be applied to a moving image, and is not limited to a still image.

  The switch button 602 is pressed while a normal moving image is displayed on the monitor 300, and the switch button 602 is pressed after a still image is captured at that time or when a still image is displayed on the monitor 300. After that, the process described below is started.

  FIG. 22 is a schematic diagram showing the flow of processing until the peak color image is displayed on the monitor 300 after the switch button 602 is operated.

  A color photographed image G1 represented by digital image data representing colors by a combination of RGB three colors is subjected to spectral estimation processing (step S401) in the spectral estimation processing unit 604. In the spectral estimation process, a plurality of spectral images GL1, GL2,... With different wavelengths from each other over a predetermined wavelength range from a color photographed image G1 that includes a plurality of spectral images with different wavelengths. GLn is obtained evenly. All of the plurality of spectral images GL1, GL2,..., GLn are transferred from the spectral estimation processing unit 604 to the peak wavelength detection unit 605.

  The peak wavelength detection unit 605 acquires the light spectrum of each pixel in the color captured image G1 based on the component values of the pixels whose positions correspond to each other among the plurality of spectral images GL1, GL2,. A wavelength (peak wavelength) indicating a peak in the spectrum of each pixel is detected (step S402). Based on the detection, the peak wavelength detection unit 605 generates wavelength information J2 indicating the peak wavelength for each pixel of the color captured image G1, and passes the wavelength information J2 to the peak color image generation unit 606.

  In the peak color image generation unit 606, RGB pixel values representing the color represented by the peak wavelength of each pixel indicated by the wavelength information J2 are calculated, and the RGB three color pixel values of each pixel of the color photographed image G1 are The calculated RGB pixel values are replaced (step S403). By this replacement, a peak color image G5 in which the color of each pixel is replaced with the color represented by the peak wavelength is generated, and this peak color image G5 is displayed on the monitor 300 by the display control unit 209.

  For example, when a red spot is reflected in the color photographed image G1, the spectral characteristic of the pixel at the red spot has a peak reflecting the reflection characteristic of hemoglobin concentrated at the red spot. In addition, when a colored portion by a pigment is reflected in the color photographed image G1, the light spectrum at the pixel of the colored portion has a peak reflecting the reflection characteristics of the pigment. In the peak color image G5, since the color of each pixel is the color of the peak wavelength in the light spectrum of the pixel, points to be found in diagnosis, such as red spots and colored lesions, are emphasized. The image is suitable for diagnosis. In the fourth endoscopic system 40 shown in FIG. 21, the user only needs to press the switch button 602 to obtain the peak color image G5 suitable for this diagnosis. Can easily obtain the peak color image G5.

  As described above, according to the fourth endoscope system 40 shown in FIG. 21, even an inexperienced user can easily perform image processing for obtaining an image suitable for diagnosis from a color photographed image. .

  In the above description, the color photographed image obtained by one photographing under illumination with white light is exemplified as the color photographed image. However, the color photographed image is not limited to this. May be obtained by synthesizing a plurality of monochrome photographed images obtained by photographing a plurality of times under illumination of each of a plurality of types of monochromatic light having different colors.

  In the above description, the spectral estimation unit 502 that performs spectral analysis on a color captured image and obtains a plurality of spectral images included in the color captured image has been exemplified. It is not limited. In this acquisition method, when a color photographed image is obtained by combining a plurality of monochrome photographed images as described above, a wavelength set by the user among the plurality of monochrome photographed images is obtained. A captured image obtained under monochromatic light may be obtained as a spectral image.

  In the above description, a color still image is illustrated as an image processing target. However, the image processing target is not limited to this, and a color moving image may be used.

The fourth endoscope system 40 described above corresponds to an embodiment of an image processing apparatus in which the processor 600 can easily perform image processing for obtaining an image suitable for diagnosis from a color captured image. Conceptually speaking,
“A color image acquisition unit that acquires a color image obtained by shooting a subject,
For each pixel in the captured image acquired by the color image acquisition unit, a component detection unit that detects a peak in the light spectrum;
The image processing apparatus includes a color replacement unit that replaces the color of each pixel in the captured image acquired by the color image acquisition unit with a color corresponding to the peak detected by the component detection unit. .

Here, the processor 600
“The color image acquisition unit acquires a color captured image obtained by one imaging,
The component detection unit also satisfies the concept that the peak is detected for each pixel in the captured image by performing spectrum analysis on the captured image.

The fourth endoscope system 40 is conceptually speaking:
"Light source that emits light;
An optical probe equipped with an imager that captures a color image by photographing a subject;
A color image acquisition unit for acquiring a color captured image obtained by the imaging device;
For each pixel in the captured image acquired by the color image acquisition unit, a component detection unit that detects a peak in the light spectrum;
An image processing apparatus comprising: a color replacement unit that replaces the color of each pixel in the captured image acquired by the color image acquisition unit with a color corresponding to the peak detected by the component detection unit; and the image processing The endoscope system includes a display device that displays a photographed image that has been subjected to image processing by the device.

  Needless to say, the fourth endoscope system 40 is a system that satisfies the concepts that the processor 600 satisfies.

DESCRIPTION OF SYMBOLS 10 Endoscope system 20 2nd endoscope system 30 3rd endoscope system 40 4th endoscope system 100 Scope 110 Light guide 111 Illumination window 120 CCD
130 Scope-side CPU
140 CCD drive circuit 150 CDS / AGC circuit 160 A / D conversion circuit 200, 400, 500, 600 processor 201 light source 202, 401, 501, 601 processor side CPU
203, 402, 602 Switching button 204, 403, 603 Switching circuit 205 First image processing unit 206, 503 Second image processing unit 207, 404, 502, 604 Spectral estimation processing unit 208 Specific processing unit 209 Display control unit 210 Memory 220 , 510 Image enhancement parameter set screen 221, 511 First radio button 222, 512 Second radio button 223, 513 Wavelength setting section 223 a, 224 a, 226 a, 227 a, 228 a, 513 a, 514 a, 515 a, 516 a Increase / decrease buttons 223 b, 224 b, 225b, 226b, 227b, 228b, 229b, 513b, 514b, 515b, 516b, 517a Display unit 224 Threshold setting unit 225 Shape setting unit 225a, 229a, 517a Menu buttons 226, 514 Loopness enhancement setting unit 227, 515 Contrast enhancement setting unit 228, 516 Color enhancement setting unit 229, 517 Name setting unit 230, 414, 518 Decision button 300 Monitor 405 Pseudo color image generation unit 410 Dye name setting screen 411 First name setting unit 411a, 412a, 413a Menu button 411b, 412b, 413b Display unit 412 Second name set unit 413 Third name set unit 605 Peak wavelength detection unit 606 Peak color image generation unit

Claims (3)

A color image acquisition unit that acquires a captured image of a color in which the color of each pixel is expressed by a combination of RGB three colors obtained by capturing an object using an input device having a predetermined spectral sensitivity distribution for each of the three RGB colors When,
A component designating unit that receives designation of the wavelength component of light;
An enhancement processing unit that performs image processing for emphasizing or de-emphasizing the wavelength component designated by the component designation unit on the captured image obtained by the color image obtaining unit;
The enhancement processing unit
The captured image is decomposed into a plurality of spectral images having different wavelengths from each other over a predetermined wavelength range,
Extracting a designated wavelength spectral image having a wavelength component designated by the component designation unit from the plurality of spectral images,
Applying emphasis or anti-enhancement processing to the specified wavelength spectral image,
Spectral distribution represented by a set of pixel values of pixels corresponding to each other in the plurality of spectral images using the spectral image after the enhancement or de-emphasis processing for the plurality of spectral images and the designated wavelength spectral image The image processing apparatus is characterized in that each of the pixels generates a processed image expressed by pixel values of RGB three colors by weighting and integrating with the spectral sensitivity distribution for each of RGB three colors. A component index indicating the wavelength component of light, a plurality of component indexes indicating different wavelength components, and a plurality of selection items that are selected by a predetermined selection operation in a one-to-one correspondence A correspondence storage unit for storing the table;
An operation receiving unit that receives the selection operation, and specifies a wavelength component indicated by the component index associated with the selection table in the correspondence table to the selection item selected by the selection operation; The image processing apparatus according to claim 1, wherein the image processing apparatus is provided. A light source that emits light;
An optical probe equipped with an imager that captures a color image by photographing a subject;
A color image acquisition unit that acquires a captured image of a color in which the color of each pixel is expressed by a combination of RGB three colors obtained by capturing an object using an input device having a predetermined spectral sensitivity distribution for each of the three RGB colors When,
A component designating unit that receives designation of the wavelength component of light;
An enhancement processing unit that performs image processing for emphasizing or de-emphasizing the wavelength component designated by the component designation unit on the captured image obtained by the color image obtaining unit;
The enhancement processing unit
The captured image is decomposed into a plurality of spectral images having different wavelengths from each other over a predetermined wavelength range,
Extracting a designated wavelength spectral image having a wavelength component designated by the component designation unit from the plurality of spectral images,
Applying emphasis or anti-enhancement processing to the specified wavelength spectral image,
Spectral distribution represented by a set of pixel values of pixels corresponding to each other in the plurality of spectral images using the spectral image after the enhancement or de-emphasis processing for the plurality of spectral images and the designated wavelength spectral image An image processing device that generates a processed image in which each pixel is expressed by a pixel value of RGB three colors by weighting and integrating the spectral sensitivity distribution for each of RGB three colors; and An endoscope system comprising a display device that displays a processed image after image processing has been performed.