JP6512320B2 - Imaging device - Google Patents

Description

  The present invention relates to an imaging apparatus which irradiates excitation light to a fluorescent dye injected into a subject and captures fluorescence generated from the fluorescent dye.

  In recent years, a technique called near infrared fluorescence imaging has been used for surgery. In this near-infrared fluorescence imaging, indocyanine green (ICG) as a fluorescent dye is injected into the affected area. When the indocyanine green is irradiated with near infrared light of 750 to 810 nm (nanometers) as excitation light, the indocyanine green emits fluorescence in the near infrared region having a peak of approximately 850 nm. This fluorescence is photographed by a camera capable of detecting near-infrared light, and the image is displayed on a display unit such as a liquid crystal display panel. According to this near-infrared fluorescence imaging, it becomes possible to observe blood vessels, lymph vessels and the like present at a depth of about 20 mm from the body surface.

  In Patent Document 1, the intensity distribution image of near-infrared fluorescence obtained by irradiating excitation light of indocyanine green to a test organ of a living body to which indocyanine green has been administered, and indocyanine green administration The cancer lesion distribution image obtained by applying X-rays, nuclear magnetic resonance or ultrasound to the previous test organ is compared with that of the cancer lesion distribution image and detected by the near-infrared fluorescence intensity distribution image, A data collection method is disclosed in which data of a region not detected in a cancer lesion distribution image is collected as cancer sub-lesion region data.

  Further, in Patent Document 2, excitation light and visible light are alternately irradiated to a subject to which a blood vessel contrast agent has been administered, and fluorescence images and visible images irradiated with excitation light by imaging means are alternately displayed. There is disclosed a surgery support method for acquiring a blood vessel image by performing threshold processing on a fluorescence image with a predetermined threshold to extract a blood vessel image, and creating a composite image in which the visible image and the extracted blood vessel image are superimposed.

International Publication No. 2009/139466 JP, 2009-226072, A

  As described in Patent Document 2, in the case of creating a composite image obtained by combining a visible image and a fluorescent image, the region of the contrast agent in the fluorescent image is expressed as lightness and thus becomes a black and white image; And the fluorescence image are synthesized, but it may be difficult to distinguish the visible image and the fluorescence image after the synthesis.

  That is, since the fluorescent image is displayed white on the visible image, its outline is not clear. In particular, the contour becomes unclear as the lightness difference decreases. For this reason, it is not possible to accurately determine how far the fluorescent region is in the fluorescent image. In such a case, there arises a problem that it is difficult to accurately perform the treatment for the patient.

  The present invention has been made to solve the above problems, and is an imaging apparatus capable of improving the visibility of a fluorescent image when the visible image of the subject and the fluorescent image are synthesized to create a synthesized image. Intended to provide.

  In the first invention, an excitation light source for irradiating the object with excitation light for exciting the fluorescent dye injected into the object, a visible light source for emitting visible light toward the object, and excitation light An imaging device comprising: a camera capable of detecting fluorescence and visible light for photographing fluorescence emitted from the fluorescent dye upon irradiation and visible light reflected on the surface of the subject; A dominant hue identification unit that identifies dominant hues in the visible image of the subject captured by the method; and a color hue identification unit that identifies different hues as colored hues with respect to the hues identified by the dominant hue identification unit. A fluorescent image coloring unit for coloring the fluorescent image of the subject photographed by the camera with the colored hue specified by the colored hue specifying unit; and the subject photographed by the camera By combining the body visible image and the fluorescence image colored fluorescence image colored with the coloring unit, characterized by comprising a synthesizing unit creating a synthesized image.

  In the second invention, the dominant hue specifying unit specifies a dominant hue using an image obtained by subtracting an image of a fluorescent region in the fluorescence image from the visible image.

  In the third invention, the dominant hue identifying unit identifies the most frequently appearing hue as the dominant hue by using a histogram.

  According to the first invention, when a visible image of a subject and a fluorescent image are combined to create a composite image, the fluorescent image and the visible image serving as the background can be clearly identified, and the fluorescent image is viewed It is possible to improve the quality.

  According to the second aspect of the invention, it is possible to appropriately identify the dominant hue in the background image by excluding the fluorescence image from the visible image.

  According to the third invention, it is possible to easily identify the dominant hue by using the histogram.

It is a schematic diagram of an imaging device concerning this invention. FIG. 2 is a perspective view of a light / shooting unit 12; It is a block diagram showing a main control system of an imaging device concerning this invention. It is a flowchart of the image synthetic | combination process which synthesize | combines a visible image and a near-infrared image by the imaging device which concerns on this invention, and produces a synthetic image.

  Hereinafter, embodiments of the present invention will be described based on the drawings. FIG. 1 is a schematic view of an imaging apparatus according to the present invention.

  This imaging apparatus includes an input unit 11 such as a touch panel, and includes a main unit 10 incorporating a control unit 30 described later, an illumination / photographing unit 12 movably supported by an arm 13, and a liquid crystal display panel etc. Display unit 14 and a treatment table 16 on which the patient 17 is placed. In addition, illumination / imaging part 12 is not limited to what was supported by arm 13, A surgeon may carry in a hand.

  FIG. 2 is a perspective view of the illumination and photographing unit 12 described above.

  The illumination / shooting unit 12 includes a camera 21 capable of detecting near infrared rays and visible light, a visible light source 22 composed of a large number of LEDs disposed on the outer periphery of the camera 21, and an outer periphery of the visible light source 22. And an excitation light source 23 composed of a large number of LEDs. The visible light source 22 emits visible light. The excitation light source 23 emits near-infrared light having a wavelength of 780 nm, which is excitation light for exciting indocyanine green.

  FIG. 3 is a block diagram showing a main control system of the imaging apparatus according to the present invention.

  The imaging apparatus comprises a CPU that executes logical operations, a ROM that stores an operation program required to control the apparatus, a RAM that temporarily stores data etc. during control, and a control unit that controls the entire apparatus. 30 is provided. The control unit 30 includes an image processing unit 31 that executes various image processing to be described later. The image processing unit 31 combines a visible image and a fluorescence image to create a composite image, a dominant hue identification unit 37 that identifies dominant hues, and a color hue identification that identifies colored hues. And a fluorescent image coloring unit 39 for coloring the fluorescent image with a coloring hue.

  The control unit 30 is connected to the input unit 11 and the display unit 14 described above. Further, the control unit 30 is connected to the illumination / photographing unit 12 provided with the camera 21, the visible light source 22 and the excitation light source 23. Furthermore, the control unit 30 is also connected to an image storage unit 33 that stores an image captured by the camera 21. The image storage unit 33 includes a near infrared image storage unit 34 for storing a near infrared image, and a visible image storage unit 35 for storing a visible image. Note that instead of including the near infrared image storage unit 34 and the visible image storage unit 35, a combined image storage unit that stores an image obtained by combining the visible image and the near infrared image may be provided.

  Hereinafter, the operation in the case of performing a surgical operation using the imaging device according to the present invention will be described. In the following description, a case of performing a breast cancer surgery on the patient 17 will be described.

  When a breast cancer surgery is performed using the imaging device according to the present invention, indocyanine green is injected by injection into the breast of the supine patient 17 on the treatment table 16. Then, the near-infrared light is emitted from the excitation light source 23 and the visible light is emitted from the visible light source 22 toward the subject including the affected area. As the near infrared light emitted from the light source 23 for excitation, as described above, the near infrared light of 780 nm which acts as excitation light for causing the indocyanine green to emit fluorescence is employed. As a result, indocyanine green emits fluorescence in the near infrared region with a peak of about 850 nm.

  Then, the vicinity of the affected area of the patient 17 is photographed by the camera 21. The camera 21 can detect near infrared light and visible light. The patient 17 is irradiated with visible light, and the image taken by the camera 21 becomes a visible image, and the patient 17 is irradiated with near infrared light, and the image from the indocyanine green taken by the camera 21 is near infrared It becomes an image. The near-infrared image and the visible image taken by the camera 21 are sent to the image processing unit 31 shown in FIG. The image processing unit 31 converts the near infrared image and the visible image into image data that can be displayed on the display unit 14. The data of the near infrared image is stored in the near infrared image storage unit 34 in the image storage unit 33. Further, data of the visible image is stored in the visible image storage unit 35 in the image storage unit 33.

  In this state, as described later, the combining unit 36 in the image processing unit 31 uses near infrared image data and visible image data to create a combined image in which a visible image and a near infrared image are fused. Do. Then, the image processing unit 31 displays the near-infrared image, the visible image, and the composite image on the display unit 14 simultaneously or selectively by dividing the region.

  Next, the operation of creating a composite image, which is a feature of the present invention, will be described. FIG. 4 is a flowchart of an image combining step of combining a visible image and a near infrared image by the imaging device according to the present invention to create a combined image.

  The image processing unit 31 in the control unit 30 shown in FIG. 3 specifies the dominant hue in the visible image by the dominant hue identification unit 37, and the R, G, B for the dominant hue by the colored hue identification unit 38. The hue whose value is the sum of the absolute values of the differences is equal to or greater than the set value and the luminance ratio is maximum is identified as the colored hue, and the fluorescent image coloring unit 39 identifies the near infrared image which is a fluorescent image. A composite image is created by combining the visible image and the near-infrared image colored by the fluorescent image coloring unit 39 by the combining unit 36 by coloring with the color hue specified by the unit 38.

  That is, the image processing unit 31 first subtracts the image of the region of indocyanine green, which is a fluorescent region in the near infrared image, from the visible image to create a visible image to be a background. Then, color extraction is performed on the visible image by the dominant hue specifying unit 37, and a dominant hue (hue) in the visible image obtained by subtracting (excluding) the fluorescent region is specified. Here, the dominant hue is also referred to as a dominant color and is a color that governs the impression of the entire image.

  This dominant hue is, for example, the hue that appears most often in the visible image. This dominant hue is identified by using a histogram for the visible image. At this time, in the peripheral area of the visible image, an image other than the affected area of the patient 17 may also be photographed. Therefore, a histogram may be applied to the image in the vicinity of the central part of the visible image. Note that the dominant hue may be calculated using gradation conversion, an average value, an intermediate value, and a mode value in an arbitrary color space.

  Next, the colored hue specifying unit 38 specifies a colored hue to which the near-infrared image showing indocyanine green should be colored. It is preferable that this colored hue is a hue that has high discrimination power with respect to the dominant hue in the previously identified visible image. Therefore, with respect to the dominant hue, this colored hue is such that the color difference ΔRGB, which is the sum of the absolute values of the differences between R, G, and B values, is equal to or greater than the set value, and The hue at which (ratio of luminance L) is maximum is selected.

  Here, the color difference ΔRGB described above is expressed by the following equation.

  ΔRGB = [max (Rb, Rf) -min (Rb, Rf)] + [max (Gb, Gf) -min (Gb, Gf)] + [max (Bb, Bf) -min (Bb, Bf)]

  Here, Rb, Gb, and Bb are RGB values of a visible image serving as a background, and Rf, Gf, and Bf are fluorescence images displayed on the visible image serving as a background (foreground). It is the RGB value of the outside image. The color difference ΔRGB, which is the sum of the absolute values of the differences among the R, G, and B values, is calculated by this equation.

  Further, the luminance L is expressed by the following equation. In addition to the following equation, a luminance calculation equation used for the color difference signal may be used.

  L = R1 × 0.2126 + G1 × 0.7152 + B1 × 0.0722

  Here, assuming that sR = R / 255, sB = B / 255, and sG = G / 255, the above R1 becomes [sR / 12.92] if sR is 0.03928 or less, and from 0.03928 If it is large, it becomes 2.4 to the power of [sR + 0.055) /1.055]. Similarly, G1 is [sG / 12.92] if sG is 0.03928 or less, and is 2.4 to the power of [sR + 0.055) /1.055] if it is greater than 0.03928. Further, B1 is [sB / 12.92] if sB is 0.03928 or less, and is 2.4 to the power of [sR + 0.055) /1.055] if it is larger than 0.03928.

  "Web content accessibility guidelines" by the World Wide Web Consortium (W3C), a standardization body established to promote standardization of various technologies used on the web, for the calculation of color difference ΔRGB and luminance L described above. WCAG) 1.0 and 2.0.

  Then, the fluorescence image coloring unit 39 in the image processing unit 31 colors the near infrared image, which is a fluorescence image, with the coloring hue specified by the coloring hue specifying unit 38. As the lightness (value) when coloring the infrared image at this time, the same lightness as the lightness of the near-infrared image which is a contrast image is adopted, and the maximum value is adopted as saturation. This makes it possible to more suitably recognize the fluorescent region of indocyanine green on the visible image.

  After that, the combining unit 36 in the image processing unit 31 combines the colored fluorescent image colored by the fluorescent image coloring unit 39 and the visible image by screen combining, additive combining, or the like to create a combined image.

  As described above, in the imaging apparatus according to the present invention, the color difference ΔRGB, which is the sum of the absolute values of the differences between R, G, and B values, becomes the set value or more as the coloring hue. In addition, since the hue having the largest luminance ratio with respect to the dominant hue is selected, it is possible to clearly distinguish the fluorescent region of indocyanine green from the visible image as the background. For this reason, it becomes possible to perform treatment etc. to patient 17 correctly.

  As the setting value for the color difference ΔRGB described above, for example, when the R, G and B values are 256 gradations each and the Hue is 360 gradation, 500 defined by the above-mentioned WCAG can be applied. At this time, ΔHue, which is the difference between the color hue and the dominant hue, is 118 to 242. However, it is possible to select a colored hue in which ΔHue is about 90 to 270 with respect to the dominant hue.

  In the embodiment described above, although the light source for emitting near infrared light having a wavelength of 780 nm is used as the light source for excitation 23, the light source for excitation 23 is to excite the indocyanine green to excite the excitation light. It is possible to use one that can emit near-infrared light of about 750 nm to 810 nm capable of generating

  Furthermore, in the embodiment described above, by using indocyanine green as a material containing a fluorescent dye, the indocyanine green is irradiated with near infrared light of 780 nm as excitation light, and the wavelength is approximately 850 nm from indocyanine green. In the case of emitting fluorescence in the near-infrared region with a peak as a peak, a light other than near-infrared light may be used.

  For example, 5-ALA (5-aminolevulinic acid / 5-aminolevulinic acid) can be used as a fluorescent dye. When this 5-ALA is used, 5-ALA which has invaded into the body of the patient 17 is converted to the fluorescent substance protoporphyrin (protoporphyrin IX / PpIX). When visible light of about 400 nm is irradiated toward the protoporphyrin, red visible light is emitted as fluorescence from the protoporphyrin. For this reason, when using 5-ALA, what is necessary is just to use what radiates | emits the visible light whose wavelength is about 400 nm as a light source for excitation.

DESCRIPTION OF SYMBOLS 10 main body 11 input part 12 illumination / imaging part 13 arm 14 display part 16 treatment stand 17 patient 21 camera 22 visible light source 23 excitation light source 30 control part 31 image processing part 33 image storage part 34 near-infrared image storage part 35 visible image Memory unit 36 Synthesizing unit 37 Dominating hue identifying unit 38 Colored hue identifying unit 39 Fluorescent image coloring unit

Claims (3)

An excitation light source for irradiating the subject with excitation light for exciting the fluorescent dye injected into the subject;
A visible light source that emits visible light toward the subject;
A camera capable of detecting fluorescence and visible light for photographing fluorescence generated from the fluorescent dye upon irradiation with excitation light and visible light reflected on the surface of the subject;
In an imaging device comprising
A dominant hue identification unit that identifies dominant hues in a visible image of the subject captured by the camera;
A colored hue identification unit that identifies, as a colored hue, a hue different from the hue identified in the dominant hue identification unit;
A fluorescent image coloring unit that colors the fluorescent image of the subject captured by the camera with the coloring hue specified by the coloring hue specification unit;
A combining unit configured to create a combined image by combining a visible image of the subject taken by the camera and a colored fluorescence image colored by the fluorescence image coloring unit;
An imaging apparatus comprising: In the imaging apparatus according to claim 1,
An imaging device that identifies a dominant hue by using an image obtained by subtracting an image of a fluorescent region in the fluorescence image from the visible image; In the imaging apparatus according to claim 1 or 2,
The dominant hue identification unit is an imaging device that identifies a hue that appears most frequently as a dominant hue using a histogram.

Images