JP2009039515A - Image processing system, image processing method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To correct blur of an object image when observing the object existing inside a substance using a reflected light from the object or a light emitted from a fluorescent substance such as ICG (Indocyanine Green). <P>SOLUTION: This image processing system is provided with: a specific wavelength image obtaining section that obtains a specific wavelength image being an image of light from an object existing inside a substance, the light belonging to a specific wavelength region; a depth calculator section that calculates a depth of the object from a surface of the substance, using a plurality of specific wavelength images corresponding to different specific wavelength regions from each other; and an object image generator that generates an image of the object according to the depth of the object calculated by the depth calculator section. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image processing system, an image processing method, and a program. In particular, the present invention relates to an image processing system, an image processing method, and a program that can clearly display an image of an object existing inside a substance that is difficult to view by surface observation, such as a blood vessel existing inside a living body.

For example, as described in Non-Patent Document 1, a technique is known in which indocyanine green (ICG) is administered to a living body and infrared laser light having a wavelength of 780 nm is irradiated to video-shoot blood perfusion inside the living body. Further, regarding the elimination of image blur in an endoscope, for example, in the endoscope described in Patent Document 1, an image is obtained using a point spread function of an objective optical system with respect to an image signal for a predetermined range. Apply recovery processing. Thereby, while ensuring a large amount of incident light to the imaging unit, it is said that blurring of an image due to insufficient depth of focus of the objective optical system can be eliminated.
“Measurement of blood perfusion of teat areola after total mammectomy with indocyanine green”, Saitama Medical University Journal, Vol. 32, No. 2, P45-50, April 2005 Japanese Patent Laid-Open No. 10-165365

  When an object existing inside a substance such as a blood vessel inside a living body is observed using reflected light from the object or light emission from a fluorescent substance such as ICG, there is a problem that the image of the object is blurred. That is, the reflected light or light emission is scattered while passing through the substance, and the outline of the object becomes unclear. When a doctor performs diagnosis or treatment using an endoscope, it is desirable to accurately recognize the position of an object such as a blood vessel. Therefore, it is desirable to correct a blur in the contour of the object and display a clear image. In addition, when imaging and displaying an object inside a substance, it is preferable that information about the depth of the object is displayed at the same time.

  In order to solve the above-described problem, in the first aspect of the present invention, a specific wavelength image that is an image of light from an object existing in a substance and belonging to a specific wavelength region is acquired. A wavelength image acquisition unit, a depth calculation unit that calculates the depth of the object from the material surface from a plurality of specific wavelength images with different specific wavelength regions, and an object corresponding to the object depth calculated by the depth calculation unit An image processing system including an object image correction unit that generates an image of

  The image processing apparatus further includes an object image acquisition unit that acquires an object image that is an image of light from an object existing inside the substance, and the object image generation unit converts the object image according to the depth of the object calculated by the depth calculation unit. You may have the object image correction part to correct | amend.

  It should be noted that the above summary of the invention does not enumerate all the necessary features of the present invention. In addition, a sub-combination of these feature groups can also be an invention.

  Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. In addition, not all the combinations of features described in the embodiments are essential for the solving means of the invention.

  FIG. 1 shows an example of an image processing system 10 of this embodiment together with an ICG injection unit 700 and a specimen 800. The image processing system 10 includes an endoscope 100, an image processing unit 200, a display unit 300, an imaging control unit 400, a light emitting unit 500, and a light emission control unit 600. In FIG. 1, the A part shows the distal end part 102 of the endoscope 100 in an enlarged manner.

  The ICG injection unit 700 injects indocyanine green (ICG), which is a luminescent substance, into the specimen 800. In addition, although ICG is illustrated as a luminescent substance in this embodiment, you may use fluorescent substances other than ICG as a luminescent substance. ICG is excited by infrared rays having a wavelength of 750 nm, for example, and emits a broad spectrum of fluorescence centering on 810 nm. When the specimen 800 is a living body, ICG can be injected into a blood vessel of the living body by intravenous injection. By imaging fluorescence from ICG, blood vessels in the living body can be imaged.

  The ICG injection unit 700 may be incorporated in the image processing system 10. By managing the ICG concentration in the living body, the ICG injection unit 700 can be controlled so that the ICG concentration in the living body is maintained constant.

  The specimen 800 may be a living body such as a human body, for example, and is an imaging target of an image processed by the image processing system 10. An object such as a blood vessel exists inside the specimen 800. The image processing system 10 according to the present embodiment corrects a blur in a captured image of an object existing deeper than the surface of the specimen 800 (including the inner surface of an organ or the like).

  The endoscope 100 includes an imaging unit 110, a light guide 120, and a forceps port 130. The distal end portion 102 of the endoscope 100 has an objective lens 112 as a part of the imaging unit 110. The distal end portion 102 has an IR light guide 122, an R light guide 124, a G light guide 126, and a B light guide 128 as a part of the light guide 120. Further, the tip portion 102 has a nozzle 140. A forceps 135 is inserted into the forceps opening 130.

  The imaging unit 110 captures images of reflected light in which light emitted from the luminescent material and reflected light reflected by the object are reflected. The imaging unit 110 may include, for example, a two-dimensional imaging device such as a CCD and an optical system, and the optical system includes an objective lens 112. When the light emitted from the luminescent material is infrared, the imaging unit 110 can capture an infrared image. When the irradiation light to the object is, for example, white light including RGB components, the imaging unit 110 can capture the component light images of the R component, the G component, and the B component.

  The component light of the infrared ray, R component, G component, and B component by the imaging unit 110 can be obtained by, for example, separating incident light with a spectral filter and imaging the filtered light with one CCD. In this case, each component light image can be picked up by switching the spectral filter arranged on the optical axis. Or incident light can be divided | segmented with a half mirror etc., and a spectral filter can be arrange | positioned for every divided | segmented optical axis. If a plurality of CCDs are arranged for each optical axis, each component light image can be taken simultaneously. Alternatively, if each infrared filter, R component, G component, and B component spectral filter is arranged for each pixel of one CCD, each component light can be simultaneously imaged by one CCD.

  The light guide 120 can be composed of, for example, an optical fiber. The light guide 120 guides light generated by the light emitting unit 500 to the distal end portion 102 of the endoscope 100. The light guide 120 can include an IR light guide 122, an R light guide 124, a G light guide 126 and a B light guide 128. The IR light guide 122 guides infrared rays, and the R light guide 124 guides R component light. The G light guide 126 guides the G component light, and the B light guide 128 guides the B component light.

  The light guide 120 includes the IR light guide 122, the R light guide 124, the G light guide 126, and the B light guide 128 for each of the component light of the infrared ray, the R component, the G component, and the B component. . When irradiating each component light separately, the imaging unit 110 does not need to include a spectral filter, and only one CCD may be used by imaging in accordance with the timing of irradiating each component light. Of course, in the case where the imaging unit 110 has a function for spectrally imaging each component light, it is not necessary to irradiate each component light in a time-sharing manner, and the light guide 120 may be integrated into one light guide. .

  The forceps port 130 guides the forceps 135 to the distal end portion 102. In FIG. 1, illustration of the tip shape of the forceps 135 is omitted. Forceps 135 having various tip shapes can be applied. The nozzle 140 can send out water or air.

  The image processing unit 200 processes the image data acquired from the imaging unit 110. The display unit 300 displays the image data processed by the image processing unit 200. The imaging control unit 400 controls imaging by the imaging unit 110.

  The light emitting unit 500 generates light emitted from the distal end portion 102 of the endoscope 100. The light generated in the light emitting unit 500 includes infrared light that is excitation light that excites the luminescent substance, and irradiation light that irradiates an object inside the specimen 800. The irradiation light includes, for example, component light of R component, G component, and B component.

  The light emission control unit 600 controls the light emission unit 500 under the control of the imaging control unit 400. For example, when the imaging unit 110 captures each component light of an infrared ray, an R component, a G component, and a B component in time division, it can be controlled to synchronize the irradiation timing of each component light and the imaging timing.

  FIG. 2 shows a configuration example of the image processing unit 200. The image processing unit 200 includes an object image acquisition unit 210, a specific wavelength image acquisition unit 212, a surface image acquisition unit 214, an object region specification unit 216, a depth calculation unit 218, an object image generation unit 230, a correction table 222, an interpolation image. A generation unit 224 and a display control unit 226 are included.

  The object image acquisition unit 210 acquires an object image that is an image of light from an object such as a blood vessel that exists inside the specimen 800 that is a substance. Examples of the light from the object include fluorescence emitted by a luminescent substance present in the object, or reflected light of light irradiated on the object. Note that the object image acquisition unit 210 may acquire an object image that is an image of scattered light in which light from the object is scattered from the object to the surface of the substance, as will be described later.

  When using fluorescence emitted from a luminescent substance as light from an object, the object image acquired by the object image acquisition unit 210 is an object that exists in a range from the surface of the substance to a depth at which excitation light for exciting the luminescent substance can enter. Is included. That is, the excitation light of the luminescent material irradiated from the distal end portion 102 of the endoscope 100 has a wavelength of, for example, 750 nm, and therefore reaches a relatively deep portion (for example, about 1 cm) of the specimen 800. Therefore, the object image acquired by the object image acquisition unit 210 includes a blood vessel image that exists relatively deep in the specimen 800. Of course, since the luminescent substance existing in the depth range where the excitation light can reach is excited by the excitation light, the object image acquired by the object image acquisition unit 210 is within the depth range where the excitation light can reach. Contains existing blood vessel images. In addition, since the fluorescence from a blood vessel is scattered by the specimen 800 as the blood vessel is present at a deeper position, the blood vessel image at a deeper position includes more blur.

  When the reflected light from the object is used as the light from the object, the object image acquired by the object image acquisition unit 210 includes an object that exists within a depth range where the irradiation light enters the object and is reflected. . Since the penetration depth of the irradiation light into the material depends on the wavelength of the irradiation light, red light can penetrate deeper into the material than blue light, and green light can penetrate to their intermediate depth. Infrared rays can penetrate deeper into the material than red light. Therefore, an object existing in the range up to the depth of penetration reflecting the wavelength range of light irradiated from the material surface is included in the object image.

  The specific wavelength image acquisition unit 212 acquires a specific wavelength image that is an image of light from an object such as a blood vessel and belonging to a specific wavelength region. The specific wavelength region may be any wavelength region, and examples thereof include a red region centered on the R component of visible light, a green region centered on the G component, and a blue region centered on the B component. The wavelength region to which the fluorescence from ICG belongs may be divided into a plurality of wavelengths. For example, among the wavelength regions in the fluorescence from ICG, a long wavelength region, a middle wavelength region, and a short wavelength region can be exemplified.

  The specific wavelength images using these specific wavelength regions use the property that the light penetration depth into the specimen 800 is different if the wavelengths are different, or the light absorption in the specimen 800 if the wavelengths are different. It can be used to detect the depth of an object such as a blood vessel by utilizing the property of different rates. That is, it can be determined that the blood vessel that can be identified by the specific wavelength image of the blue region exists in the depth region where the light of the wavelength belonging to the blue region is reflected. Similarly, a blood vessel that can be identified by a specific wavelength image in the green region can be determined to exist in a depth region where light having a wavelength belonging to the green region is reflected, and a blood vessel that can be identified by a specific wavelength image in the red region is It can be determined that the light exists in a depth region where light having a wavelength belonging to is reflected. The depth at which blue light is reflected is shallower than the depth at which red light is reflected, and the depth at which green light is reflected is located between these two.

  Further, among the fluorescence emitted by ICG in the blood vessel, the absorption of light belonging to the long wavelength region is smaller than the absorption of light belonging to the short wavelength region, so that the specific wavelength image of the long wavelength region, medium wavelength region or short wavelength region The depth of the blood vessel can be estimated from the brightness ratio of the blood vessel image included in the blood vessel. For example, if the blood vessel image related to the specific wavelength image in the short wavelength region is darker than the brightness of the blood vessel image related to the specific wavelength image in the long wavelength region, it can be estimated that the blood vessel exists at a deep position. Conversely, if the blood vessel image related to the specific wavelength image in the short wavelength region is brighter than the brightness of the blood vessel image related to the specific wavelength image in the long wavelength region, the path of light that will absorb light in the short wavelength region is It is not long and it can be estimated that the blood vessel exists in a shallow position.

  When detecting the depth of an object such as a blood vessel using the difference in the depth of penetration into the substance (the depth of penetration of reflected light) due to the wavelength of light as described above, the specific wavelength image acquisition unit 212 reflects from the object. A reflected light image that is an image of the reflected light thus obtained is acquired as a specific wavelength image. The specific wavelength image acquisition unit 212 may acquire, as the specific wavelength image, a reflected light image that is an image of reflected light belonging to the specific wavelength region among the reflected light reflected from the object by irradiating the object with white light. . Or the specific wavelength image acquisition part 212 may acquire the reflected light image which is an image of the reflected light which irradiated the irradiation light which belongs to a specific wavelength area | region, and was reflected from the object as a specific wavelength image. In addition, when detecting the depth of an object such as a blood vessel using the difference in absorption rate depending on the wavelength of the fluorescence emitted in the deep part of the substance, the specific wavelength image acquisition unit 212 uses the specific wavelength of the light emitted from the luminescent substance. A light emission image that is an image of light belonging to the region is acquired as a specific wavelength image.

  The surface image acquisition unit 214 acquires a surface image that is an image of the surface of the substance. That is, the surface image acquisition unit 214 acquires an equivalent image when visually recognized. The object area specifying unit 216 specifies the image area of the object for each of the plurality of specific wavelength images. That is, the object area specifying unit 216 specifies an image area of an object such as a blood vessel included in the specific wavelength image, and specifies a blood vessel image or the like.

  The depth calculation unit 218 calculates the depth from the surface of the substance of the object from a plurality of specific wavelength images having different specific wavelength regions. As described above, since the specific wavelength image includes the information on the light penetration depth, the depth of the object can be calculated by comparing or calculating the brightness (luminance) of the object included in the specific wavelength image.

  For example, the depth calculation unit 218 can calculate the depth based on the luminance in the image region specified by the object region specification unit 216. For example, the depth calculation unit 218 can calculate the depth based on the ratio of the luminance of the image region in the specific wavelength region in the short wavelength region to the luminance of the image region in the specific wavelength region in the long wavelength region. For example, the depth calculation unit 218 can calculate the depth based on the maximum luminance or the average luminance in the image area. For example, the depth calculator 218 can calculate the depth based on the luminance change rate at the edge of the image area.

  The depth calculation unit 218 may store the depth in advance in association with the amount of ICG existing in the blood vessel and the luminance of the image of the blood vessel. The depth calculation unit 218 calculates the amount of ICG present in the specimen 800 based on the amount of ICG injected from the ICG injection unit 700, the value corresponding to the calculated amount of ICG, and the object region specifying unit The depth stored in advance in association with the value corresponding to the luminance in the image region specified by 216 may be calculated as the blood vessel depth.

  The rate of change in luminance at the edge of the image area can be expressed, for example, by a luminance differential coefficient with the position (distance) in the image space as a variable. The differential coefficient is an example in which the size of the blur of the object related to the image area is digitized, and it can be estimated that the larger the differential coefficient, the smaller the blur and the object exists at a shallower position.

  As described above, the specific wavelength image acquisition unit 212 obtains a specific wavelength image that is light from an object and is an image of scattered light that is scattered between the surface of the substance and light belonging to the specific wavelength region. get. Then, the depth calculation unit 218 may calculate the depth based on the luminance in the image region specified by the object region specification unit 216 and the scattering characteristic that the substance scatters light. An example of the scattering characteristic is a scattering coefficient of a substance.

  The depth calculation unit 218 stores in advance the scattering characteristics of each part of the specimen 800 in association with the part. The depth calculation unit 218 stores the above-described luminance ratio in advance in association with the part. Then, the depth calculation unit 218 is based on the scattering characteristic stored in advance in association with the part where the object imaged as the specific wavelength image exists and the ratio of the brightness stored in advance in association with the part. And the depth may be calculated. As described above, the depth calculation unit 218 may calculate the depth based on the luminance in the image region specified by the object region specifying unit and the optical characteristics of the substance.

  The object image generation unit 230 generates an object image corresponding to the object depth calculated by the depth calculation unit 218. Specifically, the object image generation unit 230 corrects the object image according to the depth of the object calculated by the depth calculation unit 218. That is, the object image generation unit 230 corrects the spread of the object in the object image by applying the correction value. The corrected image corrected by the object image generation unit 230 is sent to the display unit 300 and displayed on the display unit 300. The specific processing in the object image generation unit 230 will be described in more detail with reference to FIG.

  The correction table 222 records a correction value for correcting the spread of the object in the object image in association with the depth of the object. The object image generation unit 230 can correct the object image using the correction value recorded in the correction table 222.

  When the imaging timing of the surface image and the imaging timing of the correction image are different, the interpolation image generating unit 224 generates an interpolation image at the imaging timing of the surface image from a plurality of correction images that are continuous in time series. As an interpolation method, for example, a method of obtaining the luminance of each pixel in the interpolation image from the luminance of each pixel in a plurality of corrected images can be exemplified.

  The display control unit 226 controls display of the surface image and the corrected image on the display unit 300. In addition, the display control unit 226 can display on the display unit 300 by changing the brightness or color of the object in the corrected image according to the depth. Alternatively, the display control unit 226 can display the surface image and the corrected image side by side on the display unit 300. Alternatively, the display control unit 226 can display the interpolation image on the display unit 300 as a corrected image. In addition, the display control unit 226 may display the depth information of the object on the display unit 300 as a numerical value.

  FIG. 3 shows a configuration example of the object image generation unit 230. The object image generating unit 230 includes an object image correcting unit 220, an object predicted image generating unit 232, an image comparing unit 234, a positional relationship specifying unit 236, and a size specifying unit 238.

  The object image correction unit 220 corrects the object image according to the object depth calculated by the depth calculation unit 218. Specifically, the object image correction unit 220 corrects the object image according to the object depth calculated by the depth calculation unit 218. For example, the object image correction unit 220 corrects the spread of the object in the object image by applying the correction value. Note that the object image correction unit 220 can correct the object image using the correction value recorded in the correction table 222. The corrected image corrected by the object image correction unit 220 is sent to the display unit 300 through the interpolation image generation unit 224 and the display control unit 226 and displayed on the display unit 300.

  The object predicted image generation unit 232 displays an object predicted image that is an image of an object to be obtained when an object exists at the depth calculated by the depth calculation unit 218 for each case where the objects have different sizes. Generated based on the depth calculated by the depth calculation unit 218. The image comparison unit 234 compares the object prediction images generated by the object prediction image generation unit 232 with the object images. For example, the image comparison unit 234 compares the plurality of object prediction images generated by the object prediction image generation unit 232 with the object images, and matches each of the plurality of object prediction images with the object image. The degree may be calculated.

  Then, based on the comparison result by the image comparison unit 234, the size specifying unit 238 further sets the size of the predicted object image that more closely matches the object image as the size of the object captured as the object image. Specify with priority. For example, based on the comparison result by the image comparison unit 234, the size specifying unit 238 determines the size of the predicted object image that most closely matches the object image as the size of the object captured as the object image. You may specify with priority. For example, based on the comparison result by the image comparison unit 234, the size specifying unit 238 predicts an object that matches the object image with a higher degree of matching than the predetermined value as the size of the object captured as the object image. The size of the obtained image may be specified with higher priority.

  The object image correcting unit 220 may correct the size of the object image to a size according to the size specified by the size specifying unit 238. In this way, the object image generation unit 230 can generate an image of an object having a size corresponding to the size specified by the size specifying unit 238.

  Note that the depth calculation unit 218 may calculate the depth of an object captured at each of a plurality of different positions in the specific wavelength image from the plurality of specific wavelength images for each of the plurality of positions. The object predicted image generation unit 232 converts an object predicted image that is an image of an object to be obtained when a plurality of objects are present in proximity to each other in a case where the positional relationship between the plurality of objects is different. Is generated based on the depth calculated for each of a plurality of positions. Then, the image comparison unit 234 compares the plurality of object prediction images generated by the object prediction image generation unit 232 with the object images. For example, the image comparison unit 234 compares the plurality of object prediction images generated by the object prediction image generation unit 232 with the object images, respectively, and matches each of the plurality of object prediction images with the object image. The degree may be calculated.

  Based on the comparison result by the image comparison unit 234, the positional relationship specifying unit 236 obtains an object predicted image that more closely matches the object image as a positional relationship in a region where a plurality of objects imaged as object images exist in close proximity. The obtained positional relationship is specified with higher priority. As described above, the positional relationship specifying unit 236 can specify the positional relationship of a plurality of objects captured as object images based on the depth calculated by the depth calculation unit 218 for each of a plurality of positions. Then, the object image correction unit 220 corrects the object image according to the depth calculated by the depth calculation unit 218 for each of a plurality of positions and the positional relationship specified by the positional relationship specification unit 236.

  FIG. 4 schematically shows how light is reflected inside the specimen 800. A blood vessel 810, which is an example of an object, exists inside the specimen 800. ICG, which is a luminescent substance, is injected into the specimen 800, and ICG exists inside the blood vessel 810. The specimen 800 is irradiated with an infrared ray Iir that is excitation light of ICG and red light Ir, green light Ig, and blue light Ib that are irradiation light to a blood vessel 810 that is an object.

  The infrared ray Iir can enter a relatively deep position (depth dir) of the specimen 800, and the ICG in the blood vessel 810 existing between the surface and the depth dir is excited by the infrared ray Iir. Therefore, the image of the blood vessel 810 existing at a position shallower than the depth dir is acquired as an object image by the fluorescence Rf from the ICG. Note that the image of the blood vessel 810 acquired as the object image is blurred.

  The red light Ir enters the depth dr and is reflected in the vicinity of the depth dr. Therefore, the red reflected light Rr of the red light Ir includes image information of the blood vessel 810 near the depth dr. The image of the blood vessel 810 by the red reflected light Rr is acquired as a specific wavelength image in the red region, and the specific wavelength image in the red region includes an image of the blood vessel 810 in the vicinity of the depth dr.

  The green light Ig enters the depth dg and is reflected in the vicinity of the depth dg. Therefore, the green reflected light Rg of the green light Ig includes image information of the blood vessel 810 near the depth dg. The image of the blood vessel 810 by the green reflected light Rg is acquired as a specific wavelength image of the green region, and the specific wavelength image of the green region includes an image of the blood vessel 810 near the depth dg.

  The blue light Ib enters to the depth db and is reflected in the vicinity of the depth db. Therefore, the blue reflected light Rb of the blue light Ib includes image information of the blood vessel 810 near the depth db. The image of the blood vessel 810 by the blue reflected light Rb is acquired as a specific wavelength image of the blue region, and the specific wavelength image of the blue region includes an image of the blood vessel 810 near the depth db.

  For example, the depth of the blood vessel 810 can be specified from the specific wavelength images of the red reflected light Rr, the green reflected light Rg, and the green reflected light Rg. The object image by the fluorescence Rf can be corrected from the specified depth.

  FIG. 5 shows an example of the display screen 310 in the display unit 300. On the display screen 310, the blood vessel 330 of the corrected image obtained by correcting the object image is displayed superimposed on the blood vessel 320 in the surface image. The blood vessel 320 in the surface image can be visually recognized by observing the surface of the specimen 800. The blood vessel 330 in the corrected image cannot be visually recognized by surface observation, but can be visualized by the image processing system 10 of the present embodiment.

  Since the blood vessel 320 in the surface image can be visually recognized by observing the surface of the specimen 800, the object image generation unit 230 corrects the blood vessel 810 in which the depth indicating the position shallower than the predetermined depth is calculated by the depth calculation unit 218. A corrected image may be generated for the blood vessel 810 in which a depth equal to or greater than a predetermined depth is calculated by the depth calculation unit 218 without generating an image. Then, the display control unit 226 may display the blood vessel 330 of the correction image in a color different from the color of the blood vessel 320 of the surface image. As described above, according to the image processing system 10, it is possible to provide an image in which the blood vessel 810 at a shallow position and the blood vessel 810 at a deep position can be easily distinguished and visually recognized.

  Since the blood vessel 330 in the corrected image is corrected by the image processing system 10 of the present embodiment, the blood vessel image is displayed clearly without blurring at the end of the blood vessel image. It should be noted that the blood vessel 330 of the corrected image can display the depth from the surface, for example, by changing the density or changing the color. If the image processing system 10 of this embodiment is applied, for example, when a doctor performs an operation or the like while viewing the display screen 310 of the display unit 300, the internal blood vessel 330 that cannot be visually recognized by surface observation may be clearly recognized. is there. In addition, there may be an advantage that the doctor can perform an operation or the like while referring to the depth information of the internal blood vessel 330.

  FIG. 6 shows an example of a blood vessel image that is an image of intersecting blood vessels 810. The blood vessel partial images 610a to 610d are images of blood vessel portions that do not intersect in the specific wavelength image. Further, it is assumed that the region 630 is a region where the blood vessels 810 intersect in the specific wavelength image. Note that the different blood vessels 810 in the present embodiment may be a plurality of different blood vessel portions that form a continuous blood vessel 810. In the following description, blood vessel partial images 610a to 610d may be collectively referred to as blood vessel partial images 610.

  The depth calculation unit 218 can calculate the depth of the blood vessel 810 as described above based on the luminance values of the blood vessel partial images 610a to 610d in regions where different blood vessels 810 do not intersect. On the other hand, the light from the region where the different blood vessels 810 intersect becomes light that is combined with the light from each of the different blood vessels 810. For this reason, the luminance value of the blood vessel image in the region 630 is a value indicating the sum of the intensities of light from the different blood vessels 810. For this reason, the depth calculation unit 218 may not be able to accurately calculate the depth of each of the different blood vessels 810 from only the luminance value of the blood vessel image in the region 630.

  In this case, the object image generation unit 230 specifies the positional relationship between different blood vessels 810 by a method described later, and generates a blood vessel image corresponding to the specified positional relationship. For example, the object image generation unit 230 generates a blood vessel image indicating the specified positional relationship based on the specified positional relationship, or generates a corrected image in which blurring of intersecting blood vessel images in the object image is corrected. Note that s in the figure indicates a route along the traveling direction of the blood vessel, and is used in the description in FIG.

  FIG. 7 shows an example of a running state of a blood vessel. The blood vessel 810 captured as a blood vessel image shown in FIG. 6 can be regarded as being in the traveling state of any one of (a), (b), and (c) in this figure. In this figure, two blood vessels 810 are shown separately by a solid line and a broken line.

  The positional relationship specifying unit 236 has the two blood vessels 810 running in any one of a), b), and c) based on the respective depths of the blood vessel portions captured as the blood vessel partial images 610a to 610d. To identify. Note that the depth of each of the blood vessel portions imaged as each of the blood vessel partial images 610a to 610d can be obtained by the depth calculation unit 218 as described above. In the following description, the depth of the blood vessel part imaged as each of the blood vessel partial images 610a to 610d is abbreviated as the depth from the blood vessel partial images 610a to 610d, respectively.

  For example, the positional relationship specifying unit 236 has a depth from the blood vessel partial image 610a and a depth from the blood vessel partial image 610c that are deeper than both the depth from the blood vessel partial image 610b and the depth from the blood vessel partial image 610d. If it is shallow, it can be determined that the different blood vessels 810 are in the positional relationship shown in FIG. Further, the positional relationship specifying unit 236 has a depth from the blood vessel partial image 610a and a depth from the blood vessel partial image 610b that are deeper than either the depth from the blood vessel partial image 610c or the depth from the blood vessel partial image 610d. If it is shallow, it can be determined that the different blood vessels 810 are in the positional relationship shown in FIG. In addition, the positional relationship specifying unit 236 determines whether the depth from the blood vessel partial image 610a and the depth from the blood vessel partial image 610d are any of the depth from the blood vessel partial image 610b and the depth from the blood vessel partial image 610c. If it is deeper or shallower, it can be determined that the different blood vessels 810 are in the positional relationship shown in FIG. In this manner, the positional relationship specifying unit 236 can specify the positional relationship of different blood vessels 810 by classifying the depth from each of the blood vessel partial images 610a to 610d.

  In this way, the positional relationship specifying unit 236 can specify the positional relationship of different blood vessels 810 based on the depth from each of the blood vessel partial images 610a to 610d. The positional relationship specifying unit 236 selects one positional relationship from a plurality of positional relationships based on the curvature of the blood vessel 810 predicted in each case where different blood vessels 810 are in the positional relationship, The selected positional relationship may be specified as the positional relationship of the blood vessel 810.

  For example, if different blood vessels 810 are in the positional relationship shown in (c) of the figure, the curvature is remarkably larger than that in the case of the positional relationship shown in (a) or (b) of the figure. Actually, the possibility that the blood vessel 810 is in the running state shown in (c) of the figure is significantly smaller than the possibility that the blood vessel 810 is in the running state shown in (a) or (b) of the figure. Therefore, the positional relationship specifying unit 236 may specify the positional relationship in which the curvature is smaller than a predetermined value as the positional relationship of the blood vessel 810 in preference to the positional relationship in which the curvature is greater than or equal to the predetermined value. . Thereby, possibility that the positional relationship of the unreal blood vessel 810 will be specified can be reduced significantly.

  The positional relationship specifying unit 236 differs in the same manner as the positional relationship specifying method based on the depth described above, based on the respective thicknesses of the blood vessel portions captured as the blood vessel partial images 610a to 610d. The positional relationship of the blood vessel 810 may be specified. For example, the size specifying unit 238 specifies the thickness of each blood vessel portion imaged as each of the blood vessel partial images 610a to 610d. In the following description, the thickness of each blood vessel part imaged as each of the blood vessel partial images 610a to 610d is abbreviated as the thickness from each of the blood vessel partial images 610a to 610d.

  As an example, the size specifying unit 238 can specify the thickness from each of the blood vessel partial images 610a to 610d based on the depth from each of the blood vessel partial images 610a to 610d. For example, it can be considered that the thickness of each blood vessel portion image 610 is contributed by the thickness of each blood vessel portion and the blur caused by light scattering from the blood vessel 810. Therefore, the size specifying unit 238 calculates the contribution of blur caused by light scattering based on the depth from each of the blood vessel partial images 610a to 610d. Then, the size specifying unit 238 specifies the thickness from each of the blood vessel partial images 610a to 610d based on the calculated contribution and the thickness of each of the blood vessel partial images 610a to 610d. For example, the size specifying unit 238 can specify the thickness from the blood vessel partial image 610a by subtracting the calculated contribution from the thickness of the blood vessel partial image 610a. The thickness of the blood vessel portion may be an example of the size of the object in the present invention.

  Then, the positional relationship specifying unit 236 has a larger thickness from the blood vessel partial image 610a and a thickness from the blood vessel partial image 610c than the thickness from the blood vessel partial image 610b and the thickness from the blood vessel partial image 610d. If it is thin, it can be determined that the different blood vessels 810 are in the positional relationship shown in FIG. In addition, the positional relationship specifying unit 236 has a larger thickness from the blood vessel partial image 610a and a thickness from the blood vessel partial image 610b than the thickness from the blood vessel partial image 610c and the thickness from the blood vessel partial image 610d. When it is thin, it can be determined that the different blood vessels 810 are in the positional relationship shown in FIG. In addition, the positional relationship specifying unit 236 is configured such that the thickness from the blood vessel partial image 610a and the thickness from the blood vessel partial image 610d are any of the thickness from the blood vessel partial image 610b and the thickness from the blood vessel partial image 610c. If it is thicker or thinner, it can be determined that the different blood vessels 810 are in the positional relationship shown in FIG. As described above, the positional relationship specifying unit 236 can specify the positional relationship of different blood vessels 810 by classifying the thicknesses from the blood vessel partial images 610a to 610d.

  The positional relationship specifying unit 236 may specify the positional relationship of different blood vessels 810 by classifying the depth and thickness from each of the blood vessel partial images 610a to 610d. As described above, the positional relationship specifying unit 236 may specify the positional relationship of different blood vessels 810 based on the depth and thickness from each of the blood vessel partial images 610a to 610d.

  In addition to the above-described method, the size specifying unit 238 also includes a predicted blood vessel image and blood vessel partial images 610a to 610a to be obtained when it is assumed that the blood vessel 810 exists at the depth calculated by the depth calculating unit 218. Based on the comparison result with d, the thickness of the blood vessel part imaged as each of the blood vessel partial images 610a to 610d may be calculated. Specifically, the predicted object image generation unit 232 generates a predicted blood vessel image to be obtained when a blood vessel 810 having a given thickness exists at the depth calculated by the depth calculation unit 218.

  Note that the object predicted image generation unit 232 specifies the traveling direction of the blood vessel by performing thinning processing on the blood vessel partial image 610, and the blood vessel 810 having a given thickness traveling in the identified traveling direction has a depth. A blood vessel prediction image to be obtained when the calculation unit 218 exists at the calculated depth may be generated. The object predicted image generation unit 232 generates a plurality of predicted blood vessels by generating a predicted blood vessel for each of a plurality of different thicknesses. The blood vessel prediction image may be an example of the object prediction image in the present invention. The object predicted image generation unit 232 may perform the thinning process after performing the smoothing process and the binarization process on the blood vessel partial image 610.

  The object predicted image generation unit 232 can generate a predicted blood vessel image based on the amount of light spread from the blood vessel 810 corresponding to the depth calculated by the depth calculation unit 218. For example, the object predicted image generation unit 232 uses the spatial distribution of the light intensity from the blood vessel portion when a blood vessel portion having a given thickness exists, and the light from the point at the depth calculated by the depth calculation unit 218. A blood vessel prediction image can be generated based on the point spread function for.

  Then, the image comparison unit 234 calculates the degree of coincidence of the images by comparing the plurality of blood vessel prediction images generated by the object prediction image generation unit 232 with the blood vessel partial images 610, respectively. Then, the size specifying unit 238 specifies the thickness of the blood vessel portion given when generating the predicted blood vessel image with the highest degree of coincidence as the thickness of the blood vessel portion captured as the blood vessel partial image 610. To do. In this manner, the size specifying unit 238 can specify the thickness from each of the blood vessel partial images 610a to 610d. Note that the size specifying unit 238 weights and averages the thickness of the blood vessel portion given in the case of generating a predicted blood vessel image having a higher matching degree than a predetermined matching degree according to the matching degree. The obtained value may be specified as the thickness of the blood vessel portion captured as the blood vessel partial image 610.

  The image processing unit 200 stores the depth calculated by the depth calculation unit 218 and the thickness calculated by the size specifying unit 238 in association with the luminance value of the blood vessel partial image and the thickness of the blood vessel partial image. It's okay. Then, the depth calculation unit 218 newly obtains the brightness value of the newly obtained blood vessel partial image and the depth stored in association with the thickness of the newly obtained blood vessel partial image. It may be calculated as the depth of the blood vessel part imaged as a blood vessel partial image. Then, the size specifying unit 238 newly obtains the brightness value stored in association with the brightness value of the newly obtained blood vessel partial image and the thickness of the newly obtained blood vessel partial image. It may be specified as the thickness of the blood vessel portion captured as the blood vessel partial image. As a result, the depth and thickness of the blood vessel portion can be quickly calculated, and as a result, the geometry of the blood vessel 810 can be quickly determined.

  FIG. 8 shows an example of a change in depth in the direction along the traveling direction s of the blood vessel 810. In this figure, (a), (b), (c), and (d) are the depth change from the blood vessel partial image 610a, the depth change from the blood vessel partial image 610b, and the depth from the blood vessel partial image 610c, respectively. The depth change from the blood vessel partial image 610d is shown.

  The depths of (a) and (c) in this figure increase along the traveling direction s, and the depths of (b) and (d) decrease along the traveling direction s. In such a case, the positional relationship specifying unit 236 may specify the positional relationship shown in FIG. 6A as the positional relationship between the different blood vessels 810. As described above, the positional relationship specifying unit 236 may specify the positional relationship of the blood vessel 810 by regarding the blood vessel partial images showing similar depth changes as images of the same blood vessel 810. As described above, the positional relationship specifying unit 236 can specify the positional relationship based on the spatial change in the depth of the blood vessel 810. In addition, as a spatial change of depth, the differential value of the depth of the direction along the running direction s of the blood vessel 810 can be illustrated. The order of the differential value may be first order or multi-order.

  The positional relationship specifying unit 236 can specify the positional relationship based on the change in the thickness of the blood vessel 810 instead of the change in depth described with reference to FIG. For example, the positional relationship specifying unit 236 may specify the positional relationship of the blood vessels 810 by regarding blood vessel partial images showing similar thickness changes as images of the same blood vessels 810.

  FIG. 9 shows an example of the positional relationship of the blood vessel 810. This figure shows a cut surface cut along a cross section including a center line of a blood vessel part imaged as a blood vessel partial image 610a and a blood vessel partial image 610c when the blood vessel 810 is in the positional relationship of FIG. When the blood vessel 810 is in the positional relationship shown in this figure, the depth from the surface 900 of the blood vessel part imaged as each of the blood vessel partial images 610a to 610d is substantially the same. For this reason, the positional relationship of the blood vessel 810 in the intersecting region may not be specified based on the depth of the blood vessel partial images 610a to 610d. For example, as a positional relationship at the position where the blood vessel 810 intersects, the blood vessel part imaged as the blood vessel partial image 610a and the blood vessel partial image 610c as in the example of this figure is in a deeper position than the other, There can actually be both cases where the blood vessel part imaged as the image 610a and the blood vessel partial image 610c is at a position shallower than the other.

  Even in such a case, the positional relationship specifying unit 236 determines the blood vessel based on the comparison result between the predicted blood vessel image and the image of the blood vessel 810 when it is assumed that the blood vessel 810 has a given positional relationship. The positional relationship of 810 can be specified. Specifically, the object predicted image generation unit 232 generates a predicted blood vessel image to be obtained when a blood vessel 810 having a given thickness exists in a given positional relationship.

  As the positional relationship, as an example, a plurality of positional relationships related to the traveling direction shown in FIGS. 6A and 6B and a plurality of positions related to the depth of the blood vessel 810 in the intersecting region. Includes a combination of relationships. The object predicted image generation unit 232 generates a plurality of predicted blood vessels by generating respective predicted blood vessels in different positional relationships in each case where the blood vessels 810 have different thicknesses. The blood vessel prediction image may be an example of the object prediction image in the present invention.

  Then, the image comparison unit 234 compares the plurality of predicted blood vessel images generated by the object predicted image generation unit 232 with the images of the blood vessels 810 in the region 630, thereby calculating the degree of coincidence of the images. Then, the size specifying unit 238 calculates, as the thickness of each of the different blood vessels 810, the value given as the thickness of the different blood vessel 810 when generating the predicted blood vessel image having the highest degree of coincidence. Further, the positional relationship specifying unit 236 specifies the positional relationship given when generating a predicted blood vessel image with the highest degree of coincidence as the positional relationship of different blood vessels 810. In this way, the positional relationship specifying unit 236 and the size specifying unit 238 can specify the positional relationship and the thickness of the intersecting blood vessel 810 with higher accuracy.

  Note that, in the above, the positional relationship specifying method based on the depth of the blood vessel 810, the positional relationship specifying method based on the thickness of the blood vessel 810, the positional relationship specifying method based on the depth change of the blood vessel 810, and the blood vessel 810 The positional relationship specifying method based on the change in the thickness of the blood vessel and the positional relationship specifying method based on the blood vessel prediction image have been described. The positional relationship specifying unit 236 can specify the positional relationship of different blood vessels 810 based on any combination of the positional relationship specifying methods.

  Note that the object image correction unit 220 may correct an image of an area captured by overlapping different blood vessels 810 in the object image based on the positional relationship specified by the positional relationship specifying unit 236. Specifically, the object image correcting unit 220 generates a corrected image in which the image of the blood vessel 810 at the shallow position is emphasized from the image of the blood vessel 810 at the deep position based on the positional relationship specified by the positional relationship specifying unit 236. You can do it. As described above, the object image correction unit 220 may correct the object image so that the object at a position shallower than the object is displayed with priority over the image of the object at a deeper position.

  As an example, the object image correction unit 220 corrects the image of the blood vessel 810 at the shallow position according to the depth to the image obtained by correcting the image of the blood vessel 810 at the deep position according to the depth. The obtained image may be overwritten. In the object image, the pixel value of the region where the blood vessel 810 is overlapped is the sum of the contribution to the pixel value by the blood vessel 810 at the deep position and the contribution to the pixel value by the blood vessel 810 at the shallow position. It is a value according to. The object image correction unit 220 overwrites the image of the blood vessel 810 at a shallow position with an image obtained by correcting the image according to the depth, so that the pixel value of the region where the blood vessel 810 is overlapped and captured is deep. The blood vessel 810 at the position does not contribute, and only the blood vessel 810 at the shallow position contributes. Thereby, the object image correction unit 220 can generate a corrected image in which the boundary and the running state of the blood vessel 810 at a shallow position are clearer even in the region where the blood vessel 810 is overlapped and imaged.

  6 to 9, the image processing unit 200 is exemplified by a case where the positional relationship between the two blood vessels 810 is specified and an image of the two blood vessels 810 corresponding to the specified positional relationship is generated. Explained the operation. Similarly, the image processing unit 200 can specify the positional relationship between the three or more blood vessels 810 and generate an image of the three or more blood vessels 810 according to the specified positional relationship.

  As described above, according to the image processing system 10, it is possible to provide a corrected image in which the geometry of a plurality of blood vessels 810 captured in an intersecting or overlapping manner is appropriately expressed. For this reason, a doctor or the like can easily visually recognize the traveling state of the blood vessel 810 using the corrected image.

  FIG. 10 shows an example of a hardware configuration of the image processing system 10 according to the present embodiment. The image processing system 10 according to this embodiment includes a CPU peripheral unit including a CPU 1505, a RAM 1520, a graphic controller 1575, and a display device 1580 that are connected to each other by a host controller 1582, and a host controller 1582 by an input / output controller 1584. I / O unit having a communication interface 1530, a hard disk drive 1540, and a CD-ROM drive 1560 connected to the I / O controller, and a legacy input having a ROM 1510, a flexible disk drive 1550, and an I / O chip 1570 connected to the I / O controller 1584. And an output unit.

  The host controller 1582 connects the RAM 1520 to the CPU 1505 and the graphic controller 1575 that access the RAM 1520 at a high transfer rate. The CPU 1505 operates based on programs stored in the ROM 1510 and the RAM 1520 to control each unit. The graphic controller 1575 acquires image data generated by the CPU 1505 or the like on a frame buffer provided in the RAM 1520 and displays the image data on the display device 1580. Alternatively, the graphic controller 1575 may include a frame buffer that stores image data generated by the CPU 1505 or the like.

  The input / output controller 1584 connects the host controller 1582 to the communication interface 1530, the hard disk drive 1540, and the CD-ROM drive 1560, which are relatively high-speed input / output devices. The communication interface 1530 communicates with other devices via a network. The hard disk drive 1540 stores programs and data used by the CPU 1505 in the image processing system 10. The CD-ROM drive 1560 reads a program or data from the CD-ROM 1595 and provides it to the hard disk drive 1540 via the RAM 1520.

  The input / output controller 1584 is connected to the ROM 1510, the flexible disk drive 1550, and the relatively low-speed input / output device of the input / output chip 1570. The ROM 1510 stores a boot program that the image processing system 10 executes at startup, a program that depends on the hardware of the image processing system 10, and the like. The flexible disk drive 1550 reads a program or data from the flexible disk 1590 and provides it to the hard disk drive 1540 via the RAM 1520. The input / output chip 1570 connects various input / output devices via a flexible disk drive 1550 such as a parallel port, a serial port, a keyboard port, and a mouse port.

  A communication program provided to the hard disk drive 1540 via the RAM 1520 is stored in a recording medium such as the flexible disk 1590, the CD-ROM 1595, or an IC card and provided by the user. The communication program is read from the recording medium, installed in the hard disk drive 1540 in the image processing system 10 via the RAM 1520, and executed by the CPU 1505. A communication program installed and executed in the image processing system 10 works on the CPU 1505 or the like to cause the image processing system 10 to function as the image processing unit 200 or the like described with reference to FIGS.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiment. It is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

  For example, the fluorescence intensity from ICG is considered to depend on the concentration of ICG in the specimen 800. Since ICG is reduced by being metabolized in the specimen 800, it is preferable to compensate for the fluorescence intensity by reducing the ICG concentration. An example of the compensation method is a method of additionally administering ICG according to the elapsed time from ICG administration. Further, a method of increasing the excitation light intensity so as to compensate for the decrease in the ICG concentration, or a method of compensating the correction value value depending on the fluorescence intensity can be exemplified.

  In the above-described embodiment, an example in which an image inside the specimen 800 is corrected using the endoscope 100 has been described. However, the application of the image processing system 10 of the present embodiment is not limited to the observation image inside the specimen 800. For example, the present invention may be applied to a case where an image of an object existing in a deep portion of the surface that can be observed from the outside of the specimen 800 is corrected by irradiating excitation light or irradiation light from the outside of the specimen 800.

  In the above-described embodiment, the example in which the object image is acquired using the fluorescence emitted from the luminescent substance contained in the object has been mainly described. However, the object image can also be obtained by irradiating the object with visible light, infrared light or other light in the wavelength range and imaging the reflected light.

An example of the image processing system 10 of the present embodiment is shown together with an ICG injection unit 700 and a specimen 800. 2 shows a configuration example of an image processing unit 200. 3 is a diagram illustrating a configuration example of an object image generation unit 230. FIG. A state in which light is reflected or the like inside the specimen 800 is schematically shown. An example of the display screen 310 in the display part 300 is shown. It is a figure which shows an example of the blood vessel image which is the image of the blood vessel 810 which cross | intersects. It is a figure which shows an example of the running state of the blood vessel. It is a figure which shows an example of the depth change of the direction along the running direction s of the blood vessel 810. FIG. 5 is a diagram illustrating an example of a positional relationship of a blood vessel 810. FIG. 1 shows an example of a hardware configuration of an image processing system 10 according to the present embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Image processing system 100 Endoscope 102 Tip part 110 Imaging part 112 Objective lens 120 Light guide 122 IR light guide 124 R light guide 126 G light guide 128 B light guide 130 Forceps opening 135 Forceps 140 Nozzle 200 Image processing part 210 Object image Acquisition unit 212 Specific wavelength image acquisition unit 214 Surface image acquisition unit 216 Object region specification unit 218 Calculation unit 220 Object image correction unit 222 Correction table 224 Interpolated image generation unit 226 Display control unit 230 Object image generation unit 232 Object predicted image generation unit 234 Image comparison unit 236 Position relationship specifying unit 238 Size specifying unit 300 Display unit 310 Display screen 320 Blood vessel 330 Blood vessel 400 Imaging control unit 500 Light emitting unit 600 Light emission control unit 610 Blood vessel partial image 630 region 900 surface 700 ICG injection part 800 specimen 810 blood vessel 1505 CPU
1510 ROM
1520 RAM
1530 Communication interface 1540 Hard disk drive 1550 Flexible disk drive 1560 CD-ROM drive 1570 Input / output chip 1575 Graphic controller 1580 Display device 1582 Host controller 1584 Input / output controller 1590 Flexible disk 1595 CD-ROM

Claims (25)

A specific wavelength image acquisition unit that acquires a specific wavelength image that is an image of light that belongs to a specific wavelength region and is light from an object existing inside the substance;
A depth calculator that calculates the depth of the object from the surface of the substance from a plurality of the specific wavelength images having different specific wavelength regions;
An object image generation unit that generates an image of the object according to the depth of the object calculated by the depth calculation unit;
An image processing system comprising: An object image acquisition unit that acquires an object image that is an image of light from the object;
The object image generation unit
The image processing system according to claim 1, further comprising: an object image correction unit that corrects the object image according to the depth of the object calculated by the depth calculation unit. The specific wavelength image acquisition unit acquires a reflected light image that is an image of reflected light reflected from the object as the specific wavelength image.
The image processing system according to claim 1 or 2. The specific wavelength image acquisition unit acquires, as the specific wavelength image, a reflected light image that is an image of reflected light belonging to the specific wavelength region among reflected light reflected from the object by irradiating the object with white light. To
The image processing system according to claim 3. The specific wavelength image acquisition unit acquires a reflected light image, which is an image of reflected light reflected from the object by irradiating the object with irradiation light belonging to the specific wavelength region, as the specific wavelength image.
The image processing system according to claim 3. The specific wavelength image acquisition unit acquires, as the specific wavelength image, a light emission image that is an image of light belonging to the specific wavelength region among light emitted from a luminescent substance inside the object.
The image processing system according to claim 1 or 2. An object region specifying unit that specifies an image region of the object for each of the plurality of specific wavelength images;
The depth calculation unit calculates the depth based on luminance in the image region specified by the object region specification unit.
The image processing system according to claim 1 or 2. The depth calculation unit calculates the depth based on a ratio of the luminance of the image region in the specific wavelength region in the short wavelength region to the luminance of the image region in the specific wavelength region in the long wavelength region,
The image processing system according to claim 7. The depth calculation unit calculates the depth based on a maximum luminance or an average luminance in the image region;
The image processing system according to claim 7. The depth calculation unit calculates the depth based on a change rate of luminance at an end of the image region;
The image processing system according to claim 7. The image processing system according to claim 7, wherein the depth calculation unit calculates the depth based on luminance in the image region specified by the object region specification unit and optical characteristics of the substance. The image processing system according to claim 11, wherein the object image acquisition unit acquires an object image that is an image of scattered light in which light from the object is scattered from the object to the surface. The image processing system according to claim 12, wherein the depth calculation unit calculates the depth based on luminance in the image region specified by the object region specifying unit and scattering characteristics in which the substance scatters light. . A correction table that records correction values for correcting the spread of the object in the object image in association with the depth of the object;
The object image correction unit applies the correction value to correct the spread of the object in the object image;
The image processing system according to claim 2. A light emitting unit for generating excitation light for exciting a luminescent substance inside the object and irradiation light for irradiating the object;
An imaging unit that captures an image of light emitted from the luminescent material and reflected light reflected by the object with the irradiation light;
A display unit for displaying a corrected image corrected by the object image correction unit;
The image processing system according to claim 2, further comprising: A surface image acquisition unit that acquires a surface image that is an image of the surface of the substance;
A display control unit that controls display of the surface image and the correction image on the display unit;
The image processing system according to claim 15, further comprising: The display control unit changes the brightness or color of the object in the corrected image according to the depth and displays the change on the display unit.
The image processing system according to claim 16. The display control unit displays the surface image and the correction image side by side on the display unit.
The image processing system according to claim 16. When the imaging timing of the surface image and the imaging timing of the correction image are different, an interpolation image generating unit that generates an interpolation image at the imaging timing of the surface image from the plurality of correction images that are continuous in time series is further provided. ,
The display control unit displays the interpolated image as the corrected image on the display unit.
The image processing system according to claim 16. The depth calculation unit calculates, for each of the plurality of positions, the depth of the object captured at each of a plurality of different positions in the specific wavelength image from the plurality of the specific wavelength images,
The object image generation unit
A positional relationship specifying unit that specifies the positional relationship of the plurality of objects captured as the object image based on the depth calculated by the depth calculation unit for each of a plurality of positions;
The object image correction unit corrects the object image according to the depth calculated by the depth calculation unit for each of a plurality of positions and the positional relationship specified by the positional relationship specifying unit. Image processing system. The object image generation unit
For each of the cases where the positional relationship of the plurality of objects is different, the depth calculation unit calculates the object prediction image that is an image of the object to be obtained when the plurality of the objects are close to each other. An object predicted image generation unit that generates based on the calculated depth;
An image comparison unit that compares each of the plurality of object prediction images generated by the object prediction image generation unit with the object image;
The positional relationship specifying unit is more closely matched to the object image as a positional relationship in a region where a plurality of the objects captured as the object image are present based on a comparison result by the image comparing unit. The image processing system according to claim 20, wherein the positional relationship from which the object predicted image is obtained is specified with higher priority. The image processing system according to claim 21, wherein the object image correction unit corrects the object image so that the object at a shallower position is displayed with higher priority. The object image generation unit
An object predicted image, which is an image of the object to be obtained when the object exists at the depth calculated by the depth calculation unit, is calculated for each case where the object has a different size. An object predicted image generation unit that generates based on the depth calculated by the unit;
An image comparison unit that compares the plurality of object prediction images generated by the object prediction image generation unit with the object image;
Based on the comparison result by the image comparison unit, the size of the object image captured as the object image is specified with higher priority as the size of the predicted object image that more closely matches the object image. A size specifying part,
The image processing system according to claim 2, wherein the object image correcting unit corrects the size of the object image to a size according to the size specified by the size specifying unit. A specific wavelength image acquisition step of acquiring a specific wavelength image, which is an image of light belonging to a specific wavelength region, which is light from an object existing inside the substance;
A depth calculating step of calculating a depth of the object from the surface of the substance from a plurality of the specific wavelength images having different specific wavelength regions;
An object image generation step of generating an image of the object according to the depth of the object calculated in the depth calculation step;
An image processing method comprising: A program for an image processing system, wherein the image processing system is
A specific wavelength image acquisition unit that acquires a specific wavelength image that is an image of light that belongs to a specific wavelength region and is light from an object existing inside the substance;
A depth calculation unit for calculating a depth of the object from the surface of the substance from a plurality of the specific wavelength images having different specific wavelength regions;
An object image generation unit configured to generate an image of the object according to the depth of the object calculated by the depth calculation unit;
Program to function as.

Images