JP4625956B2 - Image processing apparatus and image processing method - Google Patents

Description

  The present invention relates to an image processing apparatus and an image processing method. In particular, the present invention relates to an apparatus for creating a tissue image of an observation object covered with an obstacle.

  When performing surgery on animals including humans, it is often difficult to visually recognize the target site of surgery such as tumors and surrounding blood vessels and nerve bundles that should not be damaged by cerebrospinal fluid mixed with blood. It happens. This is a particularly remarkable phenomenon particularly in surgery that requires delicate procedures such as neurosurgery.

  In such a case, the conventional method of removing an obstacle layer such as blood has repeatedly performed an operation of exposing a site to be viewed with a gauze or a suction tube frequently during an operation. However, the operation for exposing these desired sites has been a major obstacle to the progress of surgery. In addition, when gauze containing blood is adhered on the tissue, it is difficult to visually recognize the inside without removing the gauze.

  As a conventional method for solving these problems, an X-ray fluoroscopic apparatus, an ultrasonic image diagnostic apparatus, and an MR image diagnostic apparatus are conveniently used as an image diagnostic apparatus using an ordinary optical apparatus in an operating room. It was done to be introduced. However, each of these devices also has problems.

In the X-ray fluoroscopic apparatus, since the operator cannot touch the part to be visually recognized because X-rays are irradiated, it is difficult to confirm the part to be visually recognized at the same time as performing the operation. Further, in the ultrasonic diagnostic imaging apparatus, since ultrasonic waves are not transmitted in a non-contact manner, it has been necessary to perform measurement while contacting an obstacle layer such as blood. This is an extremely difficult task, not necessarily because the obstacle layer has a fixed shape. Furthermore, in the MR imaging apparatus, the size of the apparatus itself is very large, and since the MR imaging apparatus generates a strong magnetic field, metal surgical instruments cannot be used. I have a problem.
T. Zimmermann, J. Rietdorf, R. Pepperkok, "Spectral Imaging and its applications in live cell microscopy", FEBS Letters, 546 (1), 87-92 (2003)

  In the conventional method, it is not possible to perform an operation while observing a tissue image of the operation target region without removing a substance that becomes an obstacle when the operation target region is visually recognized. In addition to surgery, it is required to clearly observe a tissue image of an observation object covered with a substance that becomes an obstacle without removing the substance that becomes an obstacle when visually recognizing. Therefore, in the present invention, the effect of reflection, scattering and absorption of light by the obstacle layer hindering visual recognition is reduced as much as possible, and the tissue image under the obstacle is clearly observed, and the tissue under the obstacle is observed. It is to realize an apparatus that easily provides an image.

An image processing apparatus according to an aspect of the present invention is an image processing apparatus that determines, by learning, an arithmetic processing method for generating a fluoroscopic image of an observation object covered by an obstacle. The obstacle and the obstacle imaging data to an imaging data with the learning object are covered, and inputs the acquired by the infrared light of a plurality of wavelengths selected based on the reflection characteristic of the transmission absorption and the observed object of the obstacle An input unit; and an image data calculation processing unit that generates perspective image data of the learning object by performing linear calculation and / or nonlinear calculation between wavelengths for infrared light having a plurality of wavelengths that have passed through the obstacle ; , and a Lud chromatography data comparing section compares the reflectance data of the fluoroscopic image data and previously acquired the learning object are of the learning object, the image data processing unit, before Kide Data comparison section Based on the comparison data, changes the processing method of the imaging data as a difference in reflectance of the corresponding point is reduced between the reflectance data of said learning object acquired in advance with the fluoroscopic image data To do.

Other image processing method according to the embodiment of the present invention, the shooting image data of the learning object being covered by an obstacle the obstacle, the reflection characteristic of the transmission absorption and the observed object of the obstacle Obtaining and inputting infrared light of a plurality of wavelengths selected on the basis of the imaging data, generating perspective image data of the learning object from the imaging data, and acquiring the perspective image data of the learning object in advance Comparing the learning object with the reflectance data of the learning object, and depending on the result of the comparison, the fluoroscopic image data of the learning object and the reflectance data of the learning object acquired in advance. A step of changing the calculation processing method of the imaging data so as to reduce the difference in reflectance between corresponding points, and using the calculation processing method , infrared light having a plurality of wavelengths that have passed through the obstacle Nitsu Step of generating a fluoroscopic image of the observed object is covered with the obstacle by performing linear operation and / or non-linear operation between wavelengths Te is an image processing method having.

  According to the image processing apparatus of the present invention, when an observation target is covered with a substance that prevents the human eye from performing visual recognition, the substance that prevents the human eye from performing visual recognition is not removed. In addition, it is possible to easily provide a tissue image of the observation object. This means that the surgery can be performed while observing the tissue image of the surgical site even when the surgical site during surgery is covered with a body fluid containing blood.

Embodiment 1 FIG.
Hereinafter, specific embodiments to which the present invention is applied will be described in detail with reference to the drawings. In the present embodiment, the present invention is applied to an image processing apparatus. In the image processing apparatus according to the present embodiment, the hyperspectral sensor is used to obtain wavelength information and intensity information in the infrared light region, and the reflected light from the part other than the observation target is reduced as much as possible to the obstacle layer. In order to extract the information of the covered observation object as effectively as possible, the calculation processing method between the data of various wavelengths observed in the hyperspectral sensor is optimized. In order to optimize the arithmetic processing method between data of various wavelengths, the image processing apparatus according to the present embodiment uses a multilayer perceptron.

  In the image processing apparatus according to the present embodiment, an apparatus for observing a surgical site covered with a body fluid containing blood will be described as an example. In this case, the obstacle layer is a body fluid containing blood, and the observation object is a surgical site of the living tissue. FIG. 1 shows the absorption spectrum characteristics of water ((a-1)), hemoglobin ((a-2)), and oxidized hemoglobin ((a-3)). The vertical axis represents the absorption coefficient, and the horizontal axis represents the wavelength of light. The absorption coefficient is calculated by dividing the logarithm of the intensity ratio between incident light and transmitted light by the transmitted optical path length. That is, the absorption spectrum characteristic shown in FIG. 1 shows the wavelength dependence of the absorption coefficient.

  As shown in FIG. 1, the absorption of hemoglobin and oxyhemoglobin constituting blood is large in the visible light region so that only the deep red color having a long wavelength in the visible light region is visible. Become. Therefore, it turns out that it is difficult for the biological tissue covered with the obstacle layer containing blood to be visually recognized by human eyes.

  On the other hand, the absorption of water contained in a large amount in blood from the visible light region to the infrared light region is less absorbed in the visible light region and has several absorption peaks in the infrared light region. In other words, hemoglobin and oxyhemoglobin that make up blood absorb visible light, and in the infrared light region, there is absorption by water, so using light in a wide wavelength region from the visible light region to the infrared light region, This suggests that it is difficult to perform image fluoroscopy through an obstacle layer including

  Based on these facts, in order to reveal the tissue image under the obstacle layer, select a plurality of wavelength regions where there is little absorption and scattering of the obstacle layer and the tissue image of the observation object has a clear contrast. There is a need. The wavelength range selected as described above is changed by the material of the obstacle layer. That is, in this embodiment, since the obstacle layer is a blood layer, it is necessary to select a wavelength range corresponding to the blood layer.

  A tissue image of the observation object is obtained by combining the spectra of a plurality of wavelength regions selected as described above. Further, by subtracting unnecessary scattered light or reflected light on the obstacle surface, the contrast of the tissue image of the observation object that should be observed can be improved.

  In the image processing apparatus according to the present embodiment, a tissue image of an observation target is captured by using a hyperspectral sensor. A hyperspectral sensor is a spectral imaging characterized by spectrally decomposing into multiple channels (usually 30 or more) for each pixel and registering a vector quantity composed of the output intensity of the multiple channels as the spectral intensity of the pixel. It is a sensor. The channel here is a wavelength region having a sufficiently narrow wavelength width with respect to the wavelength region to be observed. By using the hyperspectral sensor, the spectral characteristics of the observation object can be measured with high spatial and wavelength resolution.

  FIG. 2 is a configuration example used in the case of measurement for a small animal in the image processing apparatus according to the present embodiment. An observation object 22 covered with an obstacle layer 21 is arranged on the moving table 13. Above these, the hyperspectral sensor 11 is located. The hyperspectral sensor 11 is connected to the PC 12. Light from the light source 14 is incident on the observation object 22 covered by the obstacle layer 21. The light reflected from the obstacle layer 21 and the observation object 22 is observed by the hyperspectral sensor 11.

  In the case of the arrangement as shown in FIG. 2, the obstacle layer 21 and the observation object 22 are fixed on the moving table 13, so that the wavelength information and the light intensity are observed by moving the moving table 13. The information is combined to form an image. This configuration is not limited to this, and when observing a large animal tissue or a human tissue, a two-dimensional image of a stationary object is moved by moving an observation device attached to the hyperspectral sensor 11. It is set as the structure which obtains. There is no substantial difference in operation between these two configurations.

  In the image processing apparatus according to the present embodiment, the calculation processing method is changed by the PC 12 repeating learning before obtaining desired image data. Learning here refers to acquiring imaging data of an object (learning object) whose image data is known in advance by the hyperspectral sensor 11, inputting the imaging data to the imaging data input unit of the PC 12, and this imaging data. Is compared with the teacher data that is the image data of the learning object acquired in advance by the image data comparison unit, and according to the comparison result, the arithmetic processing learning output by performing the arithmetic processing from the teacher data and the imaging data The image data calculation processing unit changes the calculation processing method so as to reduce the difference from the image data.

  Here, the PC 12 having an imaging data input unit, an image data calculation processing unit, and an image data comparison unit will be described. FIG. 3 is a block diagram schematically illustrating the configuration of the imaging data input unit, the image data calculation processing unit, and the image data comparison unit in the PC 12.

  The imaging data 201 of the learning object from the hyperspectral sensor 11 is stored in the main memory 122 via the I / O interface 124. Further, the imaging data 201 is also recorded in the nonvolatile storage device 123 for later use. In addition, the non-volatile storage device 123 stores image data of a learning object acquired in advance.

  As an interface for capturing image data from the hyperspectral sensor 11, a capture board is typically used. A DRAM can be typically used as the main memory 122 and a hard disk drive (HDD) can be typically used as the non-volatile storage device 123.

  A CPU 121 is mounted in the PC 12. The CPU 121 operates according to the command of the program stored in the non-volatile storage device 123, thereby learning and changing the arithmetic processing method for generating a perspective image of the observation target covered by the obstacle. Execute each process. The program is loaded from the nonvolatile storage device 123 into the main memory 122 and executed by the CPU 121.

  Further, the CPU 121 can be an image data calculation processing unit 125 or an image data comparison unit 126 depending on a program. The imaging data 201 of the learning target stored in the main memory 122 via the I / O interface 124 is read out to the image data arithmetic processing unit 125. At this time, the imaging data 201 of the learning object is processed into arithmetic processing learning image data 202 by the image data processing unit 125, and the arithmetic processing learning image data subjected to the data processing is stored in the main memory.

  Further, the CPU 121 reads and stores the teacher data 203 stored in the nonvolatile storage device 123 into the main memory 122. Further, the image data comparison unit 126 reads and compares the teacher data 203 and the arithmetic processing learning image data 202 stored in the main memory 122. The comparison result is transmitted to the image data calculation processing unit 125, and in response to the comparison result, the image data calculation processing unit 125 performs calculation processing learning image data 202 output from the teacher data 203 and the imaging data. The calculation processing method has been changed so as to reduce the difference. This program can be stored in a recording medium such as a CD-ROM or DVD and installed in each computer.

  Although not shown in the figure, the PC 12 is mounted with a controller connected to the CPU 122 and the main memory 122 via a bus, a mouse and keyboard for inputting data into these, a display for image output, Connect the printer. In order to communicate with another computer, the PC 12 can be connected to a network via a communication adapter.

  FIG. 4 shows a photographic image (FIG. 4 (a)) in the visible light region when paper is assumed as an example of the observation object 22 to be observed through the blood layer as an example of the obstacle layer 21, and at that time. The wavelength dependence of the reflected light intensity observed by the hyperspectral sensor in two pixels of one image is shown (FIGS. 4B and 4C). 4B and 4C, the vertical axis represents the reflected light intensity, and the horizontal axis represents the wavelength of incident light on the observation object 22 covered with the obstacle layer 21. The two pixels at this time are taken from one pixel of the part where the blood layer covers the paper and one from the pixel of the part of the paper where the blood layer is not located.

  Looking at the wavelength dependence of the reflected light intensity from these two pixels, as shown in FIGS. 4B and 4C, in the pixel of the paper portion where the blood layer is not located, 900 nm to 1700 nm. The reflected light intensity in the infrared light region is very strong. On the other hand, in the pixel where the blood layer covers the paper, the reflected light intensity is in the region of 1100 nm to 1300 nm, but in the other region, the pixel of the paper portion where the blood layer is not located Comparison shows that there is almost no reflected light.

  In other words, in order to observe the paper portion located under the blood layer, the reflected light intensity at the pixel of the portion where the blood layer covers the paper and the pixel of the paper portion where the blood layer is not located is as shown in FIG. Since it is shown as (b) and (c), it can be understood that the calculation processing is preferably performed so as to cancel the data of 1100 to 1300 nm infrared light affected by the reflected light.

  As a means for obtaining an obstacle transmission image, an output image can be obtained by a linear sum of intensities in a plurality of wavelength regions. Each coefficient in the linear sum at this time includes a negative coefficient in order to cancel the influence of reflected light and scattered light from the obstacle layer. In addition, when the tissue image of the observation object has a specific reflectance distribution, it is expected that a noise removal effect is enhanced and a high-contrast image can be obtained by adding a nonlinear calculation process.

  As described above, in order to minimize the influence of the obstacle layer and clearly observe the tissue image of the observation object under the obstacle, the optical layers such as absorption, scattering, and surface reflection of the obstacle layer are used. Considering the characteristics and the reflection spectrum characteristics of the object, a calculation process between the reflected light intensities observed at each wavelength is required. Since the optimum arithmetic processing method varies depending on the obstacle, the object, and the lighting condition, it is desirable that the arithmetic processing method is adaptively changed according to the obstacle and the object.

  In the above arithmetic processing method, the most frequently used multi-layer perceptron of a hierarchical network, which is a network in which units are arranged in a plurality of layers and only one-way coupling from the input layer to the output layer is allowed Should be used.

  Here, the multilayer perceptron will be described. In order to describe the multilayer perceptron, a simple perceptron that is the original form of the multilayer perceptron will be described. FIG. 5 shows a calculation mechanism between wavelengths by a simple perceptron having as input the spectral components of the reflectance measured at each pixel of the observed image.

  In a simple perceptron, by inputting each spectral component in a given pixel to the input layer 32 in which the input unit 31 is arranged, the calculation mechanism 33 combines the input data and the bias 34, so that A binary image 35 is obtained. At this time, it is necessary to optimize the variable load applied to each spectral component. In the image processing apparatus according to the present embodiment, prepare a tissue image that is known in advance, and image a thing that the obstacle layer covers on the tissue image that is known in advance by a hyperspectral sensor, The variable weight is optimized by comparing a binary image 35 output from the calculation mechanism 33 with a binary image obtained from a previously known tissue image.

The input value given to this calculation mechanism is “reflectance” x _i (i = 1,..., M) obtained by dividing the spectral intensity of each pixel obtained from the hyperspectral sensor by the reflection intensity of the standard white plate. Shall be used. The subscript i represents each channel (corresponding to a wavelength) of the hyperspectral sensor.

The output y of the simple perceptron is expressed by the following equation, where w _i is a variable load.
g represents a nonlinear function representing binarization, and b represents a bias value. Here, optimization of variable weight is performed. In the optimization of variable weight, first, a tissue image with a known tissue image is prepared, and an obstacle layer covering the previously known tissue image is imaged with a hyperspectral sensor, and the calculation mechanism 33 is obtained. Is performed by minimizing the difference between the binary image 35 output from the above and a binary image obtained from a previously known tissue image.

As a method for reducing this difference, it is preferable to use the steepest descent method, which is the most basic and simple method of obtaining the optimum parameter value by repeatedly updating the parameter starting from an appropriate initial parameter. . That is, in order to minimize the above difference by first giving a random number in the initial state, the variable weight is gradually changed to an optimum value by repeatedly using the learning rule defined by the following equation. Go.
Here, o is a value of an actual network output signal, t is a value obtained by converting a known tissue image into a binary image, and η is a learning coefficient.

  A multilayer perceptron is a network in which the above simple perceptrons are connected in layers. FIG. 6 shows the inter-wavelength calculation processing by the multilayer perceptron. In the multilayer perceptron, a concealment layer 39 in which concealment units 38 are arranged exists between an input layer 32 in which input units 31 are arranged and an output layer 37 in which output units 36 are arranged. At this time, the data conversion from the input layer 32 to the concealment layer 39 and the data conversion from the concealment layer 39 to the output layer 37 use the same method as the above-mentioned simple perceptron.

  When a multilayer perceptron is used, the pixel values are coded in the output layer 37 so that not only a binary image but also a gray scale image can be handled as an output image. As this code format, T.I. O. The method of discrete thermometer encoding proposed by Jackson (Fiesler and R. Beale eds., Handbook of Neural Computation, B4.4, IOP pub. Ltd and Oxford Univ. Press) is used.

Similar to the simple perceptron, the data input to the input layer 32 is “reflectance” x _i (i = 1,...) Obtained by dividing the spectral intensity of each pixel obtained from the hyperspectral sensor by the reflection intensity of the standard white plate.・ ・, M) shall be used. As described above, arithmetic processing between the spectral components of each pixel is performed.

  FIG. 7 shows a flowchart of the learning method in the multilayer perceptron. First, a learning object and a wavelength band for measurement are manually selected (S101). Next, the data added to the input layer of the multilayer perceptron is normalized so that the maximum value and the minimum value are within a certain range (S102). Simultaneously with normalization, a signal to be presented to the output layer side of the perceptron as a signal to be added to the output layer 37 of the multilayer perceptron at the time of learning is coded into an image without an obstacle (S103). This encoded signal is input to the output layer 37, and the inter-layer load included in the multilayer perceptron is optimized by using the error back propagation method added with the moment term published by Rumelhart et al. (S104). .

  By using the learning method as described above, the multilayer perceptron in which the error is sufficiently reduced is used for image fluoroscopy in a combination of the object and the obstacle while fixing the load. Since the output obtained from the multilayer perceptron is coded, it is displayed by converting it into a normal gray scale image by inverse conversion of the coding. Thus, by using a multilayer perceptron, the influence of the noise component contained in an image and scattered light can be reduced, and a clear tissue image of an observation object can be obtained.

FIG. 8 ( b ) shows the observation of the paper characters located under the obstacle layer through the blood layer as the obstacle layer using the image processing apparatus according to the present embodiment. In the image processing apparatus at this time, the infrared spectrum for 150 channels is captured by the hyperspectral sensor, and non-linear calculation processing is performed between the infrared spectra for 150 channels, thereby printing characters “< 5 ”can be clearly confirmed.

As the compared result and, seen through the same obstacles the most permeable good single wavelength 1260nm in the infrared light region, shows an image in which the contrast in FIG. 8 (a). However, as shown in FIG. 8 ( a ), the printed character “5” under the blood layer that cannot be seen with the naked eye cannot be clearly seen. From this, it can be seen that the output result using the image processing apparatus according to the present embodiment is clear.

  9A to 9D and FIG. 10A show the output image results obtained when the number of learnings is changed from 1 to 2000 using the image processing apparatus according to the present embodiment. ) To (d). The numbers of learning in FIGS. 9A to 9D are 1 time, 67 times, 138 times, and 214 times, respectively. The numbers of learning in FIGS. 10A to 10D are 294 times, 383 times, 563 times, and 2000 times, respectively. It can be seen that the larger the number of learning times, the clearer the image. That is, the image processing apparatus according to the present embodiment preferably has a learning count of 300 times or more.

  It should be noted that the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the present invention. The method for automatically optimizing arithmetic processing between data is not limited to the multilayer perceptron.

Embodiment 2. FIG.
In the image processing apparatus according to the present embodiment, without using a multilayer perceptron, by selecting a plurality of wavelength regions according to the obstacle layer and the observation object, by combining the spectra of the selected wavelength regions, A fluoroscopic image of the observation object located under the obstacle layer is obtained. Components and operating principles similar to those of the first embodiment are omitted.

  In FIG. 11, as an example, when paper is assumed to be observed through a blood layer that is an obstacle layer, the observation object is observed with a hyperspectral sensor and optimized with a simple perceptron. The load values connected from the input terminal to the output element are shown in the order of the input spectrum.

  In FIG. 11, this load value distribution is approximated by four Gaussian distribution curves. In the case of this example, the spectrum of 990 to 1040 nm, 1040 nm to 1130 nm, 1120 nm to 1180 nm, 1230 nm to 1280 nm is measured, and the spectrum data is subjected to arithmetic processing by the load value distribution of the Gaussian distribution curve, The tissue image of the observation object can be observed.

  FIG. 12 shows a schematic diagram of an apparatus for observing spectra in these four wavelength bands. At this time, the observation object is irradiated from the light source, and light is reflected from the observation object. Half of the reflected light passes through the half mirror HM1, and half of the light is reflected. The spectrum of the light that has passed through the half mirror HM1 is narrowed to light of a desired wavelength band by the band-pass filter F1, and is observed by the CCD camera CCD1.

  Half of the light reflected by the half mirror HM1 passes through the half mirror HM2, and half of the light is reflected. The spectrum of the light that has passed through is narrowed to light of a desired wavelength band by the band-pass filter F2, and is observed by the CCD camera CCD2.

  Similarly, half of the light reflected by the half mirror HM2 passes and half of the light is reflected by the half mirror HM3. The passed light is narrowed in spectrum by the bandpass filter F3 to light of a desired wavelength band, and is observed by the CCD camera CCD3. The light reflected by the half mirror HM3 is narrowed to light of a desired wavelength band by the band pass filter F4 and observed by the CCD camera CCD4.

Here, the case of four wavelength bands is shown as an example, but the number of wavelength bands may be any number. Bandpass filters, CCD cameras, and half mirrors corresponding to the number may be prepared. By doing the above, we get four images in real time and at each pixel
The image is obtained by superimposing the image using. i and j are two-dimensional coordinates representing the pixel position of the image. Further, S _k (k = 1... 4) is the pixel value of the corresponding image position of each CCD, the left side is the output image pixel value, and W _k is a coefficient for adjusting the transmission amount of each bandpass filter. For the two bands having a negative coefficient in, negative arithmetic processing is enabled by making W _k negative.

  13 and 14 are images observed through the blood layer as the obstacle layer using the image transmission device according to the present embodiment. FIG. 13 is a view in which a blood layer as an obstacle layer is covered on the printed paper. FIG. 14 shows an image obtained by opening a guinea pig using the image transmission device according to the present embodiment. It is a thing.

  In FIGS. 13 and 14, (a) is a photograph of the observation object excluding the blood layer in the visible light region, (b) is an image observed using a certain infrared light, (c) These are fluoroscopic images observed using the image transmission device according to the present embodiment. As is apparent from this result, it can be seen that a very clear image is obtained as compared with the case where constant infrared light is used.

  As described above, by selecting a plurality of wavelength bands corresponding to the obstacle layer and the observation object from the infrared light region, by using a method of measuring the spectrum of the selected wavelength band and performing arithmetic processing The arithmetic processing to be performed can be greatly reduced compared to the arithmetic processing performed by using the multilayer perceptron, and the image processing speed can be remarkably improved. For this reason, it becomes possible to easily observe the change in the image of the observation object in real time.

Absorption characteristics of water, hemoglobin, and oxygenated hemoglobin Measurement system of reflected light by hyperspectral sensor FIG. 2 is a block diagram schematically showing the configuration of an imaging data input unit, an image data calculation processing unit, and an image data comparison unit in a PC. (A) Paper and (b) Blood reflected light intensity spectrum Calculation mechanism between each wavelength by simple perceptron Inter-wavelength processing by multilayer perceptron Flow chart in learning image processing system (A) Transmission image by wavelength λ = 1260 nm only (b) Transmission image by image processing apparatus according to Embodiment 1 The transmission image by the image processing apparatus which concerns on Embodiment 1 in the frequency | count of learning. (A) 1 time (b) 67 times (c) 138 times (d) 214 times The transmission image by the image processing apparatus which concerns on Embodiment 1 in the frequency | count of learning. (A) 294 times (b) 383 times (c) 563 times (d) 2000 times Load distribution of simple perceptron Schematic of the image transmission device according to the second embodiment 9 is a transmission image of paper printed by the image transmission device according to the second embodiment. (A) A photograph in the visible light region when the blood layer does not cover the paper. (B) Transmission image by constant infrared light. (C) A transmission image by the image transmission device according to the second embodiment. The transmission image of the site | part which the guinea pig opened by the image transmission apparatus which concerns on Embodiment 2. FIG. (A) A photograph in the visible light region when the blood layer is removed from a guinea pig laparotomy site. (B) Transmission image by constant infrared light. (C) A transmission image by the image transmission device according to the second embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 Hyperspectral sensor 13 Movement table 14 Light source 21 Obstacle layer 22 Observation object 31 Input unit 32 Input layer 33 Calculation mechanism 34 Bias 35 Binary image 36 Output unit 37 Output layer 38 Concealment unit 39 Concealment layer 121 CPU 122 Main memory 123 Nonvolatile storage device 124 I / O interface 125 Image data arithmetic processing unit 126 Image data comparison unit 201 Imaging data 202 Arithmetic processing learning image data 203 Teacher data

Claims (10)

An image processing apparatus that determines by learning an arithmetic processing method for generating a fluoroscopic image of an observation object covered by an obstacle,
Imaging data of the obstacle and the learning object covered by the obstacle is acquired by infrared light of a plurality of wavelengths selected based on the transmission and absorption characteristics of the obstacle and the reflection characteristics of the observation object. An imaging data input unit for inputting
An image data calculation processing unit that generates fluoroscopic image data of the learning object by performing linear calculation and / or nonlinear calculation between wavelengths for infrared light having a plurality of wavelengths that have passed through the obstacle;
A data comparison unit that compares the fluoroscopic image data of the learning object and the reflectance data of the learning object acquired in advance;
Based on the comparison data from the data comparison unit, the image data calculation processing unit has a difference in reflectance between corresponding points between the fluoroscopic image data and the reflectance data of the learning object acquired in advance. An image processing apparatus that changes a calculation processing method of the imaging data so as to be small. The image processing apparatus further includes a test fat,
The hyperspectral sensor capable of dividing the image measured by the detection unit for each pixel and measuring wavelength information and light intensity information for each pixel at the same time is used as the detection unit. Image processing apparatus. The image processing apparatus further includes a test fat,
The detection unit includes a band pass filter for measuring a plurality of wavelength bands of the infrared light selected based on reflected light from the obstacle and the observation object. The image processing apparatus according to claim 2. The image processing apparatus according to claim 1, wherein the calculation uses a multilayer perceptron. The image processing apparatus according to claim 1, wherein a simple perceptron is used for the calculation. The image processing apparatus according to any one of claims 1 to 5, wherein a steepest descent method is used as a method of reducing the difference.   The image according to claim 1, wherein the image data calculation processing unit generates perspective image data of the learning object by performing nonlinear calculation between wavelengths for infrared light having a plurality of wavelengths that have passed through the obstacle. Processing equipment. Imaging data of the obstacle and the learning object covered by the obstacle is acquired by infrared light of a plurality of wavelengths selected based on the transmission and absorption characteristics of the obstacle and the reflection characteristics of the observation object. Step to enter,
Generating perspective image data of the learning object from the imaging data;
Comparing the fluoroscopic image data of the learning object with the reflectance data of the learning object acquired in advance;
In accordance with the result of the comparison, the imaging data so that the difference in reflectance of corresponding points between the fluoroscopic image data of the learning object and the reflectance data of the learning object acquired in advance is reduced. Changing the processing method of
A perspective image of an observation object covered by the obstacle by performing linear computation and / or nonlinear computation between wavelengths for infrared light having a plurality of wavelengths that have passed through the obstacle using the computation processing method. Generating an image processing method. The image processing method according to claim 8, wherein the calculation uses a multilayer perceptron. The image processing method according to claim 8, wherein the calculation uses a simple perceptron.