JP2018101889A - Imaging apparatus and imaging control program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve the problem in which, generally, a reflection and transmission spectrum of a substance in a near-infrared band has temperature dependency, and, even for the same observation object, if a temperature at the time of shooting is different, impression of a generated converted color image may be significantly different.SOLUTION: An imaging apparatus includes: an imaging unit having an imaging device having sensitivity in a near-infrared band, for imaging an object to be observed and outputting near-infrared imaging data; a temperature measuring unit for measuring a temperature of the object to be observed; and an image processing unit for converting the near-infrared imaging data into color image data by associating a pixel value of the near-infrared imaging data with a visible band. The image processing unit, on the basis of a temperature difference between a measured temperature measured by the object to be observed and a reference temperature, adjusts a correspondence relationship between the pixel value of the near-infrared imaging data and the visible band.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an imaging apparatus and an imaging control program.

An imaging system is known in which three primary colors (RGB) are assigned to each of three invisible wavelength bands having different center wavelengths.
[Prior art documents]
[Patent Literature]
[Patent Document 1] International Publication No. WO2007 / 083437

  In general, the reflection / transmission spectrum in the near-infrared band of a substance is temperature-dependent, and even if it is the same observation object, if the temperature at the time of shooting is different, the impression of the generated converted color image will differ greatly. May end up.

  The imaging device according to the first aspect of the present invention has an imaging device having sensitivity in the near-infrared band, images an observation object and generates near-infrared imaging data, and the temperature of the observation object A temperature measuring unit that measures the pixel value of the near-infrared imaging data and an image processing unit that converts the near-infrared imaging data into color image data by associating the pixel value of the near-infrared imaging data with the visible band. Based on the temperature difference between the measured temperature measured by the object and the reference temperature, the correspondence between the pixel value of the near-infrared imaging data and the visible band is adjusted.

  The imaging control program according to the second aspect of the present invention includes an imaging step of imaging an observation object to generate near-infrared imaging data, a temperature measurement step of measuring the temperature of the observation object, and the near-infrared imaging data. An image processing step for converting the near-infrared imaging data into color image data by adjusting the correspondence between the pixel value of the near-infrared imaging data and the visible band based on the temperature difference between the measured temperature and the reference temperature; Is executed on the computer.

  It should be noted that the above summary of the invention does not enumerate all the necessary features of the present invention. In addition, a sub-combination of these feature groups can also be an invention.

It is a figure explaining the structure of an imaging device. It is a figure explaining the band pass filter arrange | positioned on each photoelectric conversion element of a 1st image pick-up element. It is a figure explaining the characteristic of an optical band pass filter. It is a figure explaining the temperature dependence of the transmission spectrum in an observation object. It is a figure explaining the change of the color image with respect to temperature. It is a figure explaining the example of a correction table. It is a figure explaining the concept of the adjustment process of a color image. It is a figure explaining the processing flow of an imaging device.

  Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. In addition, not all the combinations of features described in the embodiments are essential for the solving means of the invention.

  FIG. 1 is a diagram illustrating the configuration of the imaging apparatus 100 according to the present embodiment. The imaging apparatus 100 can acquire a near-infrared image and a far-infrared image with one observation target as a subject. The near-infrared image is an output signal that the first image sensor 102 photoelectrically converts and outputs a near-infrared subject light beam that passes through the first optical system 101 and forms an image on the light receiving surface of the first image sensor 102. To be acquired. The far-infrared image is an output signal that the second image sensor 112 photoelectrically converts and outputs a far-infrared subject light beam that passes through the second optical system 111 and forms an image on the light receiving surface of the second image sensor 112. To be acquired.

The first image sensor 102 is an image sensor in which photoelectric conversion elements having sensitivity in a range of, for example, 700 nm to 2500 nm as a near-infrared band are two-dimensionally arranged, and the first optical system 101 has an optical axis op ₁ . The subject luminous flux in this band incident along is condensed. In the figure, the first optical system 101 is represented by a single lens for the sake of convenience, but it may of course be configured by a plurality of lenses. The first optical system 101 may have an autofocus function. Details of the first image sensor 102 will be described later. Note that the wavelength range of near-infrared light is an example and is not limited thereto.

  The output signal output from the first image sensor 102 is processed by the image processing unit 205 under the control of the control unit 201 and converted to near-infrared imaging data. That is, the first optical system 101, the first imaging element 102, the control unit 201, and the image processing unit 205 function as the imaging unit 120 that generates near-infrared imaging data.

The second image sensor 112 is an image sensor in which photoelectric conversion elements having sensitivity in a range of, for example, 8000 nm to 15000 nm as a far-infrared band are two-dimensionally arranged, and the second optical system 111 has an optical axis op ₂ . The subject luminous flux in this band incident along is condensed. In the figure, the second optical system 111 is represented by a single lens for the sake of convenience, but it may of course be configured by a plurality of lenses. The second optical system 111 may have an autofocus function. The wavelength range of far-infrared light is an example and is not limited to this.

  The output signal output from the second image sensor 112 is processed by the image processing unit 205 under the control of the control unit 201 and converted into temperature data. Since the pixel value of the far-infrared image is proportional to the temperature, the image processing unit 205 calculates the temperature of the object to be observed from each pixel value that is the output signal, and uses the temperature data measured in a two-dimensional manner. Generate. That is, the second optical system 111, the second imaging element 112, the control unit 201, and the image processing unit 205 function as a temperature measurement unit 130 that measures the temperature of the observation object.

  In the present embodiment, the imaging unit 120 and the temperature measuring unit 130 are described as sharing the control unit 201 and the image processing unit 205, but each may include a control unit and an image processing unit. In this case, image processing to be described later may be performed by one image processing unit, or may be configured to be processed by a separate image processing unit. Further, the control unit and the image processing unit may be configured as one ASIC.

The first optical system 101 and the second optical system 111 are arranged so that each of the first image sensor 102 and the second image sensor 112 can capture one observation object as a subject. At this time, the parallax due to the optical axes op ₁ and op ₂ is preferably as small as possible. Therefore, a parallax adjustment mechanism that reduces parallax, for example, shifts the optical axis in accordance with the distance to the object to be observed may be provided.

  The imaging apparatus 100 includes a control unit 201, a system memory 202, a work memory 203, an image processing unit 205, a communication unit 206, a display unit 207, and an operation unit 208. The control unit 201 comprehensively controls the entire imaging apparatus 100. Control by the control unit 201 is executed according to a control program stored in a system memory that is a nonvolatile memory.

  In addition to the control program, the system memory 202 also stores various parameter values used for image processing and user interface screen data displayed on the display unit 207. In addition, a correction table 212 used for color conversion processing to be described later is also stored. The stored information is appropriately read out by the control unit 201 and developed in the work memory 203 for use.

  The work memory 203 is a volatile memory in which information referred to by the control unit 201 or the like is expanded. In addition to the role as a storage area in which the reference information is developed, it also serves as a work area for image processing calculation by the image processing unit 205, for example.

  As described above, the image processing unit 205 converts the output signal output from the first imaging element 102 into near-infrared imaging data, and converts the output signal output from the second imaging element 112 into temperature data. Then, the near-infrared imaging data is converted into a color image, and processing for correcting the color image and generating color image data with reference to the temperature data and the correction table 212 is executed. In addition, a file conversion process is performed in which information associated with the color image data is associated and converted into a file format in accordance with a predetermined format.

  The communication unit 206 has a communication function of outputting the color image data file generated by the image processing unit 205 to an external device and inputting necessary information from the external device. For example, an interface such as USB or LAN.

  The display unit 207 is, for example, a liquid crystal display that displays the generated color image or displays a user interface screen. The display unit 207 may be configured to be connected to the communication unit 206 as an external device independently of the imaging device 100.

  The operation unit 208 includes an operation member that receives a user operation and is configured by a device such as a touch panel provided to be superimposed on the display unit 207. The operation unit 208 detects the operation of the operation member and transmits a detection signal to the control unit 201.

FIG. 2 is a diagram illustrating a bandpass filter disposed on each photoelectric conversion element of the first image sensor 102. Referring to an enlarged view in which a part of the first image sensor 102 is enlarged, like a so-called Bayer array, the upper left photoelectric conversion element p ₁ , the upper right photoelectric conversion element p ₂ , the lower left photoelectric conversion element p ₃ , the lower right photoelectric conversion element It can be seen that a group of photoelectric conversion elements each including _four of the photoelectric conversion elements p4 is arranged two-dimensionally. Top left on the photoelectric conversion element p ₁ is, N ₁ filter is arranged an optical bandpass filter for limiting the wavelength band of the light flux incident, likewise, on the photoelectric conversion element p ₂ has N ₂ filter , on the photoelectric conversion element p ₃ is N ₂ filter, on the photoelectric conversion element p ₄ are arranged N ₃ filter. By disposing different types of optical bandpass filters on the photoelectric conversion elements, the light receiving sensitivity is partially limited for each photoelectric conversion element in the range of 700 nm to 2100 nm of the photoelectric conversion element itself. That is, the first image sensor 102 performs photoelectric conversion by separating the subject image formed on the light receiving surface into three different wavelength bands in the near-infrared band.

  FIG. 3 is a diagram illustrating an example of the characteristics of the optical bandpass filter. The horizontal axis indicates the wavelength [nm], and the vertical axis indicates the transmittance [%].

As shown in the figure, the convex shapes of the transmittances of the N ₁ filter, the N ₂ filter, and the N ₃ filter are approximately the same. The N ₁ filter has a peak wavelength λ ₁ of 1150 nm and an effective transmission band of 700 nm to 1550 nm. The N ₂ filter has a peak wavelength λ ₂ of 1400 nm and an effective transmission band of 950 nm to 1800 nm. The N ₃ filter has a peak wavelength λ ₃ of 1650 nm and an effective transmission band of 1250 nm to 2100 nm. That is, the N ₁ filter, the N ₂ filter, and the N ₃ filter are set so that the transmission bands overlap. The transmittance shape of each filter may be different. Specifically, the transmission band width and the peak wavelength transmittance may be different for each filter.

When the output signals from the first image sensor 102 having such a configuration are classified and collected according to which filter has passed through, the near infrared as a subject image corresponding to the pass band of each filter is collected. Three images are obtained. That is, the near-infrared image _I 1, _{N 2} near-infrared image corresponding to the filter _I 2, _{N 3} near-infrared image _{I 3} corresponding to the filter corresponding to _{N 1} filter is obtained. In addition, about the coordinate from which a signal is not obtained because other filters are arrange | positioned, it interpolates using the signal of a surrounding coordinate.

Here, a color conversion process for converting a near-infrared image into a color image will be described. This is a process of visualizing a near-infrared image that is not originally recognized by the human eye, and is executed by the image processing unit 205. Specifically, the near-infrared image I ₁ is regarded as an image representing the luminance distribution in the blue wavelength band, the near-infrared image I ₂ is regarded as an image representing the luminance distribution in the green wavelength band, and the near-infrared image I ₃ is It is regarded as an image representing the luminance distribution in the red wavelength band. When these are superimposed, the R value, G value, and B value are aligned in each pixel, and a single color image is obtained. That is, the image processing unit 205 associates the pixel value in each pixel of the near-infrared images I _{1 to} I ₃ with the blue, green, and red wavelength bands, which are visible bands, so that a near-infrared image is displayed. Infrared imaging data is converted into a color image. Thus, since it can display on the display part 207 if it visualizes in a color image, the user can grasp | ascertain intuitively the infrared information of an observation target object through vision. Here, when the transmission bands of the N ₁ filter, N ₂ filter, and N ₃ filter overlap as in this embodiment, a plurality of colors of R value, G value, and B value that are associated with each filter are displayed. Since the combined colors can be displayed, it is possible to express in more colors. The correspondence relationship between the near-infrared image (I1, I2, I3) and the visible image (R, G, B) can be freely combined without being restricted by the length of the wavelength. The visible band is, for example, a range from 380 nm to 780 nm, but is not limited to this range as long as it can be recognized by the human eye. Furthermore, the color associated with the visible band is not limited to blue, green, and red, and other colors can be associated.

  FIG. 4 is a diagram for explaining the temperature dependence of the transmission spectrum in the observation object. The horizontal axis indicates the wavelength [nm], and the vertical axis indicates the transmittance [%]. FIG. 4 schematically shows a transmission spectrum of water as an observation object.

Each convex shape represented by a dotted line indicates the filter characteristics of the N ₁ filter, N ₂ filter, and N ₃ filter described in FIG. The transmission spectrum W ₁ shown by the solid line represents the transmission spectrum of water at about 0 ° C., the transmission spectrum W _m represents the transmission spectrum of water at about 50 ° C., and the transmission spectrum W _h represents the transmission of water at about 100 ° C. The spectrum is schematically shown.

  As shown in the figure, the distribution of the transmission spectrum of water shifts to the short wavelength side as the temperature changes from low temperature to high temperature. Therefore, for example, when tomato is imaged as an observation object, when converting a near-infrared image to a color image as described above, if the same tomato is imaged but the tomato is cold, the overall bluish color When an image is generated and, conversely, warm, a reddish color image is generated.

  For example, when it is desired to continuously capture a plurality of tomatoes carried on a belt conveyor and visually grasp the water content of each tomato, if the tomatoes are at the same temperature, the color of the generated color image Differences in taste can be grasped as differences in water content. In particular, if irradiation light having an intensity change near 1400 nm is irradiated in accordance with the characteristics of the water transmission spectrum, the difference in color appears more remarkably, so that it becomes easier to grasp intuitively. However, if there is a difference in the temperature of each tomato, it will be difficult to judge whether the difference in color is due to the difference in the amount of water contained in the tomato or the shift in the transmission spectrum.

  Therefore, the image processing unit 205 performs color conversion processing for adjusting the color of the color image in order to reduce the color difference due to the shift of the transmission spectrum. FIG. 5 is a diagram for explaining the change of the color image with respect to the temperature.

  Of the three image schematic diagrams shown in the drawing, the upper image schematic diagram shows a state of a color image obtained by capturing an image of a tomato as an observation object in a state of 60 ° C. Similarly, the middle schematic image shows a color image obtained by imaging the same tomato at 20 ° C., and the lower schematic image is also obtained by imaging at 0 ° C. Shows the state of the color image. The density of each hatching represents, for example, the redness. That is, the color image at 60 ° C. is reddish, and the color image at 0 ° C. represents a state in which the color appears to be relatively bluish with little redness.

  If the reference temperature in evaluating the moisture content is set to 20 ° C., the image processing unit 205 performs color conversion processing as image processing after imaging both the upper color image and the lower color image, Adjustment is performed so that a color image having a color as close as possible to the middle color image can be obtained. The reference temperature may not be 20 ° C. and may be determined as appropriate. If the reference temperature is set to 60 ° C., the image processing unit 205 adjusts the middle color image and the lower color image so that a color image having a color as close as possible to the upper color image can be obtained.

  Specifically, the image processing unit 205 executes color conversion processing with reference to the correction table 212. FIG. 6 is a conceptual diagram illustrating an example of the correction table 212. The actual correction table is defined by a predetermined data structure as a lookup table, but here, the meaning is conceptually represented.

  When the user wants to compare the moisture contents of a plurality of observation objects, the user operates the operation unit 208 to designate “water”. The control unit 201 receives the designation, and reads the “water” lookup table from the correction table 212 stored in the system memory 202.

The water look-up table is prepared with a reference temperature of 20 ° C. That is, if the object to be observed is 20 ° C., color conversion is not performed, and if it is a temperature other than 20 ° C., a correction value is prepared so that color conversion is performed according to the temperature difference. Specifically, as shown in the figure, the temperature difference with respect to 20 ° C. is prepared from + 80 ° C. to −30 ° C. in increments of 2 ° C. In each temperature difference, a correction value for the pixel value (R value) of the R pixel, a correction value for the pixel value (G value) of the G pixel, and a correction value for the pixel value (B value) of the B pixel are defined. . If the temperature difference is n ° C., the R value correction value is R _n , the G value correction value is G _n , and the B value correction value is B _n . For example, when it is defined that “R ₊₂₅ = −6” as the correction value of the R value when the temperature difference is + 25 ° C., if the R value obtained by imaging is “129”, The R value is “123” (= 129−6). When such a calculation is performed on all the pixels, a color image close to the color when captured at 20 ° C., which is the reference temperature, can be obtained. That is, the image processing unit 205 first calculates the temperature difference between the measured temperature and the reference temperature based on the temperature data generated by the temperature measuring unit 130. Based on the temperature difference, the correspondence between the pixel value of the near-infrared imaging data and the visible band is adjusted. In the present embodiment, the correspondence between the pixel value and the visible band indicates how much each pixel has an RGB value in the visible band.

The correction value for each temperature difference is determined through experiments and the like. In the example of water in FIG. 6, the reference temperature is 20 ° C., but other temperatures may be used as described above. Further, the upper and lower limits of the temperature difference for preparing the correction value can be arbitrarily set. When a temperature difference exceeding the upper limit and the lower limit is detected, error processing such as displaying a warning on the display unit 207 may be executed. The temperature difference between the upper limit and the lower limit may be determined according to the characteristics of the transmission spectrum of “water” as the object. For example, it can be limited to a temperature difference within a range where it can be evaluated that a change due to temperature is shifted while maintaining the overall shape with respect to the shape of the transmission spectrum at the reference temperature. Specifically, when matching is performed on the shape at the reference temperature, it can be evaluated that the shape is maintained when the degree of coincidence is equal to or greater than a threshold value. Further, it may be determined according to the characteristics of the optical bandpass filter described with reference to FIG. For example, the shift amount that shifts until the peak wavelength of the transmission spectrum matches the peak wavelength of the N ₁ filter is defined as the upper limit temperature difference, and the shift amount that shifts until it matches the peak wavelength of the N ₃ filter is defined as the lower limit temperature difference.

  Further, the increment of the temperature difference is not limited to 2 ° C. Other temperatures may be used, and the temperature difference may be dynamically changed on the basis that the correction value changes by “1”. Each of the R pixel correction value, the G pixel correction value, and the B pixel correction value with respect to the temperature difference may be defined by a function. Further, the correction value for each temperature difference may be set or updated by machine learning by machine learning. As teacher data used for machine learning, a correction value manually corrected by a user or data acquired from another device or database via a network can be used.

  In addition to the lookup table for water, lookup tables for various substances can be prepared. In the example of FIG. 6, a lookup table for “ethanol” and “microcrystalline cellulose” is prepared in addition to water. That is, a lookup table for a substance may be prepared depending on which substance content the user observes. Conversely, if the imaging device 100 is a dedicated device for recognizing the difference in water content, only a water lookup table is required.

  In order to perform the color conversion described above, it is necessary to obtain a temperature difference with respect to the reference temperature. In addition, since the temperature distribution of the observation target object may be different for each part, it is desirable to acquire temperature difference information for each region on the image corresponding to the part of the observation target object. Therefore, the imaging apparatus 100 includes a temperature measurement unit 130. FIG. 7 is an explanatory diagram for explaining the concept of the adjustment process until obtaining an adjusted color image from the temperature data obtained from the temperature measurement unit 130 and the near-infrared image obtained from the imaging unit 120.

  It is desirable that the number of temperature measurement points based on the number of pixels in the far-infrared image and the number of pixels in the near-infrared image match, but here the case where the number of temperature measurement points is less than the number of pixels in the near-infrared image explain. That is, although the first image sensor 102 and the second image sensor 112 capture the observation object at the same angle of view, the number of output signals from the second image sensor 112 is greater than the output signal from the first image sensor 102. This is the case when it is less than the number.

  In the example of FIG. 7, the observation object is a tomato, and the background other than the tomato is excluded from processing. The non-target region may be uniformly black in the color image, for example.

In the temperature data, the whole tomato is represented by a plurality of measurement points represented by rectangles corresponding to each pixel of the far-infrared image that is the output signal of the second image sensor 112. Accordingly, each measurement point has an address and a temperature value. If the horizontal direction in the figure is the x direction and the vertical direction is the y direction, the address of each measurement point is represented by (x _i , y _i ). In the illustrated example, there are a measurement point of 22 ° C., a measurement point of 34 ° C., a measurement point of 36 ° C., and a measurement point of 38 ° C. In the figure, the difference in temperature is expressed by the difference in the type of hatching.

  Here, since the number of pixels of the near-infrared image is larger than the number of measurement points of the temperature data, a process for matching these numbers is performed. For convenience of explanation, this process is referred to as a process for matching the image sizes. The process of matching the image size is done by increasing the number of temperature data measurement points by interpolation to match the number of pixels in the near-infrared image, and reducing the number of near-infrared image pixels by thinning-out processing. There is a method to match. The former should be used if accuracy is important, and the latter should be used if calculation speed is important.

  Note that the far-distributed image and the near-infrared image have some parallax as described above, but for simplicity, the corresponding pixels at the same address after the processing for matching the image size are performed on the subject. It can be regarded as a pixel capturing the same part. If more importance is placed on the accuracy, matching processing may be performed using the contour of the entire subject, and the corresponding addresses may be converted to link the pixels together.

Here, the temperature data _{(x i, y i) (} x i, y i) of the measurement point and a near-infrared image of the pixels is regarded the same part of the subject. In the example of the figure, the temperature of the temperature data measurement point (x _i , y _i ) is 36 ° C., and the pixel value of the pixel (x _i , y _i ) of the near-infrared image is (R, G, B) = (R _ij , G _ij , B _ij ). At this time, the temperature difference from the reference temperature of 20 ° C. is calculated as + 16 ° C. The control unit 201 reads the correction value (R ₊₁₆ , G ₊₁₆ , B ₊₁₆ ) of + 16 ° C. with reference to the correction table 212, applies this correction value to (R _ij , G _ij , B _ij ), (R ′ _ij , G ′ _ij , B ′ _ij ) If this operation is performed over all pixels, a corrected color image is obtained.

  The above processing will be described as a processing flow. FIG. 8 is a diagram for explaining the processing flow. The flow starts from a state in which the power of the imaging apparatus 100 is turned on and an instruction as to which content of the substance to be focused on in the observation target is received from the user.

  In step S101, the control unit 201 waits for an imaging instruction from the user. If a user's imaging instruction is detected, the process proceeds to step S102. The imaging instruction is not limited to a user instruction, and for example, an imaging instruction signal may be automatically generated every time a tomato carried on a belt conveyor is detected.

  In step S102, the control unit 201 generates near-infrared imaging data by driving the first imaging element 102 or the like. In step S103, the temperature is measured by driving the second image sensor 112 or the like. Note that the processing order of step S102 and step S103 may be reversed, or may be partially or wholly processed in parallel.

  In step S104, the control unit 201 causes the image processing unit 205 to match the image sizes of the near-infrared image and the temperature data by interpolation processing, thinning processing, or the like. In step S105, a temperature difference between the measurement temperature at the corresponding measurement point and the reference temperature is calculated, and in step S106, the RGB correction values are read with reference to the correction table 212. In step S107, the image processing unit 205 converts the pixel value at each address using the read correction value. When the color conversion process for all the pixels is completed, the color image data whose color is adjusted is completed.

  When the adjusted color image data is completed, the control unit 201 displays the adjusted color image on the display unit 207, and records the completed color image data on an external recording device via the communication unit 206. In step S109, the control unit 201 detects whether the power is turned off. If not detected, the control unit 201 returns to step S101. If it is detected, the series of processing ends.

  Note that the control unit 201 may include temperature data and near-infrared imaging data before adjustment as sub information in the color image data to be recorded. As a format for recording the sub information, an EXIF format thumbnail image, a description in a maker note area, and the like are considered. Alternatively, it may be recorded as a second image in MPF format and Tag information in RAW format. Further, as related information, an adjustment parameter used for the conversion process may be recorded.

  Further, the imaging apparatus 100 may accept the user's selection and output the near-infrared imaging data generated by the imaging unit 120 without performing color conversion processing as it is. At this time, the temperature data generated by the temperature measurement unit 130 may be output as associated information. As described above, if the near-infrared imaging data and the temperature data are output together, the color conversion process can be performed by the external processing device. Therefore, an algorithm different from the algorithm incorporated in the imaging device 100 is applied. Or a more complicated process.

  In addition, the color conversion process according to the above embodiment is a process of shifting each value of RGB, but the color adjustment process is not limited to this. Other parameters may be adjusted to match the color image at the reference temperature. For example, gamma may be adjusted, or saturation and hue may be adjusted independently. In addition, the correction table is not limited to the color adjustment value, and may be temperature change information regarding a change in the transmission spectrum representing the degree of transmission with respect to the wavelength of the observation object. If the adjustment rule for color conversion is established, the information necessary as temperature change information is characteristic information as a substance for each observation object. In this case, the correction table is transmission spectrum information itself. Also good. Further, the correction table is not limited to being stored in the system memory 202, and may be acquired from the communication unit 206 from the outside via a network as necessary.

  In the above embodiment, when the color conversion process is performed, the color is adjusted based on the temperature dependence on the transmission spectrum of the observation object. However, the spectrum indicating the change with respect to the wavelength of the observation object is a transmission spectrum. Not limited to. For example, an absorption spectrum representing the degree of absorption with respect to the wavelength of the observation object may be used. In this case, a correction value based on the temperature dependence on the absorption spectrum of the observation object is prepared in the correction table 212.

  In the above-described embodiment, the color conversion process is performed with a correction table created in advance according to the content of the target substance, aiming more accurately at the color near the reference temperature color image. The color conversion process may be performed to the overall trend level. In other words, even if there is no correction table for each substance, if the detected actual temperature is higher than the reference temperature, it will be adjusted in the red direction, and if the actual temperature is lower than the reference temperature, it will be adjusted in the blue direction. May be performed.

  In the above embodiment, the temperature measurement unit 130 captures a far-infrared image by the second imaging element 112 in which the photoelectric conversion elements are arrayed two-dimensionally, and converts the far-infrared image into two-dimensional temperature data. Although converted, the measurement of temperature data is not limited to this. For example, the temperature measurement unit 130 may measure the temperature of one point of the subject with a point sensor, or may measure the temperature of each point of the subject by arranging a plurality of point sensors in a dot shape. When the temperature measurement unit 130 measures the temperature of one point of the subject, the image processing unit 205 uses the measured temperature for all pixels of the near-infrared image to obtain the reference temperature. Perform corrections due to temperature differences. When the temperature measurement unit 130 measures the temperature of a plurality of points on the subject, the image processing unit 205 performs interpolation processing on the temperature of the portion corresponding to the pixel of the near-infrared image from the plurality of points. Then, the correction based on the temperature difference from the reference temperature is executed using these temperatures.

  In the above embodiment, when the imaging unit 120 captures a near-infrared image, the observation target may be irradiated with light. In this case, the imaging device 100 is provided with an irradiation unit that is electrically connected to the control unit 201. The irradiation unit has a light source and can set the wavelength band of the light to be irradiated. The irradiation unit irradiates the observation target with light in a desired wavelength band according to a command from the control unit 201. When the irradiation unit irradiates light including a wavelength band of a characteristic transmission spectrum corresponding to the observation object, a change in the state of the observation object appears more prominently in the near-infrared image captured by the imaging unit 120. Note that the irradiation unit may irradiate a portion of the observation object facing the imaging unit 120, or may irradiate the back side portion of the observation object.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiment. It is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

  The order of execution of each process such as operations, procedures, steps, and stages in the apparatus, system, program, and method shown in the claims, the description, and the drawings is particularly “before” or “prior to”. It should be noted that the output can be realized in any order unless the output of the previous process is used in the subsequent process. Regarding the operation flow in the claims, the description, and the drawings, even if it is described using “first”, “next”, etc. for convenience, it means that it is essential to carry out in this order. It is not a thing.

DESCRIPTION OF SYMBOLS 100 Image pick-up device, 101 1st optical system, 102 1st image pick-up element, 111 2nd optical system, 112 2nd image pick-up element, 120 Image pick-up part, 130 Temperature measurement part, 201 Control part, 202 System memory, 203 Work memory, 205 Image processing unit, 206 communication unit, 207 display unit, 208 operation unit, 212 correction table

Claims (11)

An imaging unit having an imaging device having sensitivity in the near-infrared band and imaging an observation object to generate near-infrared imaging data;
A temperature measuring unit for measuring the temperature of the observation object;
An image processing unit that associates the pixel value of the near-infrared imaging data with a visible band and converts the near-infrared imaging data into color image data;
The image processing unit is an imaging device that adjusts a correspondence relationship between a pixel value of the near-infrared imaging data and the visible band based on a temperature difference between a measured temperature measured by the observation object and a reference temperature. The temperature measurement unit measures the temperature of the observation object two-dimensionally,
The image processing unit calculates a temperature difference of each imaging region using a measured temperature at a position corresponding to each imaging region of the near-infrared imaging data, and the correspondence in each imaging region based on the temperature difference The imaging apparatus according to claim 1, wherein the relationship is adjusted.   The imaging apparatus according to claim 2, wherein the image processing unit performs an interpolation process to match the number of measurement points of the temperature measured by the temperature measurement unit with the number of pixels of the near-infrared imaging data.   The imaging apparatus according to any one of claims 1 to 3, wherein the image processing unit corrects the correspondence relationship based on the temperature difference after the near-infrared imaging data is once converted into the color image data. . An acquisition unit for acquiring temperature change information about a change with respect to temperature of a transmission spectrum representing a transmittance with respect to a wavelength of the observation object;
The imaging apparatus according to claim 1, wherein the image processing unit adjusts the correspondence relationship based on the temperature change information. A reception unit for receiving material information of the observation object;
The imaging apparatus according to claim 1, wherein the image processing unit adjusts the correspondence relationship based on the substance information.   The image processing unit sets the visible band for associating the pixel value at the reference temperature, and adjusts the visible band to a longer wavelength side when the measured temperature is higher than the reference temperature, and the measured temperature The imaging device according to claim 1, wherein when the temperature is lower than the reference temperature, the visible band is adjusted to a short wavelength side.   The imaging apparatus according to claim 1, further comprising a selection unit that selects whether or not the image processing unit adjusts the correspondence relationship.   The imaging apparatus according to any one of claims 2 to 8, wherein the image processing unit sets the imaging areas in units of pixels.   The imaging apparatus according to claim 1, wherein the image processing unit outputs temperature data of the temperature measured by the measurement unit in association with the near-infrared imaging data. An imaging step of imaging an observation object and generating near-infrared imaging data;
A temperature measuring step for measuring the temperature of the observation object;
Based on the temperature difference between the measured temperature of the near-infrared imaging data and the reference temperature, the correspondence between the pixel value of the near-infrared imaging data and the visible wavelength band is adjusted, and the near-infrared imaging data is An imaging control program for causing a computer to execute an image processing step for converting color image data.

Images