JP4742883B2 - Far-infrared imaging device and output value correction method - Google Patents

Description

  The present invention relates to a far-infrared imaging device and an output value correction method that can accurately correct an output luminance value of a far-infrared imaging device that images the outside at night.

  In order to ensure nighttime safety of vehicles such as automobiles, many obstacle detection systems that detect the presence of obstacles such as pedestrians and bicycles using far-infrared imaging devices such as night vision have been developed. Usually, two far-infrared imaging devices are installed at a predetermined position in front of the vehicle, and the distance to the obstacle detected by stereo vision is calculated.

  For example, from an image captured by the left and right far-infrared imaging devices, a pattern that matches a predetermined reference pattern is used to detect an area where an obstacle, such as a human being, is present, and the distance to the area is calculated by stereo vision By notifying the driver when the distance is within a predetermined distance, the vehicle traveling safety is ensured.

  The far-infrared imaging device is composed of a plurality of imaging elements, A / D-converts luminance values output from the imaging elements, and outputs them as digital signals. When images are taken by far-infrared devices arranged separately on the left and right, even if the object is the same, the amount of heat between them is different, and the output value is different. Therefore, in order to accurately calculate the distance to the obstacle detected by stereo vision, the luminance values of the imaging data of the same object are the same so that the same object can be reliably detected. Therefore, it is necessary to correct the output value of the far-infrared imaging device.

For example, Non-Patent Document 1 discloses NUC (Non-Uniformity Correction), which is a representative example of a method for correcting the non-uniformity of sensitivity that differs for each image sensor, and the response characteristics of the image sensor are linear. Assuming that there is an offset, an offset correction value for correcting an output value for a predetermined temperature for each image sensor and a gain correction value for correcting a difference in output value for a predetermined temperature change rate for each image sensor are calculated. The output value from the infrared imaging device is corrected.
“Real-time implementation of two-point non-uniformity correction for IR-CCD imagery”, SPIE Proc., Vol. 2598, p. 44-50, 1995

  However, in the above-described correction method of the output value of the imaging apparatus, the offset correction value is not calculated corresponding to the absolute temperature, so even if the same luminance value is output, the surrounding environment is not affected. Depending on the situation, the area may indicate a pedestrian or the area may not indicate a pedestrian. Therefore, there is a problem that it is difficult to improve the detection accuracy of pedestrians.

  For example, in summer and winter, the degree of difference in absolute temperature of the surface temperature is greater than the degree of difference in temperature change rate between the surrounding environment and the pedestrian. However, in the conventional method, since the offset correction value is calculated regardless of the absolute temperature, the difference between the luminance value of the area indicating the pedestrian and the luminance value of the surrounding area is corrected to be small depending on conditions, There is a possibility that it is difficult to accurately detect a region indicating a pedestrian.

  In addition, the state of the far-infrared device changes with time. In particular, the amount of light received by the image sensor tends to decrease with time due to lens aging, etc., even if it is adjusted to output a predetermined luminance value at a predetermined temperature at the time of shipment, for example, the actual output There is no guarantee that the values correspond to absolute temperatures. On the other hand, the output value for a predetermined temperature change rate does not vary greatly compared to the output value for a predetermined temperature.

  The present invention has been made in view of such circumstances, and by calculating an offset correction value so as to correct the output value of the image sensor to a value corresponding to the absolute temperature, the object to be imaged from the output value of the image sensor. An object of the present invention is to provide a far-infrared imaging device and an output value correction method capable of estimating the absolute temperature.

In order to achieve the above object, a far-infrared imaging device according to a first invention calculates a plurality of imaging elements arranged in a matrix and an offset correction value for correcting an output value for a predetermined temperature for each imaging element. An offset correction value calculating means; and a gain correction value calculating means for calculating a gain correction value for correcting a difference in output value with respect to a predetermined temperature change rate for each of the image sensors, the offset correction value calculating means and the gain correction value. in the far-infrared imaging apparatus for acquiring imaging data by correcting the output value for each image sensing element by the offset correction value each is calculated and a gain correction value calculation means, the surface temperature is substantially uniform, entering into the imaging device a shutter for opening and closing the opening for light, and temperature measuring means for measuring the surface temperature of the shutter, with respect to each of the plurality of temperature husband for each of the imaging device Wherein a storage means for offset correction value calculation means prestores offset correction value calculated, the offset correction value calculation unit may query the offset correction value for each of the plurality of temperature husband said storage means stores said characterized in that the temperature measuring means are so as to correct the offset correction value for each image sensing element against the surface temperature of the meter measuring the said shutter is closed the shutter.

Moreover, far-infrared imaging device according to the second aspect, in the first shot bright, the temperature measuring means, characterized in that it is a sensor for measuring the surface temperature in contact with the surface of the shutter.

According to a third aspect of the present invention, there is provided an output value correction method for calculating an offset correction value for correcting an output value for a predetermined temperature for each of a plurality of image sensors arranged in a matrix in a far-infrared imaging device. in the output value correction method for correcting an output value for each image sensing element by the offset correction value and gain correction value calculating a gain correction value for correcting the difference of the output value, calculated for a given temperature change rate every, the imaging device The offset correction value calculated for each of a plurality of temperatures is stored in advance, the surface temperature of the shutter is acquired with the shutter having a substantially uniform surface temperature closed, and the offset correction for each of the stored plurality of temperatures is performed. The value is inquired, and the offset correction value for each image sensor with respect to the acquired surface temperature is corrected .

In the first invention and the third invention, an offset correction value for correcting an output value for a predetermined temperature is calculated for each of a plurality of image sensors arranged in a matrix in the far infrared imaging device, and a predetermined value is calculated for each image sensor. A gain correction value for correcting a difference in output value with respect to the temperature change rate is calculated, and the output value is corrected for each image sensor by the offset correction value and the gain correction value. Previously storing the offset correction value calculated with respect to each of the plurality of temperature husband for each imaging device to obtain the surface temperature of the sheet Yatta with closed shutters surface temperature is substantially uniform, for a plurality of temperatures husband stored s query the offset correction value is corrected by estimating the offset correction value for each image sensing element against the surface temperature of the obtained sheet Yatta. Thus, it is possible to correct the offset correction value corresponding to the absolute temperature, can be easily estimated the absolute temperature of the imaging object based on the luminance value. For example, it is possible to output the output value in association with the absolute temperature over a long period of time by closing the shutter and capturing an image and correcting the offset correction value at a predetermined time interval or before the start of imaging.

In the second invention, the temperature measuring means is a sensor that measures the surface temperature in contact with the surface of the shutter. This makes it possible to measure the surface temperature of the shutter with high accuracy, because it is inexpensive and compact, can be easily mounted on the shutter surface, and can directly detect the temperature change of the shutter surface. The correction value can be calculated with high accuracy.

First invention and according to the third invention, it is possible to correct the offset correction value corresponding to the absolute temperature, can be easily estimated the absolute temperature of the imaging object based on the luminance value. For example, it is possible to output the output value in association with the absolute temperature over a long period of time by closing the shutter and capturing an image and correcting the offset correction value at a predetermined time interval or before the start of imaging.

According to the second aspect of the present invention, the surface temperature of the shutter can be accurately measured because it is inexpensive and small-sized, can be easily attached to the shutter surface, and can directly detect the temperature change of the shutter surface. Thus, the offset correction value can be calculated with high accuracy.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing a configuration of a far-infrared imaging device 1 according to an embodiment of the present invention. In addition, the far-infrared imaging device 1 which concerns on this Embodiment is an imaging device using the infrared light whose wavelength is 7-14 micrometers.

  In FIG. 1, an image capturing unit 11 includes an image sensor that converts an optical signal into an electrical signal in a matrix. As the far-infrared imaging device, an infrared sensor such as a vanadium oxide bolometer type using a micromachining technique or a pyroelectric type BSTO (Barium-Titanium Oxide) is used.

  The image capturing unit 11 reads an infrared light image around the vehicle as a luminance signal, and transmits the read luminance signal to the signal processing unit 12 that is an LSI substrate. FIG. 2 is a block diagram illustrating a configuration of the signal processing unit 12 of the far-infrared imaging device 1 according to the embodiment of the present invention. The signal processing unit 12 includes an A / D conversion unit 121, an NUC (Non-Uniformity Correction) processing unit 122, a BPR (Bad-Pixel Replacement) processing unit 123, and a non-volatile memory such as a flash memory. The RAM 125 is a temporary storage memory such as an SRAM.

  The signal processing unit 12 converts the luminance signal received from the image capturing unit 11 into a digital signal by the A / D conversion unit 121, and outputs the luminance signal output for each image sensor by the NUC (Non-Uniformity Correction) processing unit 122. to correct.

In the NUC processing unit 122, as a correction coefficient by calibration, an offset correction value for correcting an output value for a predetermined temperature for each image sensor, and a gain correction for correcting a difference in output value for a predetermined temperature change rate for each image sensor. The value is calculated and the luminance signal is corrected. For example, if the output luminance value for each image sensor arranged in a matrix of i rows and j columns is V _ij , the NUC processing unit calculates the luminance value for each image sensor by performing the calculation shown in (Equation 1). Correct to _ij .

In (Expression 1), G _ij represents a gain correction value, and O _ij represents an offset correction value. For example, an output for each image sensor when an object having a uniform temperature distribution such as a black body furnace or a shutter is imaged. This is a value calculated based on the luminance value.

  In the present embodiment, a plate-like shutter 32 having a substantially uniform temperature distribution on the surface is provided so as to be able to be opened and closed between the objective lens 31 and the image pickup unit 11, and the shutter 32 is closed. The offset correction value is corrected as needed based on the output luminance value for each image sensor when the image is picked up. In FIG. 1, a mechanical plain shutter is taken as an example. However, the shutter may have any configuration as long as the temperature distribution on the surface is uniform.

  In order to perform correction so that the same luminance value is output when the temperature of the imaging object is the same by the offset correction value, the absolute temperature of the surface of the shutter 32 is acquired and the corresponding luminance value is output. It is necessary to calculate an offset correction value. Therefore, in the present embodiment, as a means for detecting the surface temperature of the shutter 32, a contact-type temperature sensor 33 that contacts the surface of the shutter 32 and measures the surface temperature, for example, a thermistor, a semiconductor temperature sensor, a thermocouple, or the like is used. Is attached to the surface of the image pickup unit 11 side. FIG. 3 is a schematic diagram showing the configuration of the thermistor 33a used as the contact temperature sensor 33 of the far-infrared imaging device 1 according to the embodiment of the present invention.

  The thermistor 33a according to the present embodiment has a substantially rectangular parallelepiped shape called a so-called chip type, in which a ceramic semiconductor 331 is laminated and an internal electrode 332 is sandwiched between layers. The internal electrode is connected to the external electrode 333 via an intermediate electrode (not shown), and a current flows by applying an external voltage. Both sides of the external electrode 333 are soldered to the surface of the shutter 32, and the resistance value of the ceramic semiconductor 331 varies according to the temperature change of the shutter 32. The value of the current flowing through the inside varies according to the variation of the resistance value. Therefore, it becomes possible to function as a highly accurate temperature measuring means by following the fluctuation of the current value. The current value corresponding to the temperature detected by the thermistor 33a is transmitted to the signal processing unit 12 as a temperature signal.

  A semiconductor temperature sensor 33b may be used instead of the thermistor 33a. The semiconductor temperature sensor 33b includes a transistor inside, and utilizes a characteristic that the base-emitter voltage Vbe changes substantially linearly with respect to the ambient temperature. That is, the temperature and the output voltage of the semiconductor temperature sensor 33b are in a linear relationship, and can be directly output from the semiconductor temperature sensor 33b without requiring an external resistor. As a result, attachment is easier and the surface temperature of the shutter 32 can be detected more accurately.

  The LSI 16 of the signal processing unit 12 acquires the surface temperature and the output luminance value when the shutter 32 is closed, and calculates the offset correction value and the gain correction value based on the acquired surface temperature and the output luminance value for each image sensor. calculate. The calculated offset correction value and gain correction value are stored in the RAM 125, and the NUC processing unit 122 corrects the luminance signal output for each image sensor based on the stored offset correction value and gain correction value. .

  The image sensor includes an image sensor in which defects are generated at a predetermined rate. Therefore, the LSI 16 uses a BPR (Bad-Pixel Replacement) processing unit 123 to set the output value of the image sensor having a defect to a value representative of the output value of the surrounding image sensor, for example, the image sensor having the defect. Is corrected to the average value of the output values of the surrounding image sensors excluding the output value of. The output value corrected for each image sensor is stored in the image memory 13 as image data.

  The communication interface unit 14 is an LSI substrate, and sends image data stored in the image memory 13 to an external device and takes a captured image in accordance with a command received from the external device via the communication line 2 such as a communication cable. The conversion of the transfer rate according to the resolution of the data, the format conversion of the data for transmitting the image data, and the like are performed.

  Hereinafter, calibration processing in the LSI 16 of the signal processing unit 12 will be described. FIG. 4 is a flowchart showing a calibration processing procedure of the LSI 16 of the signal processing unit 12 of the far-infrared imaging device 1 according to the embodiment of the present invention.

The image capturing unit 11 of the far-infrared imaging device 1 captures the shutter 32 having a uniform temperature distribution, and an output value for each image sensor is transmitted to the signal processing unit 12. The LSI 16 of the signal processing unit 12 receives a luminance signal for each image sensor from the image capturing unit 11 (step S401), and converts it into a digital signal V _ij (step S402). Further, the surface temperature of the shutter 32 is acquired from the contact-type temperature sensor 33 (step S403).

The LSI 16 uses a digitized luminance value V _ij of a predetermined imaging device, for example, an imaging device in the vicinity of the attachment position of the contact-type temperature sensor 33 as a reference luminance value, and gain correction values so that all luminance values become reference luminance values. The offset correction value is calculated for each image sensor (step S404), and stored in the RAM 125 in association with the surface temperature of the shutter 32 (step S405). That is, using the surface temperature of the shutter 32 at the time of calibration as a reference temperature, the gain correction value and the offset correction value at the reference temperature are calculated.

  At the time of actual imaging, the LSI 16 corrects the luminance signal captured by the image capturing unit 11 using the gain correction value and the offset correction value stored in the RAM 125. However, what is stored in the RAM 125 is a gain correction value and an offset correction value calculated based on the reference temperature. Since the reference temperature and the surface temperature of the shutter 32 at the time of imaging are generally different, they are stored in the RAM 125. The gain correction value and the offset correction value that have been set cannot be used as they are.

  Here, the offset correction value is calculated corresponding to the surface temperature at the time of calibration of the shutter 32 that is an absolute temperature, and is corrected according to the difference from the surface temperature of the shutter 32 at the time of imaging. It can correct | amend so that the output luminance value with respect to absolute temperature may be output correctly. On the other hand, the gain correction value for correcting the output value with respect to the temperature change rate does not fluctuate greatly with respect to the change in the surface temperature of the shutter 32. Therefore, it is necessary to correct by performing calibration at any time like the offset correction value. Poor sex.

Therefore, when the far-infrared imaging device 1 according to the present embodiment performs imaging, the offset correction value is estimated and corrected by closing the shutter 32 and executing simple calibration. FIG. 5 is a flowchart showing a procedure of correction processing of the offset correction value of the LSI 16 of the signal processing unit 12 of the far-infrared imaging device 1 according to the embodiment of the present invention.

The far-infrared imaging device 1 transmits an output value for each image sensor to the signal processing unit 12. The LSI 16 of the signal processing unit 12 receives the luminance signal for each image sensor from the image imaging unit 11 (step S501) and converts it into a digital signal V _ij (step S502).

  The LSI 16 reads the gain correction value and the offset correction value stored in the RAM 125 for each image sensor (step S503). Further, the LSI 16 acquires the surface temperature of the shutter 32 from the contact temperature sensor 33 (step S504), and corrects the offset correction value for each read image sensor based on the acquired surface temperature (step S505).

The offset correction value corresponding to the acquired surface temperature is estimated by storing a plurality of offset correction values corresponding to the reference temperature and performing linear interpolation according to the acquired surface temperature. The correction method of the offset correction value is not particularly limited to this, and for example, a linear function may be obtained in advance based on calibration results for a plurality of temperatures, and the obtained surface temperature may be calculated as a variable. .

The LSI 16 corrects the luminance value V _ij for each image sensor to the output luminance value V ′ _ij using the extracted gain correction value and the calculated offset correction value (step S506), and stores it in the image memory 13 (step S507). . The communication interface unit 14 generates packet data for sending the image data stored in the image memory 13 to the external device in accordance with a command sent from the external device, and sends the packet data to the external device.

  The output value from the far-infrared imaging device 1 according to the present embodiment is a luminance value corresponding to the absolute temperature through the above-described processing. That is, the absolute value of the surface temperature of the imaged object can be easily estimated based on the output luminance value. Therefore, when the far-infrared imaging device 1 according to the present embodiment is applied to an obstacle detection system, for example, based on the absolute value of the detected surface temperature of the obstacle, It can be determined whether or not it exists, and the type of obstacle can be specified more accurately.

  As described above, according to the present embodiment, the offset correction value can be calculated corresponding to the absolute temperature, and the absolute temperature of the imaging target can be easily estimated based on the luminance value. For example, by performing calibration with the shutter closed at a predetermined time interval or before imaging starts, the offset correction value can be corrected in association with the absolute temperature, and the output value can be correlated with the absolute temperature over a long period of time. Can be output.

  In the above-described embodiment, the LSI 16 of the signal processing unit 12 of the far-infrared imaging device 1 performs the above-described processing, but an arithmetic device that performs subsequent processing, such as obstacle detection processing, collision determination processing, and the like. May be executed by the MPU, CPU, or the like.

  In the present embodiment, the calibration is executed by closing the shutter 32 before the imaging of the far-infrared imaging device 1 is started. However, the timing for executing the calibration is not limited to this. For example, Calibration may be executed at regular intervals to correct the offset correction value.

It is a block diagram which shows the structure of the far-infrared imaging device which concerns on embodiment of this invention. It is a block diagram which shows the structure of the signal processing part of the far-infrared imaging device which concerns on embodiment of this invention. It is a schematic diagram which shows the structure of the thermistor of the far-infrared imaging device which concerns on embodiment of this invention. It is a flowchart which shows the procedure of the calibration process of LSI of the signal processing part of the far-infrared imaging device which concerns on embodiment of this invention. It is a flowchart which shows the procedure of the correction process of the offset correction value of LSI of the signal processing part of the far-infrared imaging device which concerns on embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Far-infrared imaging device 2 Communication line 11 Image pick-up part 12 Signal processing part 13 Image memory 14 Communication interface part 15 Internal bus 16 LSI
33 Contact-type temperature sensor 33a Thermistor 33b Semiconductor temperature sensor 121 A / D converter 122 NUC processor 123 BPR processor 125 RAM

Claims (3)

A plurality of imaging devices arranged in a matrix;
Offset correction value calculating means for calculating an offset correction value for correcting an output value for a predetermined temperature for each image sensor;
Gain correction value calculating means for calculating a gain correction value for correcting a difference in output value with respect to a predetermined temperature change rate for each of the image sensors;
In the far-infrared imaging device that acquires the imaging data by correcting the output value for each imaging element by the offset correction value and the gain correction value calculated by each of the offset correction value calculation unit and the gain correction value calculation unit ,
Surface temperature is substantially uniform, and a shutter for opening and closing an opening for incident light to the imaging element;
Temperature measuring means for measuring the surface temperature of the shutter ;
Storage means for storing in advance the offset correction value calculated by the offset correction value calculation means for each of a plurality of temperatures for each image sensor ;
The offset correction value calculating means, the image pickup against the storage means to query the offset correction value for each of the plurality of temperature husband stored, the surface temperature of the said shutter temperature measuring means has a total measured in a state of closing said shutter A far-infrared imaging device, wherein an offset correction value for each element is corrected . The temperature measuring means, a far-infrared imaging apparatus according to claim 1 Symbol mounting, characterized in that a sensor for measuring the surface temperature in contact with the surface of the shutter. An offset correction value for correcting an output value for a predetermined temperature is calculated for each of a plurality of image sensors arranged in a matrix on the far-infrared imaging device, and a difference in output value with respect to a predetermined temperature change rate is calculated for each of the image sensors. In an output value correction method for calculating a gain correction value to be corrected, and correcting an output value for each image sensor by the calculated offset correction value and gain correction value.
An offset correction value calculated for each of a plurality of temperatures for each image sensor is stored in advance,
Get the surface temperature of the shutter in a state where the surface temperature is closing the shutter is substantially uniform,
An output value correction method comprising: querying an offset correction value for each of a plurality of stored temperatures and correcting the offset correction value for each imaging device with respect to the acquired surface temperature .

Images