JP2007325120A - Far infrared imaging system and far infrared imaging method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a far infrared imaging system and a far infrared imaging method, which correct output values from a plurality of far infrared imaging apparatuses so that these output values are approximately the same for the same object. <P>SOLUTION: The far infrared imaging system corrects an output value of each image sensor and acquires imaging data by using a plurality of far infrared imaging apparatuses having image sensors arrayed like a matrix and connected to each other so that output values can be communicated with each other, measures the surface temperature of an object and stores the ratio of an output value change to a temperature change in each image sensor. The far infrared imaging apparatus acquires the measured surface temperature, reads an output value corresponding to the acquired surface temperature in each image sensor, corrects the output value by using the ratio of the output value change to the temperature change which is stored in each image sensor, and outputs the corrected output value to the external. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a far-infrared imaging system and a far-infrared imaging method capable of correcting output luminance values of a plurality of far-infrared imaging devices that image the outside so as to output substantially the same luminance value for the same temperature.

  In order to ensure the safety of vehicles such as automobiles during night driving, many obstacle detection systems that detect the presence of obstacles such as pedestrians and bicycles using far-infrared imaging devices such as night vision have been developed. Usually, two far-infrared imaging devices are installed at a predetermined position in front of the vehicle, the distance to the obstacle detected by stereo vision is calculated, and the presence of the obstacle is detected by feature amount detection.

  However, a bolometer or the like often used as an imaging element of a far-infrared imaging device has a large variation in temperature detection sensitivity. Therefore, offset drift or the like based on time changes is likely to occur, so that noise is easily superimposed on the captured image, and it is difficult to improve obstacle detection accuracy.

In order to solve such a problem, for example, Non-Patent Document 1 discloses NUC (Non-Uniformity Correction), which is a representative example of a method of correcting sensitivity non-uniformity that is different for each image sensor. In Non-Patent Document 1, on the premise that the response characteristic of an image sensor is linear, an offset correction value for correcting an output value for a predetermined temperature for each image sensor, and an output value for a predetermined temperature change rate for each image sensor The gain correction value for correcting the difference is calculated, and the output value from the far-infrared imaging device is corrected.
“Real-time implementation of two-point non-uniformity correction for IR-CCD imagery”, SPIE Proc., Vol. 2598, p. 44-50, 1995

  However, in the above-described correction method of the output value of the imaging device, the offset correction value is not calculated corresponding to the absolute temperature. Therefore, when using a plurality of far infrared imaging devices, different far infrared imaging devices are the same. Even when the luminance value is output, it is not guaranteed that the detected temperatures are the same. Therefore, when extracting the feature amount of an image to detect whether it is an obstacle, the brightness, contrast, etc. of the image captured for each far-infrared imaging device are different, and the AGC (Auto Even if the (Gain Control) function is used, there is a problem that it may be difficult to accurately detect an obstacle.

  In particular, when performing distance measurement based on stereo vision based on images captured by far-infrared imaging devices installed on the left and right of the front of the vehicle, it is difficult to match the brightness and contrast of the images. It becomes difficult to improve accuracy. Therefore, by correcting the output luminance value so that the brightness, contrast, and the like of the images captured by both the imaging devices are substantially the same, the obstacle detection accuracy is dramatically improved.

  The present invention has been made in view of such circumstances, and a far-infrared imaging system capable of correcting the output values of a plurality of far-infrared imaging devices so as to substantially match the same object, and An object is to provide a far-infrared imaging method.

  In order to achieve the above object, a far-infrared imaging system according to a first aspect of the present invention includes a plurality of far-infrared imaging devices having imaging elements arranged in a matrix and having output values connected so as to communicate with each other. In a far-infrared imaging system that obtains imaging data by correcting output values for each imaging device, temperature measuring means for measuring the surface temperature of the object, and the ratio of the output value change to the temperature change is stored for each imaging device The far-infrared imaging device is connected to the temperature measuring means and the storage means so as to be able to communicate with the data, the means for acquiring the measured surface temperature, and the image sensor for the acquired surface temperature. And a means for correcting the output value using the ratio of the change in the output value with respect to the temperature change for each stored image sensor and outputting the output value to the outside. And features.

  The far-infrared imaging system according to a second invention is the far-infrared imaging system according to the first invention, wherein the far-infrared imaging device has a maximum value and a minimum value of output values within a predetermined temperature range that match other far-infrared imaging devices. As described above, the output value for each image sensor is corrected.

  In the far-infrared imaging system according to the third invention, the far-infrared imaging device according to the first or second invention includes a shutter, and the temperature measuring means measures the surface temperature of the shutter. The storage means stores a ratio of an output value change for each image pickup device with respect to a temperature change in a state where the shutter is closed.

  In addition, the far-infrared imaging method according to the fourth invention uses a plurality of far-infrared imaging devices having imaging elements arranged in a matrix and connected to each other so that output values can be communicated with each other. In a far-infrared imaging method for acquiring imaging data by correcting an output value, the surface temperature of an object is measured, a ratio of an output value change with respect to a temperature change is stored for each image sensor, and the measured surface temperature, the surface The output value is corrected using the output value for each image sensor with respect to the temperature and the ratio of the output value change with respect to the temperature change for each image sensor stored, and output to the outside.

  In the first and fourth inventions, a plurality of far-infrared imaging devices having imaging elements arranged in a matrix and having output values connected so as to communicate with each other are used. The imaging data is acquired after correction. The surface temperature of the object is measured, the ratio of the output value change to the temperature change is stored for each image sensor, the measured surface temperature, the output value for each image sensor with respect to the surface temperature, and the stored imaging The output value is corrected using the ratio of the output value change to the temperature change for each element, and output to the outside. As a result, the ratio of the change in the output value to the temperature change of the image sensor of the far-infrared imaging device does not vary greatly even when the devices are different. By correcting the output value for each far-infrared imaging device with reference to the output value, the output value for the measured surface temperature can be substantially matched even when the far-infrared imaging device is different. Therefore, the brightness, contrast, and the like of images captured by a plurality of far-infrared imaging devices can be substantially matched, and the obstacle detection accuracy can be improved.

  In the second invention, the far-infrared imaging device corrects the output value for each imaging element so that the maximum value and the minimum value of the output values within a predetermined temperature range coincide with those of other far-infrared imaging devices. By limiting the temperature range to a predetermined temperature range, the ratios of changes in output values with respect to temperature changes of different far-infrared imaging devices more closely match, and the brightness, contrast, etc. of images captured by multiple far-infrared imaging devices can be further It is possible to match.

  In addition, for example, in summer and winter, the degree of difference in the absolute temperature of the surface temperature is greater than the degree of difference in temperature change between the ambient environment and the obstacle. That is, since the temperature difference between the obstacle and the surrounding environment is small in the summer, the contrast between the obstacle and the surrounding environment can be enhanced by setting the temperature range narrow. In winter, the temperature difference between the obstacle and the surrounding environment is large, so by setting a wide temperature range, it is possible to acquire an image with relatively little noise and improve the recognition accuracy of the obstacle. It becomes possible.

  In the third invention, the far-infrared imaging device includes a shutter, measures the surface temperature of the shutter, and stores the ratio of the output value change for each image sensor to the temperature change with the shutter closed. As a result, temperature calibration can be performed based on the surface temperature of the shutter, which is relatively easy to make the surface temperature uniform. For example, the offset correction value is obtained by closing the shutter before imaging, for example, at a predetermined time interval and imaging. By correcting the output value, the output value can be corrected more accurately.

  According to the present invention, since the ratio of the change in the output value with respect to the temperature change of the imaging element of the far-infrared imaging device does not vary greatly even when the devices are different, the measured surface temperature and the surface temperature By correcting the output value for each far-infrared imaging device using the output value for each imaging device as a reference, the output value for the measured surface temperature can be substantially matched even when the far-infrared imaging device is different. . Therefore, the brightness, contrast, and the like of images captured by a plurality of far-infrared imaging devices can be substantially matched, and the obstacle detection accuracy can be improved.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a diagram schematically showing a configuration of a far-infrared imaging system according to an embodiment of the present invention. In the present embodiment, a set of far-infrared imaging devices 1 and 1 that capture surrounding images while traveling are mounted in the front grille near the center in front of the vehicle (also possible in a bumper). The far-infrared imaging devices 1 and 1 are imaging devices using infrared light having a wavelength of 7 to 14 micrometers.

  In FIG. 1, far-infrared imaging devices 1 and 1 are juxtaposed in a substantially symmetrical position in the left-right direction within a front grill of a vehicle. Image data captured by the far-infrared imaging devices 1 and 1 is transmitted to an arithmetic device 3 connected via a video cable 7 corresponding to an analog video system such as NTSC or a digital video system.

  The arithmetic device 3 is connected to the far infrared imaging devices 1 and 1 and the display device 4 having an operation unit via a cable 8 corresponding to a video system such as NTSC, VGA, DVI, etc. An output device such as an alarm device 5 that emits an audible warning by a sound effect or the like is connected via an in-vehicle LAN cable 6 compliant with CAN.

  FIG. 2 is a block diagram showing the configuration of the far-infrared imaging device 1 according to the embodiment of the present invention. In FIG. 2, the image capturing unit 11 includes an image sensor that converts an optical signal into an electrical signal in a matrix. As the far-infrared imaging device, an infrared sensor such as a vanadium oxide bolometer type using a micromachining technique or a pyroelectric type BSTO (Barium-Titanium Oxide) is used.

  The image capturing unit 11 reads an infrared light image around the vehicle as a luminance signal, and transmits the read luminance signal to the signal processing unit 12 configured by one or a plurality of LSI substrates. FIG. 3 is a block diagram showing a configuration of the signal processing unit 12 of the far-infrared imaging device 1 according to the embodiment of the present invention. The signal processing unit 12 includes an A / D conversion unit 121, a NUC (Non-Uniformity Correction) processing unit 122, a BPR (Bad-Pixel Replacement) processing unit 123, the operation of which is controlled by one or a plurality of arithmetic means such as LSI. And a RAM (storage means) 125 which is a non-volatile memory such as a flash memory and a temporary storage memory such as an SRAM.

  The signal processing unit 12 converts the luminance signal received from the image capturing unit 11 into a digital signal by the A / D conversion unit 121, and outputs the luminance signal output for each image sensor by the NUC (Non-Uniformity Correction) processing unit 122. to correct.

The NUC processing unit 122 outputs, as a correction coefficient by calibration, an offset correction value for correcting a luminance value output for a predetermined temperature for each image sensor and a predetermined temperature change rate for each image sensor. A gain correction value for correcting a difference in brightness value is calculated to correct the brightness signal. For example, when the luminance value for each imaging element arranged in a matrix of i rows and j columns is V _ij , the NUC processing unit performs the calculation shown in (Equation 1) to obtain the luminance value for each imaging element as V ′ _ij. To correct.

In (Equation 1), G _ij represents a gain correction value, and O _ij represents an offset correction value. For example, the luminance for each image sensor when an object having a uniform temperature distribution such as a black body furnace or a shutter is imaged. It is a value calculated based on the value.

  In the present embodiment, a plate-like shutter 32 having a substantially uniform temperature distribution on the surface is provided so as to be able to be opened and closed between the objective lens 31 and the image pickup unit 11, and the shutter 32 is closed. The offset correction value and the gain correction value are corrected as needed based on the luminance value output for each image pickup device when the image is picked up by. In FIG. 2, a mechanical plain shutter is described as an example. However, the shutter may have any configuration as long as the temperature distribution on the surface is uniform.

  In the plurality of far-infrared imaging devices 1, 1, in order to correct the output value so that the same luminance value is output when the temperatures of the imaging objects are the same, the absolute temperature of the surface of the shutter 32 is acquired. It is necessary to calculate an offset correction value and a gain correction value so that the luminance values output corresponding to the absolute temperature are substantially the same. Therefore, in the present embodiment, the ratio of the change in the output luminance value with respect to the temperature change due to the calibration performed at the time of shipment is stored, and is output corresponding to the obtained absolute temperature of the surface of the shutter 32 and the absolute temperature. By correcting to a function including a luminance value, the output luminance values for the same temperature can be made substantially coincident.

  In the present embodiment, as a means for detecting the surface temperature of the shutter 32, a contact-type temperature sensor (temperature detection means) 33 that measures the surface temperature by contacting the surface of the shutter 32, such as a thermistor, a semiconductor temperature sensor, a thermocouple, etc. Is attached to the surface of the shutter 32 on the image pickup unit 11 side. FIG. 4 is a schematic diagram showing a configuration of the thermistor 33a used as the contact-type temperature sensor 33 of the far-infrared imaging device 1 according to the embodiment of the present invention.

  The thermistor 33a according to the present embodiment has a substantially rectangular parallelepiped shape called a so-called chip type, in which a ceramic semiconductor 331 is laminated and an internal electrode 332 is sandwiched between layers. The internal electrode is connected to the external electrode 333 via an intermediate electrode (not shown), and a current flows by applying an external voltage. Both sides of the external electrode 333 are soldered to the surface of the shutter 32, and the resistance value of the ceramic semiconductor 331 varies according to the temperature change of the shutter 32. The value of the current flowing through the inside varies according to the variation of the resistance value. Therefore, it becomes possible to function as a highly accurate temperature measuring means by following the fluctuation of the current value. The current value corresponding to the temperature detected by the thermistor 33a is transmitted to the signal processing unit 12 as a temperature signal.

  A semiconductor temperature sensor 33b may be used instead of the thermistor 33a. The semiconductor temperature sensor 33b includes a transistor inside, and utilizes a characteristic that the base-emitter voltage Vbe changes substantially linearly with respect to the ambient temperature. That is, the temperature and the output voltage of the semiconductor temperature sensor 33b are in a linear relationship, and can be directly output from the semiconductor temperature sensor 33b without requiring an external resistor. As a result, attachment is easier and the surface temperature of the shutter 32 can be detected more accurately.

  The signal processing unit 12 acquires the surface temperature and the output luminance value in a state where the shutter 32 is closed, and outputs the luminance value based on the acquired surface temperature and the luminance value output for each image sensor. The offset correction value and the gain correction value are calculated so that the luminance value changes at a ratio to the stored temperature change, including the acquired surface temperature and the luminance value output for each image sensor. The calculated offset correction value and gain correction value are stored in the RAM 125, and the NUC processing unit 122 corrects the luminance signal output for each image sensor based on the stored offset correction value and gain correction value. .

  The communication interface unit 14 is an LSI substrate, and sends image data stored in the image memory 13 to an external device and takes a captured image in accordance with a command received from the external device via the communication line 2 such as a communication cable. The conversion of the transfer rate according to the resolution of the data, the format conversion of the data for transmitting the image data, and the like are performed.

  The arithmetic device 3 receives the output image data, executes an obstacle detection process using a known image recognition technique, displays the detection result on the display device 4, and sounds the alarm device 5. To notify the driver.

  Hereinafter, correction processing of the output luminance signal (output value) in the signal processing unit 12 will be described. FIG. 5 is a flowchart showing a procedure of correction processing of the luminance signal output from the signal processing unit 12 of the far-infrared imaging device 1 according to the embodiment of the present invention. In the present embodiment, it is assumed that the ratio of the luminance value change with respect to the temperature change is stored in the RAM 125 as a slope of a linear function by calibration performed at the time of shipment of the far-infrared imaging device 1. Of course, it is not limited to interpolating the correspondence between temperature and luminance value into a linear function, as long as the correspondence can be clearly specified.

The image capturing unit 11 of the far-infrared imaging device 1 captures the shutter 32 having a substantially uniform temperature distribution, and the output value for each image sensor is transmitted to the signal processing unit 12. The signal processing unit 12 receives a luminance signal for each image sensor from the image capturing unit 11 (step S501), and converts it into a digital signal V _ij (step S502). Further, the surface temperature of the shutter 32 is acquired from the contact-type temperature sensor 33 (step S503).

The signal processing unit 12 reads the slope of the linear function indicating the ratio of the luminance value change to the temperature change stored in the RAM 125 (step S504), and the digitized luminance value V _ij and the acquired surface temperature for each image sensor. And calculates the above-mentioned offset correction value and gain correction value so as to become such a linear function (step S506), and calculates the calculated offset correction value and gain. The correction value is stored in the RAM 125 (step S507).

  Thereafter, the signal processing unit 12 corrects the luminance signal captured by the image capturing unit 11 using the calculated offset correction value and gain correction value stored in the RAM 125. By doing in this way, even if it is different far-infrared imaging devices 1, when the same subject is imaged, it is possible to output substantially the same luminance value for the same temperature, and a plurality of far-infrared imaging devices 1 The brightness, contrast, and the like of the images captured by the infrared imaging devices 1 and 1 can be substantially matched, and the obstacle detection accuracy can be improved.

  FIG. 6 is a diagram showing a concept of a luminance value adjustment method in the far-infrared imaging system according to the present embodiment. As shown in FIG. 6, the RAM 125 stores a linear function 61 indicating the relationship between temperature and luminance value obtained by calibration at the time of shipment. The signal processing unit 12 of the far-infrared imaging device 1 acquires the surface temperature T0 when the shutter 32 is closed, and receives the luminance value V0 at that time.

  The signal processing unit 12 shifts the linear function 61 stored in the RAM 125 so as to become the luminance value V0 at the surface temperature T0, generates a new linear function 62, and overwrites and saves it in the RAM 125. In this way, even the far-infrared imaging devices 1, 1,... Having different characteristics can output substantially the same luminance value for the same object.

  In order to output the same luminance value more accurately with respect to the same object, it is preferable to execute the luminance value correction processing shown in FIGS. 5 and 6 while limiting the temperature range. That is, if the temperature is within a certain temperature range, the error when the correspondence between the temperature and the brightness value is first-order approximated becomes smaller. FIG. 7 is a diagram illustrating the concept of the luminance value adjustment method when the temperature range is limited.

As shown in FIG. 7, the temperature range is set between Tmax and Tmin, the luminance value for temperature Tmin is Vmin, and the luminance value for temperature Tmax is Vmax. Since it is sufficient that the luminance values outputted only during the temperature Tmax to Tmin are the same, the luminance value Vmin with respect to the temperature Tmin is set to “0”, and the luminance value Vmax with respect to the temperature Tmax is set to “255”. This means that the minimum luminance value is “0”, the maximum luminance value is 2 ⁿ −1 (n is a natural number), and “255” when n is “8” (8-bit data). It becomes. By doing in this way, in the predetermined temperature range, different far-infrared imaging devices 1, 1,... Output the same luminance value more reliably at the same temperature.

  If the luminance value before correction is V and the luminance value after correction is Va, the luminance value Va after correction can be calculated by the conversion equation shown in (Equation 2).

  By calculating the corrected luminance value Va based on (Equation 2), the luminance value at the lower end of the set temperature range can be set to ‘0’, and the luminance value at the upper end can be set to Vmax.

  The temperature range is desirably changed flexibly depending on the season, ambient temperature, and the like. For example, the ambient temperature is high in summer, and it is difficult for a difference in surface temperature between an obstacle and another object to occur. Accordingly, by setting the temperature range to be set narrow, the difference in surface temperature between the obstacle and another object can be emphasized, and the obstacle detection accuracy can be increased.

  Specifically, for example, ambient temperature is detected, and it is determined whether or not the detected temperature continuously exceeds a predetermined temperature, for example, 25 degrees. If it is determined that the temperature exceeds 25 degrees, it is determined that it is in summer, and the temperature range is reduced. The method for determining whether or not it is the summer season is not particularly limited. For example, a calendar function may be provided, and control may be performed so that the temperature range corresponds to the summer season within a predetermined period.

  In winter, the ambient temperature is low, and the surface temperature between the obstacle and other objects tends to be different. This tendency is particularly remarkable when the obstacle is a pedestrian. Therefore, by setting a wide temperature range to be set, it is possible to extract only a portion where the difference in surface temperature is larger than a predetermined value, and an image with less noise can be acquired.

  Specifically, for example, ambient temperature is detected, and it is determined whether or not the detected temperature continues for a certain period and falls below a predetermined temperature, for example, 10 degrees. If it is determined that the temperature is less than 10 degrees, it is determined that it is winter, and the temperature range is expanded. The method for determining whether or not it is the winter season is not particularly limited. For example, a calendar function may be provided, and control may be performed so that the temperature range corresponds to the winter season within a predetermined period.

  As described above, according to the present embodiment, the far-infrared imaging devices 1, 1,... Are based on the measured surface temperature of the shutter 32 and the output luminance value for each image sensor with respect to the surface temperature. By correcting each, the output luminance value with respect to the measured surface temperature can be substantially matched even when the far-infrared imaging devices 1, 1,. Therefore, the brightness, contrast, and the like of images captured by the plurality of far-infrared imaging devices 1, 1,... Can be substantially matched, and the obstacle detection accuracy can be improved. The obstacle recognition process is not particularly limited as long as it is a known method.

  In the above-described embodiment, the output luminance value is corrected based on the surface temperature of the shutter 32. However, the correction is not limited to the surface temperature of the shutter 32. For example, the temperature of the road surface is set below the vehicle. A temperature sensor (not shown) to be detected may be installed, and a correction process similar to that described above may be performed based on the road surface temperature detected by the temperature sensor.

  In the present embodiment, the initial value of the ratio of the change in the output luminance value with respect to the temperature change is stored based on the calibration result at the time of shipment of the far-infrared imaging device 1, but the shutter 32 is closed at any time. The correction may be performed based on the calibration result performed in the state, or may be corrected by executing calibration at regular intervals.

  Furthermore, in the present embodiment, the case where the storage means is provided in the far-infrared imaging device 1 is described. However, the present invention is not particularly limited to this, and a storage built in an external storage device, for example, the arithmetic device 3 is used. You may construct | assemble the far-infrared imaging system which concerns on this Embodiment using an apparatus.

It is a figure which shows typically the structure of the far-infrared imaging system which concerns on embodiment of this invention. It is a block diagram which shows the structure of the far-infrared imaging device which concerns on embodiment of this invention. It is a block diagram which shows the structure of the signal processing part of the far-infrared imaging device which concerns on embodiment of this invention. It is a schematic diagram which shows the structure of the thermistor used as a contact-type temperature sensor of the far-infrared imaging device which concerns on embodiment of this invention. It is a flowchart which shows the procedure of the correction process of the luminance signal output of the signal processing part of the far-infrared imaging device which concerns on embodiment of this invention. It is a figure which shows the concept of the luminance value adjustment method in the far-infrared imaging system which concerns on this Embodiment. It is a figure which shows the concept of the luminance value adjustment method at the time of limiting a temperature range.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Far-infrared imaging device 11 Image pick-up part 12 Signal processing part 13 Image memory 14 Communication interface part 15 Internal bus 32 Shutter 33 Contact-type temperature sensor (temperature measurement means)
33a thermistor 33b semiconductor temperature sensor 121 A / D conversion unit 122 NUC processing unit 123 BPR processing unit 125 RAM (storage means)

Claims (4)

A far-infrared image sensor having a plurality of far-infrared imaging devices that have imaging elements arranged in a matrix and whose output values are communicably connected to each other, and that correct the output values for each imaging element and acquire imaging data In the imaging system,
Temperature measuring means for measuring the surface temperature of the object;
Storage means for storing the ratio of the output value change with respect to the temperature change for each image sensor,
The far-infrared imaging device is:
The temperature measuring means and the storage means are connected so as to be capable of data communication,
Means for obtaining the measured surface temperature;
Means for reading an output value for each image sensor with respect to the acquired surface temperature;
A far-infrared imaging system comprising: means for correcting an output value using a ratio of a change in output value with respect to a temperature change for each image sensor stored and outputting the corrected output value to the outside. The far-infrared imaging device is:
The output value for each image sensor is corrected so that the maximum value and the minimum value of the output value within a predetermined temperature range coincide with other far-infrared imaging devices. Far infrared imaging system. The far infrared imaging device includes a shutter,
The temperature measuring means is adapted to measure the surface temperature of the shutter;
The far-infrared imaging system according to claim 1, wherein the storage unit stores a ratio of a change in output value for each imaging element with respect to a temperature change in a state where the shutter is closed. A far-infrared ray having a plurality of far-infrared imaging devices that have imaging elements arranged in a matrix and whose output values are communicably connected to each other and correct the output value for each imaging element to obtain imaging data In the imaging method,
Measure the surface temperature of the object,
Stores the ratio of output value change to temperature change for each image sensor,
The output value is corrected using the measured surface temperature, the output value for each image sensor with respect to the surface temperature, and the ratio of the output value change with respect to the temperature change for each stored image sensor, and output to the outside A far infrared imaging method.

Images