JP2011044813A - Imaging apparatus, correction value calculation method, and imaging method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus, a correction value calculation method of the imaging apparatus, and an imaging method for calculating a gain correction value and an offset correction value with small capacity of memory, and correcting output values of a plurality of image pickup devices so as to be approximately the same for the same object. <P>SOLUTION: Two images having different uniform luminance captured by an imaging part are stored in two memories P1, P2, respectively. When the correction value is calculated, the two images are read from the memories, the gain correction value (or the offset correction value) is calculated using the images, and stored in one memory. The offset correction value (or the gain correction value) is calculated using the image of the other memory and the calculated gain correction value (or the offset correction value). <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an imaging device capable of accurately correcting an output luminance value of an imaging device such as an infrared imaging device, a correction value calculation method for the imaging device, and an imaging method.

  In the infrared imaging device, infrared light collected by a lens is converted into an electric signal by a photoelectric conversion unit, A / D converted by an AD converter through a signal reading unit, and output as a digital signal.

  The photoelectric conversion unit is composed of infrared sensors arranged in a matrix. The infrared sensor receives light in the infrared region (infrared ray) and converts it into an electrical signal to extract necessary information. Conventional infrared sensors are roughly classified into two types: thermal type and quantum type.

  The thermal infrared sensor utilizes the fact that the temperature of the sensor itself increases due to the received infrared light. Thermal infrared sensors include thermopiles based on the thermoelectromotive force effect, and bolometers that change electrical resistance due to temperature changes. The thermal infrared sensor has a wide operating wavelength band and can be operated at room temperature, but has the disadvantages of low detection sensitivity and low response speed.

  The quantum infrared sensor detects a photocurrent caused by light energy by using a photoelectric effect that directly converts infrared light absorbed by a solid material into an electrical signal, and is also called a photoconductive type. There are photodiodes, phototransistors, and photo ICs. Quantum infrared sensors have high detection sensitivity, excellent response speed, and a detection capability 100 to 1000 times that of thermal sensors, but the detection sensitivity is wavelength-dependent and the operating temperature is low, so liquid nitrogen There is a disadvantage that it does not work without cooling. Moreover, it is more expensive than the above-described thermal type.

  Regardless of which type of infrared sensor is used, it is necessary to install a signal readout circuit on the output side (rear stage) of the infrared sensor. The signal readout circuit reads out and amplifies the signal detected by the infrared sensor. In addition, when a plurality of infrared sensors are used, it is necessary to provide as many signal readout circuits as the number of sensors.

  Such an infrared imaging device has variations in signal output for each pixel due to manufacturing variations of infrared sensors, incident light quantity and exposure time, applied reverse bias voltage, amplifier manufacturing variations, temperature, and the like. As a result, fixed pattern noise is superimposed on the video. Fixed pattern noise can be broadly classified into gain components resulting from sensitivity variations and offset components resulting from variations in brightness. In order to output video accurately, it is necessary to correct the output value of the infrared imaging device so that the luminance values of the imaging data of the same object are the same.

  In order to solve such a problem, for example, Non-Patent Document 1 discloses NUC (Non-Uniformity Correction), which is a representative example of a method of correcting sensitivity variation (non-uniformity) that differs for each image sensor. ing. In Non-Patent Document 1, on the assumption that the output characteristics of an image sensor are linear, an offset correction value for correcting an output value for a predetermined temperature for each image sensor and an output value for a predetermined temperature change rate for each image sensor The gain correction value for correcting the difference is calculated, and the output value from the infrared imaging device is corrected.

"Real-time implementation of two-point non-uniformity correction for IR-CCD imagery", SPIE Proc. Vol. 2598, 1995

  However, in the two-point correction method of Non-Patent Document 1, two reference images with different luminances are captured, and the offset correction value and the gain correction value are calculated for each image sensor using the two reference images. In order to store the two reference images and the offset correction value and gain correction value of each image sensor, a memory for at least four frames of image data is required. For this reason, a large amount of memory is required for image processing of the infrared video signal, the size of the entire imaging apparatus becomes large, the cost increases, and an improvement measure has been demanded.

  The present invention has been made in view of such circumstances, and a main object of the present invention is to store two images having different uniform luminances picked up by the image pickup unit in two memories, and set a correction value. At the time of calculation, the two images are read from both memories, a gain correction value (or offset correction value) is calculated using these images, stored in one memory, and an image of the other memory is calculated. By using the gain correction value (or offset correction value) and calculating the offset correction value (or gain correction value), the gain correction value and the offset correction value are calculated with a small amount of memory. An imaging device capable of correcting the output values of a plurality of imaging elements so as to substantially match the same object, and effectively removing fixed pattern noise superimposed on the video; And to provide the correction value calculation method of the image device, and the imaging method. Note that the present invention is not limited to infrared imaging, and can be applied to various imaging in which output values can be corrected at two points.

  An imaging apparatus according to the present invention includes an imaging unit having a plurality of imaging elements, first and second memories, a gain correction value for correcting sensitivity variations of each imaging element, and an offset correction value for correcting offset variations. A correction value calculation unit that calculates for each imaging element, the imaging unit stores the captured first and second images in the first and second memories, respectively, and the correction value calculation unit includes: The first and second images are read from the first and second memories, the gain correction value (or offset correction value) is calculated using the first and second images, and the first and second images are calculated. The offset correction value (or gain correction value) is calculated using the second image and the calculated gain correction value (or offset correction value) stored in a memory. And

  In the imaging apparatus according to the present invention, the correction value calculation unit is configured to store the calculated offset correction value (or gain correction value) in the second memory.

  In addition, the imaging apparatus of the present invention calculates a reference value for calculating first and second reference values that are average values of pixel values related to the brightness of a plurality of pixels included in each of the first and second images. The correction value calculating unit determines a straight line of an output characteristic indicating an output of the imaging unit with respect to illuminance of the imaging target using the first and second reference values, and the imaging element The gain correction value and the offset correction value are calculated so that each output value coincides with the straight line.

  The imaging apparatus of the present invention further includes a random noise removing unit that removes random noise superimposed on an image captured by the imaging unit, and the first and second images are removed by the random noise removing unit. After the random noise is removed, the random noise is stored in the first and second memories.

  In the imaging apparatus of the present invention, the imaging unit captures a plurality of frames for each of the first and second images, and the random noise removal unit averages pixel values between the plurality of frames for each pixel. It is configured to perform an operation.

  The image pickup apparatus according to the present invention is characterized in that an average calculation of pixel values between the plurality of frames is performed by moving average or IIR filtering.

  In the imaging apparatus of the present invention, the imaging unit captures a plurality of frames for each of the first and second images, and converts the plurality of frames for each of the first and second images into an image for each frame. An intra-frame average value calculating unit that calculates an average value of pixel values related to the brightness of a plurality of pixels included as an intra-frame average value, and an intra-frame average value of each frame calculated by the intra-frame average value calculating unit. For each of the first and second images, an inter-frame average value calculation unit that calculates an average between the plurality of frames and calculates an average value between the frames, and a plurality of frames of the first and second images, respectively. A pixel average value calculation unit that averages pixel values between the plurality of frames for each pixel, calculates the pixel average value, and stores the average value in the first and second memories; Then, using the average value between the frames of the first and second images, a straight line of output characteristics indicating the output of the imaging unit with respect to the illuminance of the imaging target is determined, and the output value for each imaging element is The gain correction value and the offset correction value are calculated based on pixel average values of the first and second images so as to match.

  Further, the imaging apparatus of the present invention is characterized in that the inter-frame average value calculation unit and / or the pixel average value calculation unit performs an average calculation using moving average or IIR filtering.

  The image pickup apparatus according to the present invention further includes a NUC processing unit that corrects an output value for each image pickup device using the offset correction value and the gain correction value.

  In the imaging apparatus of the present invention, the imaging element is an infrared imaging element.

  According to the correction value calculation method of the present invention, an imaging device including an imaging unit having a plurality of imaging devices and first and second memories that store images captured by the imaging unit has a sensitivity variation of each imaging device. In the correction value calculation method for calculating the gain correction value to be corrected and the offset correction value for correcting the offset variation for each of the image sensors, the imaging unit images two objects to be imaged having different uniform illuminances. An imaging step for acquiring the first and second images, a storage step for storing the first and second images in the first and second memories, respectively, and the first and second memories from the first and second memories. A first calculation step of reading the first and second images, calculating the gain correction value (or offset correction value) using the first and second images, and storing the gain correction value (or offset correction value) in the first memory; ,in front Using said gain correction value second image and the calculated (or offset correction value), characterized in that it comprises a second calculation step of calculating the offset correction value (or gain correction value).

  In the correction value calculation method of the present invention, the second calculation step stores the calculated offset correction value (or gain correction value) in the second memory.

  The imaging method according to the present invention is an imaging method used in an imaging apparatus including an imaging unit having a plurality of imaging elements. The offset correction value and the gain correction value calculated by the correction value calculation method described above are used for each imaging element. The output value is corrected and output.

  In the present invention, the gain correction value for each image sensor is obtained by using two types of light and dark uniform first and second images obtained by imaging two imaging objects having uniform illuminances different from each other. (Or an offset correction value) is calculated, and the offset correction value (or gain correction value) can be calculated using the second image and the calculated gain correction value (or offset correction value). Further, in the correction value calculation process, after calculating the gain correction value (or offset correction value), the pixel value of the first image becomes unnecessary, and the calculated gain correction value (or offset correction value) and Taking advantage of the fact that the timing to save data does not overlap in time, the memory is shared, and the memory for 3 frames of image data is sufficient for the calculation process, and the required memory can be greatly reduced. Become.

  According to the present invention, in the correction value calculation processing, the pixel value of the second image and the calculated offset correction value (or gain correction value) indicate that the timing for storing data does not overlap in time. By utilizing the memory, the memory for two frames of image data is sufficient for the calculation process, and the required memory can be further reduced.

  In the present invention, using the two-point non-uniformity correction method, the average value of the output values of each image sensor with respect to the illuminance of the imaging target, that is, the average output characteristic is determined by the first and second reference values. The gain correction value and the offset correction value for each image sensor are calculated using the straight line. Thereby, the gain correction value and offset correction value of each image sensor can be calculated easily and with high accuracy.

  In the present invention, the first and second images are supplied to the gain correction value and offset correction value calculation processing after the random noise superimposed on the image is removed. Thereby, the gain correction value and the offset correction value of each image sensor are calculated with higher accuracy.

  In the present invention, random noise can be effectively removed by interframe average calculation for each pixel by using temporal irregularity of the distribution of random noise.

  In the present invention, inter-frame average calculation of pixel values is performed by moving average or IIR filtering. In particular, in IIR filtering, a memory for storing pixel values of past frames is not required, and random noise can be effectively removed with a simple arithmetic circuit only for addition and bit shift.

  In the present invention, random noise is effective by capturing two types of bright and dark uniform first and second images of the two imaging objects and performing an intra-frame average calculation and an inter-frame average calculation. The first and second images from which random noise has been effectively removed are obtained by calculating an average value between frames that has been removed and performing an average calculation between frames for each pixel. Then, a gain correction value and an offset correction value for each image sensor are calculated using a two-point non-uniformity correction method. Thereby, the gain correction value and offset correction value of each image sensor can be calculated easily and with high accuracy.

  In the present invention, the interframe average calculation and / or the interframe average calculation for each pixel is performed by moving average or IIR filtering. In particular, in IIR filtering, a memory for storing pixel values of past frames is not required, and a random noise removal unit can be configured with a simple arithmetic circuit only for addition and bit shift.

  In the present invention, it is possible to produce and display a clear video by performing a process of removing fixed pattern noise superimposed on the video by the NUC processing unit.

  In the present invention, gain correction values and offset correction values due to manufacturing variations of infrared elements, operating environment variations, and the like are calculated with a small amount of memory, and a plurality of infrared imaging is performed with the calculated gain correction values and offset correction values. The output value of the element can be corrected so as to substantially match the same object, and fixed pattern noise superimposed on the video can be effectively removed.

  In the present invention, the output correction values of a plurality of image sensors are corrected so as to substantially match the same object with the gain correction value and the offset correction value calculated by the correction value calculation method of the present invention. An imaging method can be provided.

  According to the present invention, the memory required for the calculation process can be greatly reduced by sharing the memory for the correction value calculation process. Furthermore, the overall size of the imaging apparatus is reduced, and the cost can be reduced.

1 is a block diagram illustrating a configuration of an infrared imaging device according to Embodiment 1. FIG. 2 is a block diagram illustrating a configuration of a signal processing unit of the infrared imaging device according to Embodiment 1. FIG. FIG. 3 is a block diagram illustrating a configuration of a correction value calculation unit according to the first embodiment. 5 is a flowchart illustrating a procedure of correction processing by a signal processing unit according to the first embodiment. 4 is a flowchart illustrating a procedure of correction value calculation processing according to the first embodiment. It is explanatory drawing which shows the output of the infrared image sensor with respect to illumination intensity. 10 is a block diagram illustrating a configuration of a correction value calculation unit according to Embodiment 2. FIG. 10 is a flowchart illustrating a procedure of correction value calculation processing according to the second embodiment. It is a block diagram which shows the structure of the correction value calculation part which concerns on Embodiment 3. FIG. 10 is a flowchart illustrating a procedure of correction value calculation processing according to the third embodiment. It is a block diagram which shows the structure of the correction value calculation part which concerns on Embodiment 4. 14 is a flowchart illustrating a procedure of correction value calculation processing according to the fourth embodiment.

Hereinafter, the present invention will be described in detail with reference to the drawings illustrating embodiments thereof.
(Embodiment 1)
FIG. 1 is a block diagram showing a configuration of an infrared imaging device 1 according to Embodiment 1 of the present invention. The infrared imaging apparatus 1 according to the present embodiment receives a bolometer type such as VOx or amorphous silicon, an uncooled type far-infrared imaging element using an SOI diode or a thermopile, etc., having a sensitivity band of 8 to 12 μm, and InGaAs. A near-infrared imaging device having a sensitivity band of 1.0 to 3.0 μm adopted for the surface is used.

  In FIG. 1, the infrared imaging device 1 includes an image imaging unit 11, a signal processing unit 12, and a communication interface unit 13. The image pickup unit 11 includes image pickup elements that convert optical signals into electric signals in a matrix. The image capturing unit 11 reads a surrounding infrared light image as a luminance signal, and transmits the read luminance signal to the signal processing unit 12. Note that the imaging elements may be arranged in a one-dimensional array.

  The signal processing unit 12 is configured by an LSI or the like, converts the luminance signal input from the image imaging unit 11 into a digital signal, corrects variations in the imaging device, and calculates an output value (luminance signal) of the imaging device. Processing, defect element correction processing, gain control processing, and the like are performed. The processed video data is output to an external storage device, display device or the like via the communication interface unit 13.

  FIG. 2 is a block diagram illustrating a configuration of the signal processing unit 12 of the infrared imaging device 1 according to the first embodiment of the present invention. The signal processing unit 12 includes an A / D conversion unit 121, an NUC (Non-Uniformity Correction) processing unit 122, a correction value calculation unit 123, a flash memory, and the like whose operation is controlled by one or more arithmetic units such as LSI. The memory unit 124 is a temporary storage memory such as a nonvolatile memory or an SRAM.

  The signal processing unit 12 converts the luminance signal received from the image capturing unit 11 into a digital signal by the A / D conversion unit 121 and outputs the digital signal to the NUC processing unit 122. The NUC processing unit 122 corrects the luminance signal output for each image sensor with the correction value calculated in advance by the correction value calculation unit 123 and stored in the memory unit 124.

  The correction value calculation unit 123 corrects a luminance value output for a predetermined illuminance for each image sensor as a correction coefficient by calibration in order to remove fixed pattern noise superimposed on the luminance signal. The gain correction value for correcting the value and the variation of the luminance value output with respect to the predetermined illuminance variation for each image sensor is calculated and stored in the memory unit 124.

  The memory unit 124 includes a memory P2 for storing a gain correction value for each image sensor, a memory P1 for storing an offset correction value for each image sensor, and two reference values used for correction value calculation processing described later. Are assigned to the reference value memories A2 and A1. The memories P2 and P1 are used to temporarily save two images captured by the image capturing unit 1 in the correction value calculation process described later. The memories P2 and P1 are frame memories. The reference value memories A2 and A1 are negligible with respect to the pixel memories and the memories P2 and P1, and the reference value memories A2 and A2 are negligible.

When the luminance signal input from the image capturing unit 11 and converted by the A / D conversion unit 121 is input to the NUC processing unit 122, the gain correction value stored in the memory P2 and the offset stored in the memory P1 The luminance signal is corrected using the correction value. For example, when the luminance value of the image sensor (i, j) arranged in a matrix of N rows and M columns is V _ij , the NUC processing unit 122 performs the calculation shown in Equation 1 to perform the luminance value V for each image sensor. _ij is corrected to the output luminance value V _ij ′.

Here, G _ij represents a gain correction value, and O _ij represents an offset correction value.

  In the first embodiment, two imaging objects having uniform and different illuminances of a black body furnace and a shutter are used, for example, on the assumption that the output characteristics of the imaging element are linear using a two-point non-uniformity correction method. To obtain two uniform images of brightness and darkness, linearly approximate the slope of its output characteristics, calculate the tilt of each image sensor and the average tilt of the entire image sensor, and the tilt of each image sensor is the average tilt The gain correction value is calculated so as to be the same. When the gain correction value is calculated, a reference luminance value that is an average of the luminance values of the respective image sensors under the reference illuminance is calculated, and for each image sensor, the luminance value and the reference luminance value are considered in consideration of sensitivity variation. The difference is calculated as an offset correction value.

FIG. 6 is an explanatory diagram showing the output of the infrared imaging device with respect to the illuminance. In FIG. 6, the horizontal axis represents illuminance, and the vertical axis represents the output luminance of the image sensor. In FIG. 6, straight lines S <b> 1 to S <b> 3 are linear approximations of the output characteristics of the three image sensors in the two-dimensional array, and output characteristics that are outputs of the image sensors with respect to the illuminance e are represented by straight lines. The output for the same illuminance may differ from one image sensor to another due to variations in offset and sensitivity of each image sensor. Further, the fluctuation of the output of the image sensor, that is, the gain due to the fluctuation of the illuminance e has a substantially linear characteristic and can be expressed by a slope. Therefore, when the average value V (e _H ) and bar V (e _L ) of the output of the image sensor at the illuminance e _H and the illuminance e _L are calculated, a straight line representing the average output characteristic of the entire image sensor is determined. Is possible. By correcting the output of each image sensor with the inclination of the straight line, it is possible to eliminate sensitivity variations. Specifically, the gain correction value G _ij for each image sensor is calculated according to the following formula 2.

However, V _ij (e) indicates an output value at the illuminance e of the image sensor (i, j).

Further, the offset correction value O _ij for each image sensor is calculated after the gain correction value G _ij is calculated, for example, with the illuminance e _L as the reference illuminance and taking into account sensitivity variations, the output average value bar V (e _L ) Can be calculated according to the following Equation 3.

  FIG. 3 is a block diagram illustrating a configuration of the correction value calculation unit 123 according to the first embodiment. FIG. 5 is a flowchart showing a procedure of correction value calculation processing according to the first embodiment. Hereinafter, the correction value calculation process of the first embodiment will be described in detail based on FIGS. 3 and 5.

As illustrated in FIG. 3, the correction value calculation unit 123 includes a random noise removal unit 231, a reference value calculation unit 232, an offset correction value calculation unit 233, and a gain correction value calculation unit 234. When calculating the correction value, first, the image capturing unit 11 captures an imaged object having a substantially uniform first illuminance e _H and acquires a first video having a first luminance (step S501). The acquired first video is converted into a digital signal by the A / D converter 121 (step S502) and output to the random noise removing unit 231. The random noise removing unit 231 extracts several frames, for example, n frames from the A / D converted first video, and calculates an average luminance value for the n frames individually for each pixel by an inter-frame average calculation for each pixel. Is calculated (step S503). For example, the random noise removing unit 231 performs the calculation shown in Equation 4 to perform a frame from the luminance value V _ij (e _H ) for each pixel, that is, for each imaging device, with respect to the first video under the first illuminance e _H. A mean value bar V _ij (e _H ) is calculated. Thereby, random noise can be removed.

Here, V _ijf (e) is the illuminance e and indicates the value of the pixel (i, j) in the f-th frame.

  Further, instead of calculating the average value, the calculation may be performed using a moving average or an IIR (Infinite Impulse Response) filter. In particular, when the IIR filter is used, a memory for storing the luminance value of the past frame is not required, and random noise can be effectively removed with a simple arithmetic circuit only for addition and bit shift.

The calculated inter-frame average value bar V _ij (e _H ) for each pixel is stored in the memory P 1 and is output to the reference value calculation unit 232. The reference value calculation unit 232 calculates the average value of all pixels in one frame (one screen) from the inter-frame average value bar V _ij (e _H ) for each input pixel by the intra-frame average calculation shown in Equation 5. is calculated as a first reference brightness value V _H is the reference value of the output of the image sensor under a first illumination e _H (step S504). The calculated first reference luminance value V _H is stored in the reference value memory A1.

Next, image capturing unit 11 captures an object to be imaged with a second illuminance e _L lower than the first illumination intensity e _H substantially uniform, obtaining a second image of the second luminance (step S505). The acquired second video is converted into a digital signal by the A / D converter 121 (step S506) and output to the random noise removing unit 231. The random noise removing unit 231 calculates the inter-frame average bar V _ij (e _L ) of the luminance for each pixel for the A / D converted second video as in the first video described above. In addition, the data is stored in the memory P2 (step S507) and output to the reference value calculation unit 232. The reference value calculation unit 232 calculates the average value of all pixels in one frame (one screen) from the inter-frame average value bar V _ij (e _L ) for each input pixel by the intra-frame average calculation shown in Equation 5. Is calculated as the second reference luminance value V _L which is the reference value of the output of the image sensor under the second illuminance e _L (step S508). The calculated second reference luminance value V _L is stored in the reference value memory A2.

Then, the gain correction value calculation unit 234 reads the inter-frame average value of the pixels from the memories P1 and P2 for each pixel, and obtains the first and second reference luminance values V _H and V _L from the reference value memories A1 and A2. Read and use these values to calculate the gain correction value (step S509). For example, when the gain correction value is calculated for the pixel (i, j), the gain correction value calculation unit 234 reads the first and second reference luminance values V _H and V _L from the reference value memories A1 and A2, and the memory The inter-frame average value bar V _ij (e _H ) and bar V _ij (e _L ) of the pixel (i, j) are read from P1 and P2, and the gain correction value G for the pixel (i, j) according to Equation 6 Calculate _ij .

As described above, when the gain correction value G _ij is calculated, the offset correction value O _ij for each image sensor is, for example, the illuminance e _L as the reference illuminance and the reference luminance value V _{L in} consideration of sensitivity variations. Is calculated as the difference between That is, when the gain correction value G _ij is calculated, the inter-frame average value bar V _ij (e _H ) for each pixel of the first video becomes unnecessary. Therefore, when the gain correction value G _ij is calculated, the interframe average value bar V _ij (e _H ) for each pixel of the first video is discarded, and the gain correction value G _ij can be stored in the memory P1. it can.

Next, the offset correction value calculation unit 233 reads the gain correction value from the memory P1 and the inter-frame average value of the pixel from the memory P2 for each pixel, and reads the second reference luminance value V _L from the reference value memory A2. An offset correction value is calculated (step S510). For example, when calculating the offset correction value for the pixel (i, j), the offset correction value calculation unit 233 outputs the gain correction value G _{ij of} the pixel (i, j) from the memory P1 and the pixel (i, j) from the memory P2. The inter-frame average value bar V _ij (e _L ) and the second reference luminance value V _L are read from the reference value memory A 2, and the offset correction value O _ij for the pixel (i, j) is calculated according to Equation 7. .

When the offset correction value O _ij is calculated, the inter-frame average bar V _ij (e _L ) for each pixel of the second video stored in the memory P2 becomes unnecessary. Therefore, the offset correction value O _ij is stored in the memory P2.

  The offset correction value and gain correction value calculated as described above are stored in the memory unit 124, and during actual imaging, the NUC processing unit 122 is based on the stored offset correction value and gain correction value. The luminance signal output for each image sensor is corrected.

  Hereinafter, correction processing of the luminance signal (output value) output by the signal processing unit 12 of the infrared imaging device 1 according to the first embodiment will be described during actual imaging. FIG. 4 is a flowchart showing a procedure of correction processing by the signal processing unit 12 according to the first embodiment.

The image capturing unit 11 of the infrared imaging device 1 captures an object to be imaged, and an output value for each image sensor is transmitted to the signal processing unit 12. The signal processing unit 12 receives a luminance signal for each image sensor from the image capturing unit 11 (step S401), and converts it into a digital signal V _ij (step S402).

The signal processing unit 12 reads the offset correction value and the gain correction value stored in the memory unit 124 (step S403), and corrects the luminance signal captured by the image capturing unit 11 (step S404). The corrected luminance signal is output to the outside as an output luminance value V _ij ′ (step S405).

  As described above, according to the first embodiment, a gain (sensitivity) correction value and an offset correction value for each image sensor are calculated based on two types of reference images having different luminances. Using this calculation result, it is possible to perform a NUC process for removing fixed pattern noise (a gain component and an offset component) superimposed on the video, and to create and display a clear video.

  Further, in the process of calculating the gain correction value and the offset correction value, the luminance value for each pixel of the two types of reference images, the gain correction value, and the offset correction value are stored in the memory because the data storage timing does not overlap in time. It is possible to significantly reduce the memory required for the calculation process by utilizing the fact that it is possible to share the memory.

(Embodiment 2)
Hereinafter, an infrared imaging device according to Embodiment 2 of the present invention will be described in detail with reference to the drawings. In the infrared imaging device according to the second embodiment, the configuration of the correction value calculation unit 125 is different from that of the first embodiment. Other than that, the imaging apparatus according to the second embodiment is configured in the same manner as the imaging apparatus according to the first embodiment, and a description thereof will be omitted.

FIG. 7 is a block diagram illustrating a configuration of the correction value calculation unit 125 according to the second embodiment. FIG. 8 is a flowchart showing a procedure of correction value calculation processing according to the second embodiment. Hereinafter, the correction value calculation processing according to the second embodiment will be described in detail with reference to FIGS. As shown in FIG. 7, the correction value calculation unit 125 includes an intra-frame average calculation unit 251, an inter-frame average calculation unit 252, a pixel average value calculation unit 255, an offset correction value calculation unit 253, and a gain correction value calculation unit 254. . When calculating the correction value, first, the image capturing unit 11 captures an imaged object having a substantially uniform first illuminance e _H and acquires a first video having a first luminance (step S801). The acquired first video is converted into a digital signal by the A / D converter 121 (step S802), and is output to the intra-frame average calculator 251 and the pixel average value calculator 255. The intra-frame average calculation unit 251 extracts several frames, for example, n frames from the A / D-converted first video, and executes the average intra-frame calculation according to Formula 8 for each frame, thereby calculating the average luminance of each frame. The value bar V _f (e _H ) is calculated and output to the inter-frame average calculation unit 252 (step S803).

The inter-frame average calculation unit 252 averages the average luminance value of each frame among n frames by the inter-frame average calculation shown in Equation 9, and is the reference value of the output of the image sensor under the first illuminance e _H. It is calculated as a reference luminance value V _H of 1 (step S804). The calculated reference luminance value V _H is stored in the reference value memory A1.

Also, the pixel average value calculation unit 255 calculates the luminance average value for the n frames individually for each pixel by the inter-frame average calculation for each pixel for the n frames of the first video. For example, the average value bar V _ij (e _H ) between frames is calculated for each pixel, that is, the image sensor, for the first video under the first illuminance e _H by the calculation according to Equation 4. The calculated bar V _ij (e _H ) is stored in the memory P1 (step S805).

Next, image capturing unit 11 captures an object to be imaged with a second illuminance e _L lower than the first illumination intensity e _H substantially uniform, obtaining a second image of the second luminance (step S806). The acquired second video is converted into a digital signal by the A / D converter 121 (step S807), and is output to the intra-frame average calculator 251 and the pixel average value calculator 255. The intra-frame average calculation unit 251 calculates an intra-frame average value for n frames for the A / D-converted second video in the same manner as the first video described above. The data is output to 252 (step S808). The inter-frame average calculation unit 252 calculates the average value of n frames as the second reference luminance value V _L that is the reference value of the output of the image sensor under the second illuminance e _L and saves it in the reference value memory A2. (Step S809). Further, the pixel average value calculation unit 255 calculates an inter-frame average value bar V _ij (e _L ) for each pixel for the n frames of the second video, and stores it in the memory P2 (step S810).

  As a result, the correction value calculation unit 125 according to the second embodiment can remove random noise superimposed on the video, similarly to the correction value calculation unit 123 according to the first embodiment. Further, instead of calculating the average value, the calculation may be performed using a moving average or an IIR filter.

Next, gain correction value and offset correction value calculation processing will be described. The calculation operations of the offset correction value calculation unit 253 and the gain correction value calculation unit 254 of the second embodiment are the same as the calculation operations of the offset correction value calculation unit 233 and the gain correction value calculation unit 234 of the first embodiment. First, the gain correction value calculation unit 234 reads the inter-frame average value of the pixels from the memories P1 and P1 for each pixel, and obtains the first and second reference luminance values V _H and V _L from the reference value memories A1 and A2. The gain correction value G _ij is calculated in accordance with the read-out, and is stored in the memory P1 (step S811). Next, the offset correction value calculation unit 253 reads the gain correction value from the memory P1 and the interframe average value of the pixel from the memory P2 for each pixel, and reads the second reference luminance value V _L from the reference value memory A2. The offset correction value O _ij is calculated according to Equation 7 and stored in the memory P2 (step S812).

  Accordingly, the correction value calculation unit 125 of the second embodiment, like the correction value calculation unit 123 of the first embodiment, corrects gain (sensitivity) for each image sensor based on two types of reference images having different luminances. Value and offset correction value are calculated. Using this calculation result, it is possible to perform a NUC process for removing fixed pattern noise superimposed on the video, and to produce and display a clear video.

  Further, in the process of calculating the gain correction value and the offset correction value, the luminance value for each pixel of the two types of reference images, the gain correction value, and the offset correction value are stored in the memory because the data storage timing does not overlap in time. It is possible to significantly reduce the memory required for the calculation process by utilizing the fact that it is possible to share the memory.

(Embodiment 3)
Hereinafter, an infrared imaging device according to Embodiment 3 of the present invention will be described in detail with reference to the drawings. The infrared imaging device according to the third embodiment is different from the first embodiment in the configuration of the correction value calculation unit 126. Other than that, the imaging apparatus according to the third embodiment is configured in the same manner as the imaging apparatus according to the first embodiment, and a description thereof will be omitted.

FIG. 9 is a block diagram illustrating a configuration of the correction value calculation unit 126 according to the third embodiment. FIG. 10 is a flowchart illustrating a procedure of correction value calculation processing according to the third embodiment. Hereinafter, the correction value calculation process of the third embodiment will be described in detail with reference to FIGS. 9 and 10. As shown in FIG. 9, the correction value calculation unit 126 includes a random noise removal unit 261, a reference value calculation unit 262, an offset correction value calculation unit 263, and a gain correction value calculation unit 264. The configurations and operations of the random noise removal unit 261 and the reference value calculation unit 262 of the third embodiment are the same as those of the random noise removal unit 231 and the reference value calculation unit 232 of the first embodiment. When calculating the correction value, first, the image capturing unit 11 captures an imaged object having a substantially uniform first illuminance e _H and acquires a first video having a first luminance (step S1001). The acquired first video is converted into a digital signal by the A / D conversion unit 121 (step S1002) and output to the random noise removal unit 261. The random noise removing unit 261 extracts several frames, for example, n frames from the A / D-converted first video, and performs inter-frame averaging of the n frames individually for each pixel by the inter-frame averaging calculation shown in Equation 4. The value bar V _ij (e _H ) is calculated, stored in the memory P1, and output to the reference value calculation unit 262 (step S1003). Thereby, random noise can be removed.

Next, the reference value calculation unit 262 performs an intra-frame average calculation shown in Equation 5 to calculate all the pixels in one frame (one screen) from the input inter-frame average value bar V _ij (e _H ) for each pixel. The average value is calculated as the first reference luminance value V _H that is the reference value of the output of the image sensor under the first illuminance e _H (step S1004). The calculated reference luminance value V _H is stored in the reference value memory A1.

Next, image capturing unit 11 captures an object to be imaged with a second illuminance e _L lower than the first illumination intensity e _H substantially uniform, obtaining a second image of the second luminance (step S1005). The acquired second video is converted into a digital signal by the A / D converter 121 (step S1006) and output to the random noise removing unit 261. The random noise removing unit 261 calculates the inter-frame average bar V _ij (e _L ) of the luminance for each pixel, similar to the first video described above, for the A / D converted second video. Then, the data is stored in the memory P2 (step S1007) and output to the reference value calculation unit 262. The reference value calculation unit 262 calculates the average value of all the pixels in one frame (one screen) from the inter-frame average value bar V _ij (e _L ) for each input pixel by the average calculation in the frame shown in Equation 5. Is calculated as the second reference luminance value V _L which is the reference value of the output of the image sensor under the second illuminance e _L (step S1008). The calculated second reference luminance value V _L is stored in the reference value memory A2.

Next, the offset correction value calculation unit 263 reads the inter-frame average value of the pixels from the memories P1 and P2 for each pixel, and the first and second reference luminance values V _H and V _L from the reference value memories A1 and A2. , And using these values, an offset correction value is calculated and stored in the memory P1 (step S1009). In the first and second embodiments described above, the straight line representing the average output characteristic of the image sensor is determined by the first and second reference luminance values V _H and V _L , and the gain correction value G _ij for each image sensor is calculated. In the above description, the offset correction value O _ij is calculated by taking the illuminance e _L as the reference illuminance and considering sensitivity variation. However, the present invention is not limited to this, and after calculating the offset correction value O _ij for each image sensor. The gain correction value G _ij may be calculated. For example, in the case of far-infrared imaging, the second illuminance e _{L is set} to an illuminance under an absolute temperature, and the offset correction value O _ij is calculated as a difference from the output average value of the image sensor under the absolute temperature. In addition, the offset correction value O _ij may be calculated according to the following formula 10 obtained from the formulas 6 and 7.

Next, the gain correction value calculation unit 264 reads the offset correction value from the memory P1 and the inter-frame average value of the pixel from the memory P2 for each pixel, and reads the second reference luminance value V _L from the reference value memory A2. The gain correction value G _ij is calculated according to Equation 11 and stored in the memory P2 (step S1010).

  As a result, the correction value calculation unit 126 of the third embodiment, similar to the correction value calculation units 123 and 125 of the first and second embodiments, gains for each image sensor based on two types of reference images having different luminances. (Sensitivity) correction value and offset correction value are calculated. Using this calculation result, it is possible to perform a NUC process for removing fixed pattern noise superimposed on the video, and to produce and display a clear video.

  Further, in the process of calculating the gain correction value and the offset correction value, the luminance value for each pixel of the two types of reference images, the gain correction value, and the offset correction value are stored in the memory because the data storage timing does not overlap in time. It is possible to significantly reduce the memory required for the calculation process by utilizing the fact that it is possible to share the memory.

(Embodiment 4)
Hereinafter, an infrared imaging device according to Embodiment 4 of the present invention will be described in detail with reference to the drawings. In the infrared imaging apparatus according to the fourth embodiment, the configuration of the correction value calculation unit 127 is different from that of the first embodiment. Other than that, the imaging apparatus according to the fourth embodiment is configured in the same manner as the imaging apparatus according to the first embodiment, and a description thereof will be omitted.

FIG. 11 is a block diagram illustrating a configuration of the correction value calculation unit 127 according to the fourth embodiment. FIG. 12 is a flowchart illustrating a procedure of correction value calculation processing according to the fourth embodiment. Hereinafter, the correction value calculation processing of the fourth embodiment will be described in detail based on FIGS. 11 and 12. As shown in FIG. 11, the correction value calculation unit 127 includes an intra-frame average calculation unit 271, an inter-frame average calculation unit 272, a pixel average value calculation unit 275, an offset correction value calculation unit 273, and a gain correction value calculation unit 274. . When calculating the correction value, first, the image capturing unit 11 captures an object to be imaged having a substantially uniform first illuminance e _H and acquires a first video having a first luminance (step S1201). The acquired first video is converted into a digital signal by the A / D converter 121 (step S1202), and is output to the intra-frame average calculator 271 and the pixel average value calculator 275. The intra-frame average calculation unit 271 extracts several frames, for example, n frames from the A / D-converted first video, calculates the average luminance value of each frame according to Formula 8 for each frame, and calculates the inter-frame average calculation The data is output to the unit 272 (step S1203).

The inter-frame average calculation unit 272 averages the average luminance value of each frame among n frames by the inter-frame average calculation shown in Equation 9, and is the reference value of the output of the image sensor under the first illuminance e _H. It is calculated as a reference luminance value V _H of 1 (step S1204). The calculated reference luminance value V _H is stored in the reference value memory A1.

In addition, the pixel average value calculation unit 275 performs inter-frame average calculation for each pixel for each of the n frames of the first video by the inter-frame average calculation for each pixel according to Equation 4, and the inter-frame average value bar V _ij (e _H ) is calculated and stored in the memory P1 (step S1205).

Next, image capturing unit 11 captures an object to be imaged with a second illuminance e _L lower than the first illumination intensity e _H substantially uniform, obtaining a second image of the second luminance (step S1206). The acquired second video is converted into a digital signal by the A / D converter 121 (step S1207), and is output to the intra-frame average calculator 271 and the pixel average value calculator 275. The intra-frame average calculator 271 calculates an intra-frame average value for n frames for the A / D converted second video in the same manner as the first video described above. It outputs to 272 (step S1208). The inter-frame average calculation unit 272 calculates the average value of n frames as the second reference luminance value V _L that is the reference value of the output of the image sensor under the second illuminance e _L and saves it in the reference value memory A2. (Step S1209). Further, the pixel average value calculation unit 275 calculates an inter-frame average value bar V _ij (e _L ) for each pixel for the n frames of the second video, and stores it in the memory P2 (step S1210).

  Thereby, the correction value calculation unit 127 according to the fourth embodiment can remove random noise superimposed on the video as in the first to third embodiments. Further, instead of calculating the average value, the calculation may be performed using a moving average or an IIR filter.

Next, gain correction value and offset correction value calculation processing will be described. The calculation operations of the offset correction value calculation unit 273 and the gain correction value calculation unit 274 of the fourth embodiment are the same as the calculation operations of the offset correction value calculation unit 263 and the gain correction value calculation unit 264 of the third embodiment. First, the offset correction value calculation unit 273 reads the average value between pixels of the pixels from the memories P1 and P2 for each pixel, and obtains the first and second reference luminance values V _H and V _L from the reference value memories A1 and A2. The offset correction value O _ij is calculated using these values and stored in the memory P1 (step S1211). Next, the gain correction value calculation unit 274 reads the offset correction value from the memory P1 and the inter-frame average value of the pixel from the memory P2 for each pixel, and reads the second reference luminance value V _L from the reference value memory A2. A gain correction value G _ij is calculated and stored in the memory P2 (step S1212).

  Accordingly, the correction value calculation unit 127 of the fourth embodiment, like the correction value calculation units of the first to third embodiments, corrects gain (sensitivity) for each image sensor based on two reference images having different luminances. Value and offset correction value are calculated. Using this calculation result, it is possible to perform a NUC process for removing fixed pattern noise superimposed on the video, and to produce and display a clear video.

  Further, in the process of calculating the gain correction value and the offset correction value, the brightness value for each pixel of the two reference images, the gain correction value, and the offset correction value are stored in the memory because the timing for storing the data does not overlap in time. By utilizing the fact that it can be shared, it becomes possible to greatly reduce the memory required for the calculation process.

  As mentioned above, although embodiment of this invention was explained in full detail based on drawing, this invention is not limited to the said embodiment, A various deformation | transformation implementation is possible within the range of the summary of this invention.

  For example, in the first to fourth embodiments, the correction value calculation process is performed before the imaging of the infrared imaging device 1 is started. However, the timing at which the correction value calculation process is performed is not limited to this, and is, for example, constant. The correction value calculation process may be executed every time to update the gain correction value and the offset correction value. In addition, when the infrared imaging device 1 is shipped, initial values of the gain correction value and the offset correction value are stored, the correction value calculation process is executed with the shutter closed at any time, and the gain correction value and the offset are calculated based on the calculation result. The correction value may be updated.

In the first to fourth embodiments, the offset correction value is calculated as a difference from the output reference value of the image sensor at the reference illuminance, taking the illuminance e _L as the reference illuminance and taking into account sensitivity variations. is not limited, for example, the illuminance e _H a reference illuminance, may be calculated taking into account the sensitivity variation as a difference between the output reference value of the image sensor at the reference illuminance. In the case of far-infrared imaging, an offset correction value is calculated based on the illuminance under the absolute temperature, the temperature of the imaging target is estimated based on the luminance value at the time of actual imaging, and the calculated offset correction value is You may supplement according to this temperature.

  Further, in the first to fourth embodiments, it has been described that the gain correction value and the offset correction value related to the infrared imaging device are calculated. It is possible to apply to various types of imaging that can correct the above.

11 Image capturing unit 124 Memory unit 231 261 Random noise removal unit 233 253 263 273 Offset correction value calculation unit 234 254 264 274 Gain correction value calculation unit 232 262 Reference value calculation unit 251 271 Average calculation unit 252, 272 Average calculation unit between frames 255, 275 Pixel average value calculation unit

Claims (13)

An imaging unit having a plurality of imaging elements;
First and second memories;
A correction value calculation unit that calculates a gain correction value for correcting sensitivity variations of each image sensor and an offset correction value for correcting offset variations for each of the image sensors; and
The imaging unit stores the captured first and second images in the first and second memories, respectively.
The correction value calculation unit reads the first and second images from the first and second memories, and calculates the gain correction value (or offset correction value) using the first and second images. Then, the offset correction value (or gain correction value) is calculated using the second image and the calculated gain correction value (or offset correction value) stored in the first memory. An imaging apparatus characterized by being configured.   The imaging apparatus according to claim 1, wherein the correction value calculation unit is configured to store the calculated offset correction value (or gain correction value) in the second memory. A reference value calculation unit that calculates first and second reference values that are average values of pixel values related to the brightness of a plurality of pixels included in each of the first and second images;
The correction value calculation unit uses the first and second reference values to determine a straight line of output characteristics indicating the output of the imaging unit with respect to the illuminance of the imaging target, and the output value for each imaging device is The imaging apparatus according to claim 1, wherein the gain correction value and the offset correction value are calculated so as to coincide with a straight line. A random noise removing unit that removes random noise superimposed on the image captured by the imaging unit;
The random noise removing unit removes random noise from the first and second images, and then stores the first and second images in the first and second memories. The imaging device according to any one of the above. The imaging unit captures a plurality of frames for each of the first and second images,
The imaging apparatus according to claim 4, wherein the random noise removing unit is configured to perform an average calculation of pixel values between the plurality of frames for each pixel.   6. The imaging apparatus according to claim 5, wherein an average calculation of pixel values between the plurality of frames is performed by moving average or IIR filtering. The imaging unit captures a plurality of frames for each of the first and second images,
An intra-frame average value calculation unit that calculates an average value of pixel values related to the brightness of a plurality of pixels included in the image for each frame for each of the plurality of frames of the first and second images. ,
The inter-frame average value calculated by averaging the intra-frame average value of each frame calculated by the intra-frame average value calculation unit between the plurality of frames for each of the first and second images. An arithmetic unit;
For each of the plurality of frames of each of the first and second images, a pixel value is averaged between the plurality of frames for each pixel to be calculated as a pixel average value, and is stored in the first and second memories An average value calculation unit, and
The correction value calculation unit determines a straight line of an output characteristic indicating an output of the imaging unit with respect to illuminance of the imaging target, using an average value between frames of the first and second images, and for each imaging element. The gain correction value and the offset correction value are calculated based on pixel average values of the first and second images so that an output value matches the straight line. Item 3. The imaging device according to Item 1 or 2.   The imaging apparatus according to claim 7, wherein the inter-frame average value calculation unit and / or the pixel average value calculation unit performs average calculation using moving average or IIR filtering.   9. The imaging apparatus according to claim 1, further comprising a NUC processing unit that corrects an output value for each imaging device based on the offset correction value and the gain correction value. 10.   The image pickup apparatus according to claim 1, wherein the image pickup element is an infrared image pickup element. A gain correction value and offset variation for correcting sensitivity variation of each imaging device are added to an imaging device including an imaging unit having a plurality of imaging devices and first and second memories that store images captured by the imaging unit. In the correction value calculation method for calculating the offset correction value to be corrected for each image sensor,
The imaging unit captures two imaging objects having uniform illuminances different from each other and acquires first and second images, and
Storing the first and second images in the first and second memories, respectively;
The first and second images are read from the first and second memories, the gain correction value (or offset correction value) is calculated using the first and second images, and the first and second memories are calculated. A first calculation step stored in the memory of
And a second calculation step of calculating the offset correction value (or gain correction value) using the second image and the calculated gain correction value (or offset correction value). Value calculation method.   The correction value calculation method according to claim 11, wherein the second calculation step stores the calculated offset correction value (or gain correction value) in the second memory.   The imaging method used for an imaging device provided with the imaging part which has a some imaging device WHEREIN: The output for every said imaging device by the offset correction value and gain correction value calculated by the correction value calculation method of Claim 11 or 12 An imaging method, wherein a value is corrected and output.

Images