JP5423935B1 - Imaging device - Google Patents

Abstract

The photoelectric conversion characteristics of the image sensor (110) under the reference imaging conditions are classified, and the classification number is stored in the classification number memory (121). The photoelectric conversion characteristics of the pixels with the classification number i are averaged to obtain the representative characteristics of the classification number i under the reference imaging condition, and the LUT for converting the representative characteristics of the classification number i into a predetermined output characteristic is stored in the characteristic conversion LUT (125). To do. The photoelectric conversion characteristics under the non-reference imaging conditions are averaged for each classification number i to obtain the representative characteristics of the classification number i under the non-reference imaging conditions. The LUT of the representative characteristic with the classification number i under the non-reference imaging condition is stored in the characteristic conversion LUT (125). The photoelectric conversion characteristic of the pixel j under the non-reference imaging condition and each representative characteristic under the non-reference imaging condition are compared, and the difference between the classification number of the representative characteristic closest to the photoelectric conversion characteristic of the pixel j and the classification number of the pixel j Are stored in the classification number correction value memory (122) as the classification number correction value of the pixel j in the imaging condition B.

Description

  The present invention relates to an imaging apparatus that corrects variation in output characteristics in an imaging apparatus having different photoelectric conversion characteristics for each pixel.

  In recent years, in order to expand the dynamic range, an imaging apparatus having a linear characteristic on the low luminance side and a log characteristic on the high luminance side with an inflection point as a boundary is known. Such photoelectric conversion characteristics are called linear log characteristics. Hereinafter, in the linear log characteristics, an area having linear characteristics is described as a linear area, and an area having log characteristics is described as a log area.

  Due to variations in the characteristics of the pixels included in the image sensor, the photoelectric conversion characteristics vary from pixel to pixel, and variations in the photoelectric conversion characteristics of each pixel when the imaging conditions change also vary from pixel to pixel. For this reason, when the imaging conditions (the temperature of the imaging device, the shutter speed, etc.) change, it is necessary to correct the pixel value output by each pixel according to the characteristic change for each pixel by converting it to a predetermined output characteristic. is there.

  Patent Document 1 describes a method of measuring photoelectric conversion characteristics and temperature characteristics for each pixel, holding them as correction coefficients for each pixel, and correcting captured image data.

  However, in the case of the method described in Patent Document 1, if the correction coefficient of each pixel is held for each photographing condition, the amount of data is large, so that the memory capacity increases, and the substrate increases as the cost increases and the memory IC increases. There is a problem that it is difficult to reduce the size. Further, correction of image data using a correction coefficient has poor correction accuracy. In particular, the sensor current at the inflection point is the same for all pixels, but actually differs for each pixel. Therefore, the correction error near the inflection point becomes large and the image quality is deteriorated.

JP 2002-344817 A

  An object of the present invention is to provide an imaging apparatus that corrects pixel values with high accuracy while suppressing the amount of data necessary for correction.

  An imaging device according to an aspect of the present invention classifies an image sensor having a plurality of pixels, photoelectric conversion characteristics of each pixel under a predetermined reference imaging condition, and assigns a classification number to each pixel and the classification number. And a photoelectric conversion characteristic for each imaging condition including the reference imaging condition of each pixel is averaged for each classification number to obtain a representative characteristic in advance, and the representative characteristic A look-up table storage unit for storing a look-up table for converting image data into predetermined output characteristics, and a photoelectric conversion characteristic for each imaging condition of each pixel based on a classification number of each pixel in the reference imaging condition. A correction information storage unit that stores in advance classification number correction information for specifying a lookup table corresponding to a close representative characteristic, and the classification number storage for each pixel at the time of shooting A reading unit that reads out the classification number from the correction information storage unit, reads out the classification number correction information corresponding to the imaging condition from the correction information storage unit, and a correction unit that corrects the classification number based on the read out classification number and the classification number correction information; Based on the correction classification number corrected by the correction unit, the lookup table of each pixel is read from the lookup table storage unit, and the pixel value output by each pixel is converted using the lookup table. A conversion unit.

1 is a block diagram illustrating an overall configuration of an imaging apparatus according to a first embodiment. The figure which showed the flow of the production method of the data memorize | stored in the classification number memory in 1st Embodiment, the classification number correction value memory, and the characteristic conversion LUT. The figure which showed the flow of the production method of the data memorize | stored in the classification number memory in 1st Embodiment, the classification number correction value memory, and the characteristic conversion LUT. The figure which showed an example of the data structure of the classification number memory in 1st Embodiment. The figure which showed an example of the data structure of a classification number memory. The figure which showed the photoelectric conversion characteristic in the imaging condition A of the pixel classified into the classification number i, and the representative characteristic which is the average value. The figure which showed the example of a representative characteristic and LUT, and the output after conversion. The figure which showed an example of the data structure of LUT. The photoelectric conversion characteristic of each pixel belonging to the classification number i of the imaging element under the imaging condition B and the photoelectric conversion characteristic of each pixel belonging to the classification number i + 1. FIG. 10 is an enlarged view of a part of the graph of FIG. 9. The figure which showed an example of the data structure of the classification number correction value memory in 1st Embodiment. The figure which showed the flow of the conversion process of the pixel value which the image sensor output at the time of imaging | photography. The block diagram which shows the whole structure of the imaging device in 2nd Embodiment. The block diagram which shows the whole structure of the imaging device in 3rd Embodiment. The figure which showed the flow of the production method of the data memorize | stored in the classification number memory in 3rd Embodiment, the classification number correction | amendment LUT, and the characteristic conversion LUT. The figure which showed the flow of the production method of the data memorize | stored in the classification number memory in 3rd Embodiment, the classification number correction | amendment LUT, and the characteristic conversion LUT. The figure which showed the photoelectric conversion characteristic of the pixel in each imaging condition, and the classification number of the representative characteristic nearest to the photoelectric conversion characteristic. A table summarizing classification number correction values. The figure which showed an example of the data structure of the classification number memory in 3rd Embodiment. The figure which showed an example of the data structure of the classification number correction | amendment LUT in 3rd Embodiment. The figure which showed the flow of the conversion process of the pixel value which the image pick-up element at the time of imaging | photography in 3rd Embodiment output. The block diagram which shows the whole structure of the imaging device in 4th Embodiment.

[First Embodiment]
FIG. 1 is a block diagram illustrating the overall configuration of the imaging apparatus 1 according to the first embodiment. An imaging apparatus 1 illustrated in FIG. 1 is an imaging apparatus having a linear log characteristic photoelectric conversion characteristic in which a linear characteristic and a log characteristic are switched at an inflection point. Specifically, the image pickup apparatus 1 according to the first embodiment is an image pickup apparatus having a photoelectric conversion characteristic of a linear log characteristic having a linear characteristic on the low luminance side from the inflection point and a log characteristic on the high luminance side.

  The imaging device 1 includes an imaging element 110 and an image processing unit 120 (reading unit, conversion unit). The image sensor 110 and the image processing unit 120 may be configured in one IC chip or may be configured as separate IC chips.

  The image processing unit 120 includes a classification number memory 121 (classification number storage unit), a classification number correction value memory 122 (correction information storage unit), an adder 124 (correction unit), and a characteristic conversion LUT 125 (lookup table storage unit). . Note that the image processing unit 120 normally includes a control unit that generates a control signal for controlling the operation of the image sensor 110, but description thereof is omitted in the present embodiment.

  The image sensor 110 outputs a pixel value (image signal) to the image processing unit 120. The image processing unit 120 reads out the classification number from the classification number memory 121 and further reads out the classification number correction value from the classification number correction value memory 122 according to the imaging conditions. The adder 124 adds the read classification number and the classification number correction value, and outputs the result as a corrected classification number. The image processing unit 120 extracts a corresponding LUT from the characteristic conversion LUT 125 based on the imaging condition and the correction classification number, converts the pixel value based on the extracted LUT, and outputs it as a pixel output value to an external device. Here, the external device corresponds to a display device such as a liquid crystal panel or an organic EL panel, a memory for holding an image output value, or the like.

  As the imaging condition, at least one or a combination of the shutter speed of the image sensor 110 and the temperature of the image sensor 110 is employed.

  Hereinafter, a method for creating data and a data configuration stored in the classification number memory 121, the classification number correction value memory 122, and the characteristic conversion LUT 125 will be described in detail. 2 and 3 are diagrams showing a flow of a method for creating data stored in the classification number memory 121, the classification number correction value memory 122, and the characteristic conversion LUT 125. FIG. Each data is measured and calculated before factory shipment of the imaging apparatus 1, and is stored in each memory.

  First, the photoelectric conversion characteristics of the image sensor 110 under the reference imaging conditions (hereinafter referred to as imaging conditions A) are measured (step S11). Then, the photoelectric conversion characteristics acquired by measurement are classified, and a classification number is associated with each pixel and stored in the classification number memory 121 (step S12).

  The graph on the left side of FIG. 4 shows the photoelectric conversion characteristics of each pixel of the image sensor 110 under the imaging condition A. The graph on the right side is an enlarged partial area of the graph. A maximum and minimum range of pixel values at a predetermined luminance is divided into allowable conversion error widths, and a classification number is assigned to each pixel. FIG. 5 is a diagram showing an example of the data configuration of the classification number memory 121. In the figure, i and n are integers of 1 or more, n is the maximum value of the classification number, x and y are integers of 0 or more, and indicate the number of pixels arranged in a matrix on the image sensor 110.

  Subsequently, the photoelectric conversion characteristics of the pixels classified into the classification number i are averaged to obtain a representative characteristic of the classification number i under the imaging condition A (i = 1 to n, step S13). FIG. 6 is a diagram showing the photoelectric conversion characteristics of the pixels classified into the classification number i under the imaging condition A and the representative characteristics that are average values thereof.

  Subsequently, an LUT for converting the representative characteristic of the classification number i under the imaging condition A into a predetermined output characteristic is created and stored in the characteristic conversion LUT 125 (step S14). The predetermined predetermined output characteristic is, for example, a linear characteristic or a log characteristic. If the predetermined output characteristic is a log characteristic, the representative characteristic having the linear log characteristic is converted into the log characteristic via the LUT. FIG. 7 is a diagram showing an example of representative characteristics and LUTs, and output characteristics after conversion. The first quadrant in the upper right is a representative characteristic of classification number i, the horizontal axis indicates the sensor surface illuminance (logarithmic scale), and the vertical axis indicates the pixel value. The lower left third quadrant indicates the LUT of classification number i. The fourth quadrant in the lower right indicates an output characteristic obtained by converting the representative characteristic using the LUT.

  Subsequently, the photoelectric conversion characteristics of the image sensor 110 under non-reference imaging conditions (hereinafter referred to as imaging conditions B) other than the reference imaging conditions are measured (step S15), and the photoelectric conversion characteristics of the pixels classified into the classification number i are averaged. Then, the representative characteristic of the classification number i under the imaging condition B is set (i = 1 to n, step S16). Then, an LUT for converting the representative characteristic of the classification number i under the imaging condition B into a predetermined output characteristic is created and stored in the characteristic conversion LUT 125 (step S17). FIG. 8 is a diagram illustrating an example of a data configuration of the characteristic conversion LUT 125. The characteristic conversion LUT 125 stores the LUT of each classification number for each imaging condition.

  Subsequently, the photoelectric conversion characteristic of the pixel j under the imaging condition B is compared with each representative characteristic under the imaging condition B, and the classification number of the representative characteristic closest to the photoelectric conversion characteristic of the pixel j and the classification number of the pixel j are calculated. Calculate the difference. Then, the difference is stored in the classification number correction value memory 122 as the classification number correction value of the pixel j in the imaging condition B (j = 1 to m, step S18). Here, m represents the number of pixels of the image sensor 110. In FIG. 5, xy coordinates are used to indicate each pixel, but for the sake of simplicity of explanation, a variable j is appropriately used below. Each indicates each pixel.

  FIG. 9 shows the photoelectric conversion characteristics (solid line) of each pixel belonging to the classification number i of the image sensor 110 under the imaging condition B and the photoelectric conversion characteristics (dotted line) of each pixel belonging to the classification number i + 1. Further, in FIG. 9, the thick solid line indicates the representative characteristics of the classification numbers i and i + 1.

  FIG. 10 is an enlarged view of a part of the graph of FIG. For example, the photoelectric conversion characteristic of the pixel (x0, y0) belonging to the classification number i is closer to the representative characteristic of the classification number i + 1 than the representative characteristic of the classification number i. That is, if the pixel value of the pixel (x0, y0) is converted using the LUT based on the representative characteristic of the classification number i, the correction accuracy is deteriorated.

  Therefore, the pixel value of the pixel (x0, y0) is converted using the LUT based on the representative characteristic of the classification number i + 1. That is, the difference between the classification number i to which the pixel (x0, y0) belongs and the classification number i + 1 to which the LUT used to convert the pixel value of the pixel (x0, y0) under the shooting condition B is +1. The classification number correction value memory 122 stores the difference +1 in association with the imaging condition B for the pixel (x0, y0).

  FIG. 11 is a diagram showing an example of the data configuration of the classification number correction value memory 122. As shown in FIG. In FIG. 11, since the imaging condition A is an imaging condition in which a classification number is created as a reference imaging condition, correction of the classification number is unnecessary. Therefore, the imaging condition A in the classification number correction value memory 122 is set to all pixels 0. Alternatively, the classification number correction value of the reference imaging condition may not be stored.

  In the above description, only the imaging condition B is described as the non-reference imaging condition. However, the output characteristics of the imaging element 110 vary depending on the temperature of the imaging apparatus 1, the shutter speed, and the like. Therefore, it is preferable to repeat the processing of steps S15 to S18 shown in FIG. 2 for a plurality of different non-reference imaging conditions, and store the data for each imaging condition in the classification number correction value memory 122 and the characteristic conversion LUT 125.

  Next, processing during actual shooting will be described. FIG. 12 is a diagram illustrating a flow of conversion processing of pixel values output from the image sensor 110 at the time of shooting. First, the image processing unit 120 reads out the classification number of the pixel j from the classification number memory 121 (step S21), and further displays the classification number correction value of the pixel j under the imaging condition (imaging condition Z) at the time of shooting. Read from 122 (step S22). The adder 124 adds the read classification number and the classification number correction value, and outputs the result as a corrected classification number (step S23).

  For example, in the case of the imaging condition A, as shown in FIGS. 5 and 11, since the classification number of the pixel (0, 0) is 1 and the classification number correction value is 0, the adder 124 is a sum of 1 + 0. Is output as a correction classification number. In the case of the imaging condition A, the classification number correction value is 0 for all the pixels, so that the image processing unit 120 uses the classification number of each pixel read from the classification number memory 121 without using the adder. A corresponding LUT may be extracted from the conversion LUT 125.

  In the case of the imaging condition B, since the classification number of the pixel (0, 0) is 1 and the classification number correction value is +2, the adder 124 outputs 3 which is the sum of 1 + 2 as the correction classification number.

  Next, the image processing unit 120 reads the correction classification number LUT under the imaging condition Z from the characteristic conversion LUT 125, and converts the pixel value of the pixel j output from the imaging element 110 (step S24). The image processing unit 120 performs the processes of steps S21 to S24 for all pixels.

  As described above, the photoelectric conversion characteristics of each pixel are classified, the classified photoelectric conversion characteristics are averaged as representative characteristics, and a look-up table for converting the representative characteristics to predetermined characteristics is used as the characteristic conversion LUT 125. Therefore, the amount of data can be reduced compared to storing a lookup table for each pixel.

  The classification number memory 121 stores only the classification number in the basic imaging condition (imaging condition A), and the adder 124 classifies the classification number of the representative characteristic closest to the photoelectric conversion element of each pixel in the non-reference imaging condition. It can be acquired by adding a difference to the classification number read from the storage unit. That is, since the classification number is not stored for each pixel and each imaging condition, the data amount can be reduced.

  In addition, when the imaging conditions change, conversion is performed using a look-up table with representative characteristics closest to the photoelectric conversion characteristics under the changed imaging conditions, so characteristics including inflection points and non-linearities in the log area Conversion is possible, and correction can be performed with higher accuracy than in the past.

[Second Embodiment]
FIG. 13 shows ROM (Read Only Memory) 81 as data in the classification number memory 121 (classification number 811) and data in the classification number correction value memory 122 (classification number correction value 812) in the imaging apparatus 1 of the first embodiment. (Classification number storage unit, correction information storage unit) Characteristic that is stored in a software process by a CPU (Central Processing Unit) 84 (reading unit, correction unit, conversion unit) for adding a classification number and a classification number correction value. It is a block diagram of the imaging device 2 which performs conversion. A classification number RAM (Random Access Memory) 82 stores the classification number of each pixel corrected by software processing by the CPU 84. The bus 83 is a data line wired between the CPU 84, the ROM 81, and the classification number RAM 82.

  The flow of conversion processing of the pixel value output from the image sensor 110 at the time of shooting according to the present embodiment will be described with reference to FIG. First, the CPU 84 reads out the classification number of each pixel from the ROM 81 at the start of shooting or when the imaging condition changes (step S21), and further reads out the correction value of the classification number of each pixel under the imaging condition (imaging condition Z) from the ROM 81 ( Step S22). Then, the CPU 84 adds the read classification number and the classification number correction value, and stores them in the classification number RAM 82 as the corrected classification number of each pixel (step S23).

  At the time of shooting, the correction classification number corresponding to the pixel position is read from the classification number RAM 82, and the pixel value output from the image sensor 110 using the LUT corresponding to the correction classification number of the imaging condition Z in the characteristic conversion LUT 125 is obtained. Conversion is performed (step S24).

[Third Embodiment]
In the first embodiment, the imaging apparatus 1 in which the classification number correction value memory 122 holds the difference between the classification numbers of the pixels has been described. However, in this method, if the number of pixels is large, the data amount of the classification number correction value memory 122 increases, and it is highly likely that further memory capacity reduction is required. Therefore, in the present embodiment, a method for further reducing the amount of data used for correction of the classification number by classifying the change patterns through the entire imaging conditions of the difference of the classification numbers (classification number correction value) will be described.

  FIG. 14 is a block diagram illustrating an overall configuration of the imaging apparatus 3 according to the third embodiment. In addition, the same code | symbol is attached | subjected to the thing similar to the component of the imaging device 3 shown in FIG. 1 in 1st Embodiment, and description is abbreviate | omitted.

  The image processing unit 220 (reading unit, conversion unit) includes a classification number memory 221 (a classification number storage unit, a correction information storage unit, a characteristic change classification number storage unit), and a classification number correction LUT 222 (a correction information storage unit, a characteristic change pattern storage). Section), an adder 124 (correction section), and a characteristic conversion LUT 125 (lookup table storage section).

  The image sensor 110 outputs the pixel value to the image processing unit 220. The image processing unit 220 reads out the classification number and the characteristic change classification number from the classification number memory 221 and converts the characteristic change classification number into a classification number correction value based on the classification number correction LUT 222 set according to the imaging condition. The adder 124 adds the classification number correction value and the classification number read from the classification number memory 221 and outputs the result as a corrected classification number. The image processing unit 220 extracts a corresponding LUT from the characteristic conversion LUT 125 based on the imaging condition and the correction classification number, converts the pixel value based on the extracted LUT, and outputs it as a pixel output value to an external device.

  Hereinafter, a method of creating data and a data configuration stored in the classification number memory 221 and the classification number correction LUT 222 will be described in detail. 15 and 16 are diagrams showing a flow of a method for creating data stored in the classification number memory 221, the classification number correction LUT 222, and the characteristic conversion LUT 125. Here, steps S31 to S37 in the flowchart shown in FIG. 15 are the same as steps S11 to S17 in the flowchart shown in FIG.

  After step S37, the photoelectric conversion characteristic of the pixel j under the imaging condition B is compared with each representative characteristic under the imaging condition B, and the classification number of the representative characteristic closest to the photoelectric conversion characteristic of the pixel j and the classification number of the pixel j are Are calculated and summarized in a table (j = 1 to m, step S38). Here, Step S35 to Step S38 are repeated for a plurality of different non-reference imaging conditions.

  FIG. 17 shows the photoelectric conversion characteristics of the pixels under each imaging condition and the representative characteristic classification numbers closest to the photoelectric conversion characteristics. Hereinafter, for easy understanding, description will be made using four pixels a, b, c, and d as representatives among the pixels of the image sensor 110.

  FIG. 17A shows the photoelectric conversion characteristics of the pixels a to d under the imaging condition A. The photoelectric conversion characteristic of the pixel a is closest to the representative characteristic of the classification number # 100 under the imaging condition A, and the photoelectric conversion characteristic of the pixel b is closest to the representative characteristic of the classification number # 101 under the imaging condition A. The pixels c and d are as shown in the figure.

  FIG. 17B shows the photoelectric conversion characteristics of the pixels a to d under the imaging condition B. The photoelectric conversion characteristic of the pixel a is closest to the representative characteristic of the classification number # 101 under the imaging condition B, and It is assumed that the photoelectric conversion characteristic is closest to the representative characteristic of classification number # 102 under the imaging condition B. The pixels c and d are as shown in the figure.

  FIG. 17C shows the photoelectric conversion characteristics of the pixels a to d under the imaging condition C. The photoelectric conversion characteristic of the pixel a is closest to the representative characteristic of the classification number # 102 under the imaging condition C, and It is assumed that the photoelectric conversion characteristic is closest to the representative characteristic of classification number # 103 under the imaging condition C. The pixels c and d are as shown in the figure. In each imaging condition, the difference (classification number correction value) between the classification number of each pixel and the classification number of the representative characteristic closest to the photoelectric conversion characteristic of the pixel is collected.

  FIG. 18 is a table summarizing the above results. Then, the same characteristic change classification number is assigned to the pixels having the same difference change pattern (step S39). For example, the pixel a and the pixel b have the same change pattern of the difference, that is, the classification number correction value under the imaging condition B is +1 and the classification number correction value under the imaging condition C is +2. Therefore, a unique characteristic change classification number is assigned to this change pattern. Similarly, a characteristic change classification number is assigned to each of the change pattern of the pixel c and the change pattern of the pixel d.

  Next, the classification number and characteristic change classification number of each pixel are stored in the classification number memory 221 (step S40). FIG. 19 is a diagram showing an example of the data configuration of the classification number memory 221. As shown in FIG. Pixels having the same change pattern of differences summarized in step S38 are stored in association with the same characteristic change classification number.

  Next, the change pattern of the difference corresponding to each characteristic change classification number is stored in the classification number correction LUT 222 (step S41). FIG. 20 is a diagram illustrating an example of the data configuration of the classification number correction LUT 222. Since the imaging condition A is a reference imaging condition, each characteristic change classification number is 0. Alternatively, the characteristic classification number of the reference imaging condition may not be stored.

  Next, processing during actual shooting will be described. FIG. 21 is a diagram illustrating a flow of conversion processing of the pixel value output from the image sensor 110 at the time of shooting. First, the image processing unit 220 reads out the classification number and characteristic change classification number of the pixel j from the classification number memory 221 (step S51). Further, the image processing unit 220 derives from the classification number correction LUT 222 a classification number correction value under the imaging condition (imaging condition Z) of the read characteristic classification number (step S52). The adder 124 adds the classification number read in step S51 and the classification number correction value derived in step S52, and outputs the result as a corrected classification number (step S53).

  For example, in the case of the imaging condition A, the classification number of the pixel a is 100 and the characteristic change classification number is 0. Further, the classification number correction value under the imaging condition A with the characteristic change classification number 0 is 0. Therefore, the adder 124 outputs 100, which is the sum of 100 + 0, as the correction classification number. In the case of the imaging condition A, since the classification number correction value is 0 for all pixels, the characteristic is based on the classification number of each pixel read from the classification number memory 121 by the image processing unit 220 without using the adder 124. A corresponding LUT may be extracted from the conversion LUT 125.

  In the case of the imaging condition B, the classification number of the pixel a is 100 and the characteristic change classification number is 0. Furthermore, the classification number correction value in the imaging condition B with the characteristic change classification number 0 is +1. Therefore, the adder 124 outputs 101, which is the sum of 100 + 1, as the correction classification number.

  Next, the image processing unit 220 extracts the correction classification number LUT under the imaging condition Z from the characteristic conversion LUT 125, and converts the pixel value of the pixel j output from the imaging device 110 (step S54). The image processing unit 120 performs the processes of steps S51 to S54 for all the pixels.

  In this embodiment, four pixels and three imaging conditions have been described as examples. However, since the number of characteristic change classification numbers increases as the number of pixels and imaging conditions increase, the contents of the classification number correction LUT 222 are all classified. It is not realistic to prepare as many numbers as possible. In such a case, in step S39 in FIG. 15, the characteristic change classification number is assigned so that the error of the classification number correction value is minimized (that is, the approximate change pattern is assigned the same characteristic change classification number). You may make it reduce the number of change classification numbers below the number which can be mounted in classification number correction | amendment LUT222.

  As described above, the photoelectric conversion characteristics of each pixel are classified, the classified photoelectric conversion characteristics are averaged as representative characteristics, and a look-up table for converting the representative characteristics to predetermined characteristics is used as the characteristic conversion LUT 125. Therefore, the amount of data can be reduced compared to storing a lookup table for each pixel. In addition, when the imaging conditions change, conversion is performed using a look-up table with representative characteristics closest to the photoelectric conversion characteristics under the changed imaging conditions, so characteristics including inflection points and non-linearities in the log area Conversion is possible, and correction can be performed with higher accuracy than in the past.

  Further, the same characteristic change classification number is assigned to pixels whose difference between the classification number of each pixel and the classification number of the representative characteristic closest to the photoelectric conversion characteristic of each pixel is the same or approximated through all imaging conditions, The classification number correction LUT 222 stores the difference between the imaging conditions for each characteristic change classification number. Therefore, it is only necessary to store the classification number and the characteristic change classification number for each pixel (classification number memory 221). Therefore, the data amount can be reduced.

  When the imaging condition changes, the image processing unit 220 reads the characteristic change classification number from the classification number memory 221 and causes the adder 124 to perform addition processing, and then extracts a new LUT from the characteristic conversion LUT 125. Less data is read than in the first embodiment. Therefore, quick processing is possible with respect to changes in imaging conditions.

[Fourth Embodiment]
FIG. 22 shows the data of the classification number memory 221 (classification number 911) and the data of the classification number correction LUT 222 (classification number correction LUT912) in the imaging apparatus 4 of the third embodiment stored in the ROM 91, and the classification number and the classification number. It is a block diagram of imaging device 4 which performs characteristic conversion which performs addition processing of correction value by software processing by CPU84 (reading part, amendment part, conversion part).

  The flow of the conversion process of the pixel value output from the image sensor 110 at the time of shooting according to the present embodiment will be described with reference to FIG. First, the CPU 84 reads out the classification number and characteristic change classification number of each pixel from the ROM 91 (classification number storage unit, correction information storage unit) at the start of imaging or when the imaging condition changes (step S51). Further, the CPU 84 derives the classification number correction value of the read characteristic classification number under the imaging condition Z from the classification number correction LUT 912 (step S52), and the classification number read in step S51 and the classification number derived in step S52. The correction value is added and stored in the classification number RAM 82 as a correction classification number (step S53).

  At the time of shooting, the correction classification number corresponding to the pixel j is read from the classification number RAM 82 and the pixel j output from the image sensor 110 using the LUT corresponding to the correction classification number of the imaging condition Z in the characteristic conversion LUT 125. Pixel values are converted (step S54).

(Summary of this embodiment)
The imaging apparatus classifies an image sensor having a plurality of pixels, photoelectric conversion characteristics of each pixel under a predetermined reference imaging condition, assigns a classification number, and associates each pixel with the classification number. A classification number storage unit that stores in advance, and photoelectric conversion characteristics for each imaging condition including the reference imaging condition of each pixel are averaged for each classification number to obtain a representative characteristic in advance, and the representative characteristic is output in a predetermined manner Based on the look-up table storage unit for storing the look-up table for conversion into characteristics in advance and the classification number of each pixel in the reference imaging condition, the representative characteristic closest to the photoelectric conversion characteristic for each imaging condition is obtained. A correction information storage unit that stores in advance classification number correction information for specifying a corresponding lookup table, and the above-mentioned classification number storage unit for each pixel at the time of shooting. A reading unit that reads out the classification number correction information corresponding to the imaging condition from the correction information storage unit, a correction unit that corrects the classification number based on the read out classification number and the classification number correction information, and the correction A conversion unit that reads the lookup table of each pixel from the lookup table storage unit based on the correction classification number corrected by the unit, and converts the pixel value output by each pixel using the lookup table; .

  According to this configuration, the photoelectric conversion characteristics of each pixel are classified, and the classified photoelectric conversion characteristics are averaged to be representative characteristics, and a lookup table that converts the representative characteristics into predetermined characteristics is a lookup table. Stored in the storage unit. Therefore, the amount of data can be reduced as compared with storing a lookup table for each pixel.

  The classification number storage unit stores only the classification number in the basic imaging condition. For the classification numbers in other imaging conditions, the correction unit stores the classification number stored in the classification number storage unit as the classification number correction information. It can be acquired by using and correcting. That is, since the classification number is not stored for each pixel and each imaging condition, the data amount can be reduced.

  In addition, when the imaging conditions change, pixel values are converted using a look-up table with representative characteristics closest to the photoelectric conversion characteristics under the changed imaging conditions. It is possible to convert the characteristics including those, and to correct with higher accuracy than in the past.

  In the above configuration, the correction information storage unit associates each pixel with a difference between the classification number of the representative characteristic closest to the photoelectric conversion characteristic for each imaging condition of the pixel and the classification number of the pixel. The correction unit may perform correction by adding the difference of each pixel to the classification number of each pixel read by the reading unit.

  According to this configuration, the classification number storage unit stores only the classification number in the basic imaging condition, and for the classification number of the representative characteristic closest to the photoelectric conversion characteristic of each pixel in the other imaging conditions, the correction unit It can be acquired by adding the difference read from the correction information storage unit to the classification number stored in the storage unit. That is, since the classification number is not stored for each pixel and each imaging condition, the data amount can be reduced.

  In the above configuration, the correction information storage unit has the same difference between the classification number of the representative characteristic closest to the photoelectric conversion characteristic for each imaging condition of each pixel and the classification number of each pixel through all the imaging conditions. Alternatively, a characteristic change classification number storage unit that assigns the same characteristic change classification number to pixels that are approximated and stores the same in advance, and a characteristic that stores the difference in each imaging condition in advance in association with the characteristic change classification number A change pattern storage unit, and the reading unit reads out the classification number from the classification number storage unit for each pixel, and reads out the characteristic change classification number from the characteristic change classification number storage unit, Based on the read characteristic change classification number and the imaging condition, the difference is read from the characteristic change pattern storage unit, and the correction unit reads each pixel read The difference of each pixel in the classification number may be corrected by adding.

  According to this configuration, the classification number of each pixel, the classification number of the representative characteristic closest to the photoelectric conversion characteristic of each pixel, and the same characteristic change classification number for pixels whose differences are the same or approximated through all imaging conditions. Further, since the characteristic change pattern storage unit stores the difference between the imaging conditions for each characteristic change classification number, it is only necessary to store the classification number and the characteristic change classification number for each pixel. Therefore, the data amount can be reduced.

  In the above configuration, the imaging condition may be defined by at least one of shutter speed and temperature of the imaging device.

  According to this configuration, even if the shutter speed of the image sensor or the temperature of the image sensor fluctuates, the pixel value is converted using an appropriate lookup table.

  In the above configuration, the photoelectric conversion characteristic is a linear log characteristic in which the low luminance side is a linear characteristic and the high luminance side is a log characteristic, and the lookup table converts the linear log characteristic into an output characteristic of a linear characteristic or a log characteristic. May be.

  According to this configuration, the photoelectric conversion characteristic of each pixel value is unified from the linear log characteristic to the linear characteristic or log characteristic, so that the pixel value is converted into data suitable for image processing.

Claims (5)

An imaging device having a plurality of pixels;
A classification number storage unit that classifies photoelectric conversion characteristics of each pixel in a predetermined reference imaging condition and assigns a classification number, and associates and stores the respective pixels and the classification number in advance;
A photoelectric conversion characteristic for each imaging condition including the reference imaging condition for each pixel is averaged for each classification number to obtain a representative characteristic in advance, and a lookup table for converting the representative characteristic into a predetermined output characteristic A lookup table storage unit for storing in advance;
Correction information for storing in advance classification number correction information for specifying a lookup table corresponding to the representative characteristic closest to the photoelectric conversion characteristic for each imaging condition of each pixel based on the classification number of each pixel in the reference imaging condition A storage unit;
A reading unit that reads out the classification number from the classification number storage unit and reads out the classification number correction information corresponding to the imaging condition from the correction information storage unit for each pixel during shooting;
A correction unit that corrects the classification number based on the read classification number and classification number correction information;
Conversion that reads the lookup table of each pixel from the lookup table storage unit based on the correction classification number corrected by the correction unit, and converts the pixel value output by each pixel using the lookup table And
An imaging apparatus comprising: The correction information storage unit stores in advance the difference between the classification number of the representative characteristic closest to the photoelectric conversion characteristic for each imaging condition of the pixel and the classification number of the pixel in association with the pixel,
The imaging apparatus according to claim 1, wherein the correction unit performs correction by adding a difference of each pixel to a classification number of each pixel read by the reading unit. The correction information storage unit
The same characteristic change classification for pixels in which the difference between the classification number of the representative characteristic closest to the photoelectric conversion characteristic for each imaging condition of each pixel and the classification number of each pixel is the same or approximated through all imaging conditions A characteristic change classification number storage unit that assigns numbers and stores them in advance;
A characteristic change pattern storage unit that stores the difference in each imaging condition in advance in association with the characteristic change classification number;
Have
The reading unit reads out the classification number from the classification number storage unit for each of the pixels, and reads out the characteristic change classification number from the characteristic change classification number storage unit. Read the difference from the characteristic change pattern storage unit based on,
The imaging apparatus according to claim 1, wherein the correction unit performs correction by adding a difference of each pixel to the read classification number of each pixel.   The imaging apparatus according to claim 1, wherein the imaging condition is defined by at least one of a shutter speed and a temperature of the imaging element. The photoelectric conversion characteristics are linear log characteristics in which the low luminance side is linear characteristics and the high luminance side is log characteristics,
The imaging device according to claim 1, wherein the look-up table converts the linear log characteristic into a linear characteristic or an output characteristic of a log characteristic.

Images