JP2012134779A - Image signal processing apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image signal processing apparatus for solving a problem of inability to perform an automatic white balance processing properly because of inability to accurately determine the color temperature of a light source from an image signal when performing an automatic white balance processing using a camera equipped with an imaging device having a nonlinear input-output characteristics.SOLUTION: An image signal processing apparatus includes: an image pickup device whose photoelectric transfer characteristic is nonlinear; image input means for inputting signals from the image pickup device; signal level detection means for detecting a signal level from the image input means; white balance adjustment amount calculation means for calculating a white balance adjustment amount based on the signals from the image input means; automatic white balance processing means for performing automatic white balance processing based on the signal from the white balance adjustment amount calculation means; and white balance system control means for collectively controlling the white balance. The white balance system control means controls the white balance processing means in accordance with the white balance adjustment amount calculated by the white balance adjustment amount calculation means.

Description

  The present invention relates to an image signal processing apparatus.

  As background art in this technical field, for example, there is JP 2010-114667 A (Patent Document 1). In this publication, “[PROBLEM] To obtain an automatic white balance adjustment system that accurately adjusts white balance of an image. [Solution] Color range information or luminance indicating a predetermined range of a color difference signal in an automatic white balance adjustment system. A white balance information calculation unit 16 that calculates an adjustment gain for white balance adjustment using pixels in the color range information or the luminance range information based on luminance range information indicating a predetermined range of the signal, and is used for calculating the adjustment gain. At least one of color range information or luminance range information, and in-frame region information indicating a predetermined region in one frame used for calculation of adjustment gain when used for calculation of adjustment gain, is a captured image. Register switching unit 19 that changes every frame of the captured image according to the white balance of the image, and white balance information A multiplier 12 for adjusting the white balance of the captured image using the adjusted gain output unit 16 is calculated, and is described as comprising. "(See abstract).

  Further, for example, there is JP-A-2002-77733. In this publication, “[Problem] The present invention is capable of automatically switching between a logarithmic conversion operation and a linear conversion operation according to the amount of incident light incident on the photoelectric conversion unit without switching the bias voltage. [Solution] The gate of the MOS transistor T1 at the start of imaging is given to the signal φVPS by applying a pulse signal having a voltage VL lower than the voltage VH applied to the source of the MOS transistor T1 during imaging. The voltage is set to be lower than the source voltage, so that during imaging, the MOS transistor T1 is cut off until the subject exceeds a predetermined luminance value, so that a linearly converted electric signal is output, When the subject exceeds a predetermined luminance value, the MOS transistor T1 operates in the subthreshold region. A number-converted electrical signal is output ”(see summary).

JP 2010-114667 A JP 2002-77733 A

  The imaging device that outputs the image signal after photoelectrically converting the light of the subject as input is set to nonlinear input / output characteristics so as to suppress pixel saturation as much as possible, and shooting with the imaging device having the nonlinear input / output characteristics enables wide dynamic range. There is technology to acquire images.

  In general, it is known that an image signal processing apparatus that performs photographing performs automatic white balance processing for automatically obtaining appropriate white balance. In this method, the color temperature of the light source is determined from the photographed image, and the white balance is automatically controlled to be appropriate. For example, when a subject illuminated by sunlight at dusk is projected, a white subject appears red without white balance processing, but a white subject can be photographed white by performing white balance processing.

  However, in the wide dynamic range image generation technique (Japanese Patent Laid-Open No. 2002-77733), an image sensor having nonlinear input / output characteristics realizes nonlinear input / output characteristics by logarithmic compression conversion or the like. It becomes a compressed image. For this reason, a camera using an imaging element having nonlinear input / output characteristics by automatic white balance processing (Japanese Patent Laid-Open No. 2010-114667) targeted for an image processing technique having linear input / output characteristics as in the above example. When performing automatic white balance processing at, there is a problem in that the color temperature of the light source cannot be accurately determined from the image signal and automatic white balance processing cannot be performed properly.

  In order to solve the above object, the configuration described in the claims is adopted.

  According to the present invention, it is possible to automatically provide an appropriate white balance when using a non-linear image processing technique.

  Problems, configurations, and effects other than those described above will be clarified by the following description of embodiments.

The figure explaining an example of the basic composition in Example 1 of this invention Diagram explaining an example of nonlinear input / output characteristics The figure explaining UV coordinate system of a color difference signal The figure explaining an example of the method of calculating white balance adjustment amount Flow chart explaining the flow of processing to calculate the white balance adjustment amount Flow diagram for explaining the flow of the technique to cope with the processing when calculating the white balance adjustment amount when the signal output from the imaging unit is non-linear The figure explaining an example of white balance adjustment amount calculation processing The figure explaining an example of the basic composition in Example 2 Flow chart for explaining the flow of white balance processing using correction processing in the output characteristic control unit 809 (FIG. 8) Flow chart explaining the flow of a technique for performing white balance processing corresponding to a nonlinear signal in the input / output characteristic correspondence processing unit and the white balance adjustment amount calculation unit FIG. 6 is a diagram for explaining an example of a basic configuration in the third embodiment. The flowchart which shows the flow of the white balance process in the structure of FIG. The figure explaining an example of the effect of Example 3

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(1) System Configuration of Embodiment 1 Embodiment 1 according to the present invention will be described in detail. FIG. 1 is a diagram illustrating an example of a basic configuration according to the first embodiment of the present invention. The imaging unit 101 is a diaphragm that adjusts the amount of incident light from the subject, a lens that collects the light that passes through the diaphragm, photoelectrically converts the light collected by the lens, and outputs it as an image signal. It is composed of an image sensor having non-linear characteristics. The image input unit 102 performs predetermined processing on the image signal from the imaging unit 101 and outputs it. The predetermined processing is input interface processing for performing image signal processing in a block that inputs a signal from the image sensor and performs subsequent signal processing, for example, AD (Analog to Digital) conversion processing, Further, for example, it is a synchronization adjustment process for adjusting the processing timing of the image sensor and the signal processing unit. The image input unit 102 may include predetermined image signal processing such as gain control and filter processing for signal level adjustment. The exposure control unit 103 controls the diaphragm of the imaging unit 101 so that the amount of the image signal input from the image input unit 102 is kept at a predetermined level. The signal level detection unit 104 detects the signal level of the image signal from the image input unit 102. The white balance (WB) system control unit 105 adjusts the white balance (WB) so as to perform a white balance adjustment amount calculation process according to the detection result of the signal level detection unit 104 and the input / output characteristics of the signal from the image input unit 102. An instruction is given to the quantity calculation unit 106. Further, a command is given to the white balance (WB) processing unit 107 to perform white balance processing on the image signal from the image input unit 102 using the white balance adjustment amount obtained from the white balance adjustment amount calculation unit 106. To do. Thereby, an appropriate white balance can be obtained for the signal from the imaging unit 101 having nonlinear input / output characteristics. The image output unit 108 performs predetermined processing on the image signal from the white balance processing unit 107 and outputs it. The predetermined processing refers to image signal processing such as noise removal, gamma correction, contour enhancement, filter processing, zoom processing, camera shake correction, and image recognition on the image signal from the white balance processing unit 107, and TV and storage. This is output interface processing for conversion to the signal format of the output device. The output interface processing is, for example, conversion into NTSC or PAL video output, for example, conversion into an HDMI signal, for example, conversion into a predetermined signal for network transmission. In the first embodiment, the white balance processing unit and the image output unit that performs image signal processing are individually illustrated, but the white balance processing unit may be included in a part of the image output unit.

  With the above configuration, it is possible to realize white balance processing according to the input / output characteristics of the signal from the imaging unit, and therefore it is possible to obtain appropriate white balance even when using a nonlinear imaging device.

  FIG. 2 is a diagram illustrating an example of nonlinear input / output characteristics. As described above, the imaging unit 101 (FIG. 1) includes an imaging element whose input / output characteristics are nonlinear. However, in the example of FIG. 2, the low luminance (low incident light signal level) region has linear characteristics, and high The luminance region has logarithmic characteristics, and the higher the luminance, the higher the compression rate. Thereby, the dynamic range can be widened. In addition, both the low-brightness area and the high-brightness area are linear characteristics, but with different slopes, the compression ratio can be changed between the low-brightness area and the high-brightness area, or the logarithmic characteristics can be different between the low-brightness area and the high-brightness area. The compression ratio may be changed between the low luminance region and the high luminance region. Of course, one kind of logarithmic characteristic may be provided from the low luminance region to the high luminance region. Further, in addition to the logarithmic characteristic, a non-linear characteristic compressed by a predetermined function may be provided.

  Details of the signal level detector 104 (FIG. 1) will be described. The numerical value detected by the signal level detection unit 104 is, for example, the absolute value of the input pixel signal. Further, for example, the signal of the image region such as the histogram distribution, maximum value minimum value, average value of the signal in a predetermined image region (n × m pixel region) composed of the peripheral pixels around the input pixel signal. It is a numerical value serving as information indicating the luminance distribution state of. If the above information is detected, the amount of incident light when the pixel signal is obtained can be estimated. In order to estimate the amount of incident light, it is sufficient if there is absolute value information of the pixel signal, but the histogram distribution, maximum value, minimum value, and average value are calculated using peripheral pixels that are highly correlated with the input pixel signal. If detected, noise components to be superimposed at the time of imaging can be separated with high accuracy, and high-precision estimation is possible regardless of shooting conditions.

  Details of the white balance adjustment amount calculation unit 106 (FIG. 1) will be described. The white balance adjustment amount calculation unit 106 receives a signal from the image input unit 102 and calculates the white balance adjustment amount of the signal, for example, when the color of the light source is close to or from the image signal from the image input unit 102. The estimated color difference signals of the pixels are integrated to calculate an average value of the color difference signals. The color difference signal average value corresponds to a deviation from the origin of the UV coordinate system. Therefore, a white balance adjustment amount is calculated with the color difference signal average value as the origin. Further, for example, when the signal corrected by the input / output characteristic correspondence processing unit 106 maintains a nonlinear input / output characteristic, the white balance adjustment amount calculation method is switched between the low luminance region and the high luminance region to perform white balance adjustment. Calculate the amount. The UV coordinate system represents a color difference signal with U as the horizontal axis and V as the vertical axis as shown in FIG. 3, and the origin (0, 0) has no color.

  With the above configuration, automatic white balance processing can be performed so that an appropriate white balance can be obtained even when an image sensor having nonlinear characteristics is used.

(2) White Balance Processing FIG. 3 is a diagram for explaining a UV coordinate system for color difference signals. When the color difference signal is expressed with U as the horizontal axis and V as the vertical axis, the origin (0, 0) indicates a state of no color. The color of each pixel is R (red), Mg (magenta), B (blue), Cy (cyan), G (green), Ye (yellow) on the UV coordinate system with this origin as the center as shown in the figure. ) Etc.

  FIG. 4 is a diagram illustrating an example of a method for calculating the white balance adjustment amount. In this embodiment, it is estimated that the color temperature of the light source is a certain area on the UV coordinates, and this area is set as the light source color determination area 401.

  FIG. 5 is a flowchart for explaining the flow of processing for calculating the white balance adjustment amount. In step S501, the white balance adjustment amount calculation unit 107 determines whether or not a color difference signal exists in the light source color determination area 401 for each pixel of the image signal from the image input unit 102. In step S502, the color difference signals are integrated. If not, the process is terminated without integrating the color difference signals. After the processing of FIG. 5 is performed on all the pixels, the color difference signal average value is calculated by dividing the integrated value of the color difference signals by the total number of pixels. The average value of the color difference signals corresponds to a deviation from the origin of the UV coordinate system as described above. An appropriate white balance can be obtained by correcting the deviation to be the origin.

  FIG. 6 is a flowchart for explaining the flow of a technique corresponding to the processing at the time of calculating the white balance adjustment amount when the signal output from the imaging unit is nonlinear. In step S601, when a signal is input to the white balance system control unit 105 (FIG. 1), in step S602, the white balance system control unit 105 applies to each pixel based on information from the signal level detection unit 104. Then, it is determined whether or not it is a high luminance region. If the pixel belongs to the high luminance area, in step S603, the white balance system control unit 105 uses the light source color determination area for the high luminance area to the white balance adjustment amount calculation unit 106 to perform white balance. Command to perform adjustment amount calculation processing. If the pixel belongs to the low luminance area, in step S604, an instruction is given to perform white balance adjustment amount calculation processing using the light source color determination area for the low luminance area. The content of the white balance adjustment amount calculation process is as described with reference to FIG. In step S605, the white balance system control unit 105 instructs the white balance processing unit 107 to perform white balance processing using the white balance adjustment amount obtained from the white balance adjustment amount calculation processing unit. finish.

  Note that the range of the light source color determination area used in step S603 and step S604 stores information associating the input level of the signal output from the imaging unit with information indicating the range of the light source color determination area as a table. The method of using it is sufficient. In the white balance process in step S605, the color difference signal average value may be converted so as to coincide with the origin of the UV coordinate system, and the color difference signal average value is converted to the UV coordinate in order to improve the appearance of the image. Of course, the conversion may be made so as to coincide with an arbitrary point different from the origin of the system.

  By the above processing, the white balance adjustment amount calculation processing can be switched according to the input / output characteristics from the imaging unit, so that even when the output signal from the imaging unit is nonlinear, an appropriate white balance can be obtained. Can do.

  FIG. 7 is a diagram illustrating an example of the white balance adjustment amount calculation process described in FIG. The light source color determination area 701 for the low luminance area is used when calculating the white balance adjustment amount for the low luminance area in step S604 (FIG. 6). Specifically, this means that the light source color determination area 701 for the low luminance area is used in the determination in step S501 (FIG. 5). The light source color determination area 702 for the high luminance area is used when the white balance adjustment amount for the high luminance area is calculated in step S603. Specifically, this means that the light source color determination area 702 for the high luminance area is used in the determination in step S501. Here, in the present embodiment, since the high luminance area has a logarithmic characteristic with a higher compression ratio than the low luminance area, the high luminance area light source color determination area 702 has an area larger than that of the low luminance area light source color determination area. Make it smaller. This is because the signal is compressed because the high luminance region has a logarithmic characteristic, and the color difference signal is considered to be close to the origin as a whole compared to the low luminance region that has a linear characteristic. Note that the range of the light source color determination area in the high luminance region that is logarithmic characteristics is variably controlled using a table that associates the input level of the signal output from the imaging unit with the information indicating the range of the light source color determination area. It doesn't matter. Thereby, highly accurate white balance adjustment amount calculation can be realized. Alternatively, the logarithmic characteristics may be approximated by a linear expression, the compression rate may be regarded as constant, and the light source color determination area for the high luminance area may be fixed to one type. This facilitates the control of the light source color determination area for the high luminance area. In this embodiment, the low luminance region is assumed to have a linear characteristic and the high luminance region is assumed to have a logarithmic characteristic. However, the present invention is also applied to other combinations in which the signal compression rate of the low luminance region is different from that of the high luminance region. The method is applicable.

  By changing the range of the light source color determination area used when calculating the white balance adjustment amount by the above means, an appropriate white balance can be obtained for a signal having nonlinear input / output characteristics.

(1) System Configuration of Embodiment 2 An embodiment of the present invention will be described in detail. FIG. 8 is a diagram illustrating an example of a basic configuration according to the second embodiment of the present invention. The blocks 801 to 808 perform the same processing as the blocks 101 to 108 in FIG. 1, respectively, but the white balance (WB) system control unit 805, the white balance (WB) adjustment amount calculation unit 806, and the white balance ( WB) An input signal to the processing unit 807 is an image signal from the input / output characteristic correspondence processing unit 809 instead of the image input unit 802. Based on the information obtained from the signal level detection unit 804, the input / output characteristic correspondence processing unit 809 performs processing corresponding to the input / output characteristics on the image signal from the image input unit 802. This processing is, for example, performing correction processing of the image signal from the image input unit 802 according to the detection result of the signal level detection unit 804 and the input / output characteristics of the signal from the image input unit 802. As a result, low luminance side and high luminance side signals with different characteristics are suitable for white balance processing, for example, the output signal from the image input unit is a low luminance side signal with linear characteristics and a high luminance side signal with logarithmic characteristics. , The decompression process is performed on the compressed high luminance side signal of the logarithmic characteristic to convert it into a linear characteristic. Further, for example, the low luminance side signal having linear characteristics is compressed to approximate the compression rate of the high luminance side signal.

  By the above means, the white balance adjustment amount is calculated after the signal having nonlinear input / output characteristics is brought close to the input / output characteristics of the signal having normal linear characteristics by the input / output characteristics control, and the white balance processing is performed. Appropriate white balance can be obtained even when the signal output from the imaging unit is nonlinear. As described above, the input / output characteristic control unit 809 performs processing according to the input / output characteristics of the signal from the image input unit 102. However, the input level and the output level of the signal output from the image input unit 102 are associated with each other. The information may be stored as a table and used. When using a table, it is possible to accurately grasp the input / output characteristics. In addition, a method of holding a function expression representing input / output characteristics may be used. When a function formula is used, the input / output characteristics can be accurately grasped, and data to be held can be reduced as compared with the method of storing as a table. In addition, a method of holding information capable of estimating input / output characteristics such as compression rate information at the luminance levels of a plurality of incident lights may be used. When using information that can estimate the input / output characteristics, for example, it is possible to accurately grasp the input / output characteristics by approximating the data with the data thinned out and holding it as a table. Compared to this, the amount of data to be retained can be reduced. In the method of holding the function formula, the calculation amount increases as the function becomes a polynomial. However, if the thinned data is realized by approximate interpolation, the calculation amount can be relatively reduced. In addition to the method of expanding high-luminance signals, for example, compression processing of low-luminance areas so as to be equivalent to the high-luminance compression rate brings the characteristics close to the characteristics of an image sensor with normal linear characteristics. In addition, since it is possible to perform processing with a smaller number of bits than in the case of performing expansion processing on a high luminance signal, the circuit cost can be reduced.

  FIG. 9 is a flowchart illustrating the flow of white balance processing using correction processing in the input / output characteristic control unit 809 (FIG. 8). In step S901, a signal is input to the input / output characteristic correspondence processing unit 809. In step S902, the input / output characteristic correspondence processing unit 809 is compressed by performing nonlinear processing on the nonlinear signal from the image input unit 802. Instructs the decompression process to be performed on the high luminance signal. As a result, the input / output characteristics of the imaging unit having normal linear characteristics are brought close to. In step S903, the white balance system control unit 805 instructs the white balance adjustment amount calculation unit 806 to perform the processing in FIG. 5 to calculate the white balance adjustment amount. In step S904, the white balance system control unit 805 The unit 805 instructs the white balance processing unit 807 to perform white balance processing on the image signal obtained from the input / output characteristic correspondence processing unit 809 using the white balance adjustment amount obtained from step S903. finish. At this time, the white balance processing in step S904 may perform conversion so that the color difference signal average value coincides with the origin of the UV coordinate system. In order to improve the appearance of the video, the color difference signal average value is UV. Of course, the conversion may be made so as to coincide with an arbitrary point different from the origin of the coordinate system.

  FIG. 10 is a flowchart for explaining a flow of a technique for performing white balance processing corresponding to a non-linear signal in the input / output characteristic correspondence processing unit and the white balance adjustment amount calculation unit. The processing in FIG. 10 is performed, for example, by performing a decompression process on a signal in a high luminance region having a logarithmic characteristic and bringing the signal in the high luminance region closer to a linear characteristic in the same manner as in the low luminance region. It is effective when applied when the region has linear characteristics with different compression ratios. When an image signal is input to the input / output characteristic correspondence processing unit 809 (FIG. 8) in step S1001, the input / output characteristic correspondence processing unit 809 performs nonlinear processing on the nonlinear signal from the imaging unit 801 in step S1002. Thus, the decompression process is performed on the compressed high-intensity signal, so that the process approximates to the input / output characteristics of the imaging unit having a normal linear characteristic. Next, in step S1003, it is determined whether each pixel is a high luminance region. If the pixel belongs to the high luminance area, white balance adjustment amount calculation processing for the high luminance area is performed in step S1004. If the pixel belongs to the low luminance area, a white balance adjustment amount calculation process for the low luminance area is performed in step S1005. In step S1006, white balance processing is performed using the white balance adjustment amount obtained in each white balance adjustment amount calculation processing, and the processing ends. When calculating the white balance adjustment amount, as described with reference to FIG. 7, the area of the light source color determination area is adjusted according to the compression rate of the signal.

  If the output signal from the imaging unit is a non-linear signal having a linear characteristic and a logarithmic characteristic through the above processing, the input / output characteristic correspondence processing unit 809 converts the logarithmic characteristic region into a linear characteristic in step S1002, and step S1003. In step S1005, the white balance adjustment amount calculation unit 806 can perform the white balance adjustment amount calculation process by switching the light source color determination area for each of the non-linear low luminance region and high luminance region. As a result, it is only necessary to calculate the white balance adjustment amount with respect to the signal having the linear characteristic, so that the control for determining the range of the light source color area is facilitated.

(1) System Configuration of Embodiment 3 FIG. 11 is a diagram for explaining an example of a basic configuration in Embodiment 3 of the present invention. The imaging unit 1101 to the white balance (WB) system control unit 1105 perform the same processing as in FIG. A white balance (WB) adjustment amount calculation unit 1106 receives a command from the white balance system control unit 1105 and calculates a white balance adjustment amount for the image signal from the image input unit 1102. At this time, the white balance adjustment amount is calculated for each of the low luminance region and the high luminance region having different input / output characteristics. For example, the white balance adjustment amount of the low luminance region is set to the system a, and the white balance adjustment amount of the high luminance region is calculated. Output as system b. The white balance (WB) processing unit 1107 receives an instruction from the white balance system control unit 1105 and uses the white balance adjustment amounts obtained from the white balance adjustment amount calculation unit 1106 to reduce the image signal from the image input unit. White balance processing is performed for each of the luminance region and the high luminance region, and a signal of the system a in which the white balance processing is applied to the low luminance region and a signal of the system b in which the white balance processing is applied to the high luminance region are output. The signal synthesis unit 1109 synthesizes each image signal from the white balance processing unit and outputs it as one image signal. The image output unit 1108 performs the same processing as the image output unit 108 in FIG.

  With the above configuration, since the white balance processing can be individually performed on the signals of the image signal system a in the low luminance area and the signal of the image signal system b in the high luminance area, an appropriate white balance can be obtained in each of the systems ab. Obtainable. Thereby, as will be described later, a more appropriate white balance can be obtained in a scene where there are a plurality of light sources having different color temperatures. Note that the same effect can be obtained even if the system configuration is such that an input / output characteristic correspondence processing unit is added to FIG. 11 as shown in FIG.

  FIG. 12 is a flowchart showing the flow of white balance processing in the configuration of FIG. In step S1201, when a signal is input to the white balance system control unit 1105 (FIG. 11), in step S1202, the white balance system control unit 1105 applies to each pixel based on information from the signal level detection unit 1104. Then, it is determined whether or not it is a high luminance region. If the pixel belongs to the high brightness region, in step S1203, the white balance system control unit 1105 performs white balance adjustment amount calculation processing for the high brightness region on the white balance adjustment amount calculation unit 1106. Command. If the pixel belongs to the low luminance area, in step S1204, an instruction is given to perform white balance adjustment amount calculation processing for the low luminance area. In this embodiment, the white balance adjustment amount for the high luminance region and the white balance adjustment amount for the low luminance region are output separately. The content of the white balance adjustment amount calculation process is as described with reference to FIG. In step S1205, the white balance system control unit 1105 performs white balance processing on the white balance processing unit 1107 using the white balance adjustment amount (system b) for the high brightness area obtained from the white balance adjustment amount calculation processing unit. Order to do. In step S1206, the white balance system control unit 1105 performs white balance processing on the white balance processing unit 1107 using the white balance adjustment amount (system a) for the low luminance area obtained from the white balance adjustment amount calculation processing unit. Order to do. In step S1207, the image signals from the white balance processing unit 1107 are combined and the process ends. Note that the range of the light source color determination area used in step S1203 or step S1204 holds information associating the input level of the signal output from the imaging unit with information indicating the range of the light source color determination area as a table. The method of using it is sufficient. Note that the white balance processing in step S1205 and step S1206 may be performed such that the average color difference signal coincides with the origin of the UV coordinate system, and the average color difference signal is used to improve the appearance of the video. Of course, the conversion may be made so as to coincide with an arbitrary point different from the origin of the UV coordinate system.

  With the above processing, since the white balance processing can be switched according to the luminance, an appropriate white balance can be obtained even when the color temperatures of the light sources in the low luminance region and the high luminance region are different.

(2) Effects of Embodiment 3 FIG. 13 is a diagram for explaining an example of the effects of the present embodiment. FIG. 13 shows an example in which an indoor 1301 using a fluorescent lamp as a light source and an outdoor 1302 using sunlight as a light source exist within the angle of view. In the case of a scene as shown in FIG. 13, the dynamic range of an image captured in a dark room and a bright room is widened, so that it is easy to obtain a wide dynamic range effect by using a non-linear image sensor. However, when a general white balance process is applied in this scene, a constant white balance process is applied to the entire screen, so that it is not possible to obtain an optimal white balance for both indoors and outdoors. is there. On the other hand, in the case of the configuration of FIG. 11, different white balance processes can be applied to each of the low-brightness region using a fluorescent lamp as a light source and the high-brightness region using sunlight as a light source. Appropriate white balance can be obtained even in a scene.

  In addition, this invention is not limited to an above-described Example, Various modifications are included. For example, the above-described embodiments have been described in detail for easy understanding of the present invention, and are not necessarily limited to those having all the configurations described. In addition, a part of the configuration of a certain embodiment can be replaced with the configuration of another embodiment, and the configuration of another embodiment can be added to the configuration of a certain embodiment. Further, it is possible to add, delete, and replace other configurations for a part of the configuration of each embodiment.

  In addition, each of the above-described configurations may be configured such that some or all of them are configured by hardware, or are implemented by executing a program by a processor. Further, the control lines and information lines indicate what is considered necessary for the explanation, and not all the control lines and information lines on the product are necessarily shown. Actually, it may be considered that almost all the components are connected to each other.

DESCRIPTION OF SYMBOLS 101 ... Imaging part, 102 ... Image input part, 103 ... Exposure control part, 104 ... Signal level detection part, 105 ... White balance system control part, 106 ... White balance adjustment amount calculation part, 107 ... White balance processing part, 108 ... Image output unit 401 ... Light source color determination area 701 ... Light source color determination area for low brightness area, 702 ... Light source color determination area for high brightness area, 801 ... Imaging section, 802 ... Image input section, 803 ... Exposure control section, 804: Signal level detection unit, 805 ... White balance system control unit, 806 ... White balance adjustment amount calculation unit, 807 ... White balance processing unit, 808 ... Image output unit, 809 ... Input / output characteristic correspondence processing unit, 1101 ... Imaging unit 1102 ... Image input unit 1103 ... Exposure control unit 1104 ... Signal level detection unit 1105 ... White bar Nsu system control unit, 1106 ... white balance adjusting amount calculating unit, 1107 ... white balance processing unit, 1108 ... image output unit, 1109 ... signal combining unit, 1301 ... dark indoors, 1302 ... bright outdoors

Claims (9)

An image sensor having non-linear photoelectric conversion characteristics;
Image input means for inputting a signal from the image sensor;
Signal level detection means for detecting the signal level from the image input means;
White balance adjustment amount calculating means for calculating a white balance adjustment amount based on a signal from the image input means;
Automatic white balance processing means for performing automatic white balance processing based on a signal from the white balance adjustment amount calculating means;
A white balance system control means for controlling and controlling the white balance;
Comprising
The white balance system control means includes:
Controlling the white balance processing means according to the white balance adjustment amount calculated by the white balance adjustment amount calculating means;
An image signal processing apparatus. An image sensor having non-linear photoelectric conversion characteristics;
Image input means for inputting a signal from the image sensor;
Signal level detection means for detecting the signal level from the image input means;
Input / output characteristic corresponding processing means for correcting a signal from the image input means according to the input / output characteristics;
White balance adjustment amount calculating means for calculating a white balance adjustment amount based on a signal from the input / output characteristic correspondence processing means;
Automatic white balance processing means for performing automatic white balance processing based on a signal from the white balance adjustment amount calculating means;
A white balance system control means for controlling and controlling the white balance;
Comprising
The white balance system control means includes:
Controlling the white balance processing means according to the white balance adjustment amount calculated by the white balance adjustment amount calculating means;
An image signal processing apparatus. An image sensor having non-linear photoelectric conversion characteristics;
Image input means for inputting a signal from the image sensor;
Signal level detection means for detecting the signal level from the image input means;
White balance adjustment amount calculation means for calculating a plurality of white balance adjustment amounts according to the signal level detected by the signal level detection means based on the signal from the image input means;
Automatic white balance processing means for performing automatic white balance processing based on a plurality of signals from the white balance adjustment amount calculating means;
Signal combining means for combining a plurality of signals from the automatic white balance processing means;
A white balance system control means for controlling and controlling the white balance;
Comprising
The white balance system control means includes:
Controlling the white balance processing means according to a plurality of white balance adjustment amounts calculated by the white balance adjustment amount calculating means;
An image signal processing apparatus. The image processing apparatus according to claim 1 or 2,
The white balance adjustment amount calculation means determines a high luminance signal area and a low luminance signal area based on information from the signal level detection means, and switches a white balance adjustment amount calculation method for each area;
An image signal processing apparatus. In the image signal processing device according to any one of claims 1 to 4,
The signal level detecting means detects a signal for estimating the amount of incident light when photoelectric conversion is performed in the imaging device, and the pixel signal input from the image input means is the center, and its peripheral pixels Detecting any one of histogram distribution, maximum value minimum value, average value in the pixel area including
An image signal processing apparatus. The image processing apparatus according to claim 2.
The input / output characteristic correspondence control means holds information associating the input level and output level of the image sensor as information on the input / output characteristics of the image sensor, and is detected by the table information and the signal level detection means. The high luminance signal area is expanded based on the information obtained,
An image signal processing apparatus. The image signal processing apparatus according to claim 2, wherein
The input / output characteristic correspondence control means holds information associating the input level and output level of the image sensor as information on the input / output characteristics of the image sensor, and is detected by the table information and the signal level detection means. Based on the information obtained, the compression process of the low luminance signal is performed so as to be equivalent to the compression ratio of the high luminance signal,
An image signal processing apparatus. The image processing apparatus according to claim 2.
The input / output characteristic correspondence control means holds, as information on the input / output characteristics of the image sensor, as a function expression representing the input / output characteristics of the image sensor, and based on the function formula and information detected by the signal level detection means Performing a decompression process on the compressed high luminance signal;
An image signal processing apparatus. The image processing apparatus according to claim 2.
The input / output characteristic correspondence control means holds, as information on the input / output characteristics of the image sensor, as a function expression representing the input / output characteristics of the image sensor, and based on the function formula and information detected by the signal level detection means , Compressing the low-brightness signal so that it is equivalent to the compression rate of the high-brightness signal,
An image signal processing apparatus.

Images