JPH05207501A - Light source estimate device - Google Patents

Abstract

PURPOSE:To attain excellent reproduction of a picture by using a picture signal and a diaphragm signal. CONSTITUTION:The device is provided with an object brightness calculation circuit 14 calculating an object brightness and a light source estimate section 15 estimating a light source based on a chromaticity signal and an object brightness signal. When a light source is estimated with the device, the chromaticity and the brightness of an object are processed equally to set the brightness of the object being a criterion of outdoor or indoor for each color thereby improving the precision of discrimination of outdoor or indoor and then realizing the light source estimate device estimating a light source with high precision.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a light source estimating device for reproducing a correct color tone in a video camera or the like.

[0002]

2. Description of the Related Art In recent years, with the spread of video cameras, means for following photographing conditions have become more and more important in order to obtain beautiful colors. Among them, it is very important for color reproduction to accurately estimate the light source at the time of shooting.

First, as a conventional means for estimating a photographing light source, there are known a means for performing estimation using a plurality of photosensitive elements having different spectral characteristics and a means for performing estimation through image processing. In general, the light source estimation device is
From the viewpoints of design, cost, etc., the method has been changed to a method using a light-sensitive element, which has a limitation, and a method for estimating a light source by processing an image signal.

The rationale for the method of estimating the light source based on image processing is that the color of the subject in the screen is achromatic when mixed and averaged, that is, the gray world assumption (hereinafter abbreviated as GWA). is there. As a method for performing color adjustment based on this assumption, Japanese Patent Laid-Open No. 56-3
The one described in Japanese Patent No. 6291 is known. Further, a method for determining the establishment of GWA and increasing the estimation accuracy has been filed by the present applicant (Japanese Patent Application No. 3-14871).
5).

[0005]

However, in the above configuration, the light source estimation may be erroneous for a monotonous image because only the image signal that does not reflect the brightness of the subject is used. The present invention solves the above problems, and an object of the present invention is to provide a light source estimation apparatus that enables excellent image reproduction by using an image signal and an aperture signal.

[0006]

In order to achieve this object, a light source estimation apparatus of the present invention comprises a sampling circuit for obtaining image data suitable for light source estimation from color-separated image signals, and a diaphragm of a video camera. An aperture value input terminal for receiving the value signal F, and a sampling luminance signal Y and a sampling chromaticity signal ((BY) /
Y, (R−Y) / Y), a signal conversion circuit that calculates the sampled subject brightness HY from the aperture value signal F and the sampling brightness signal Y, and a sampling chromaticity signal ((B− Y) / Y, (RY) / Y)
And a light source estimation unit that estimates the light source from the sampled subject luminance signal HY.

[0007]

With this configuration, when the light source is estimated, the chromaticity and the luminance of the subject are treated equally, so that it is not necessary to make the separation plane for separating the outdoor and the indoor parallel to the chromaticity plane. The separating surface can be configured by any curved surface in the configured three-dimensional space. This makes it possible to set the brightness of the subject, which is the judgment boundary between indoors and outdoors, for each color. For example, it is possible to give a small threshold value to green with low brightness and a large threshold value to yellow with high brightness. This makes it possible to improve the accuracy of outdoor / indoor determination, and thus realize a light source estimation device with high light source estimation accuracy.

[0008]

(Embodiment 1) An embodiment of a light source estimation apparatus of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing the configuration of a light source estimation device in one embodiment of the present invention.

In FIG. 1, 11 is a sampling circuit for obtaining image data suitable for light source estimation from color-separated image signals, and 12 is a diaphragm value signal F of a video camera.
A reference numeral 13 denotes an aperture value input terminal for receiving a signal, and 13 is a signal conversion circuit for converting the sampled image data into a sampling luminance signal Y and a sampling chromaticity signal ((BY) / Y, (RY) / Y). Yes, 14 is a subject luminance calculation circuit that calculates the sampled subject luminance HY from the aperture value signal F and the sampling luminance signal Y, and 15 is a sampling chromaticity signal ((BY) / Y, (RY) / Y) and a sampled subject brightness signal HY, a light source estimation unit that estimates a light source. Reference numeral 151 is a signal evaluation circuit that evaluates whether the sampling chromaticity signal and the sampled subject brightness signal are good or bad based on the sampling brightness signal. Yes,
Reference numeral 152 denotes an average value calculation circuit that calculates the average value of each of the sampling chromaticity signal and the sampled subject luminance signal, which are good data in the signal evaluation circuit 151,
Reference numeral 153 has a determination table internally having an outdoor / indoor separation plane formed by a plane or a curved surface that is not parallel to the chromaticity plane in a three-dimensional space formed by chromaticity and luminance, and the average from the average value calculation circuit 152. It is a light source determination circuit that determines a light source by drawing a table according to points specified by the sampling chromaticity and the average sampling subject luminance signal.

The operation of the light source estimation device configured as described above will be described. First, the color-separated image signal sampling circuit 11 samples the image signal according to a preset sampling rule suitable for performing light source estimation. As an example of the image data created by the sampling circuit 11, it is possible to divide the screen into several blocks and consider the average value of the luminance signal and the color difference signal for each block. Signal conversion circuit 1
3 receives the image data from the sampling circuit 11 and converts it into a sampling luminance signal Y and a sampling chromaticity signal ((BY) / Y, (RY) / Y). The subject luminance calculation circuit 14 calculates the sampled subject luminance HY from the sampling luminance signal Y from the signal conversion circuit 13 and the aperture signal F input from the aperture value input terminal 12.

Usually, this sampling subject brightness HY
Is given by HY = F ² * Y when the f value is used as the aperture signal. The light source estimation unit 15 estimates the light source using the sampling chromaticity signal and the sampling subject luminance signal. First, the signal evaluation circuit 151, based on the sampling chromaticity signal and the sampling subject luminance signal, mainly based on the sampling luminance signal, for example, when the sampling luminance signal is higher than the maximum luminance setting value and when the sampling luminance signal is lower than the minimum luminance setting value. In this case, the signal is evaluated as good or bad by setting it as a bad signal.

Next, the average value calculation circuit 152 calculates each average value using the sampling chromaticity signal and the sampling subject luminance signal which are good data in the signal evaluation circuit 151. The light source determination circuit 153, based on the points designated by the average sampling chromaticity signal and the average sampling subject luminance signal from the average value calculating circuit 152, within the three-dimensional space constituted by the chromaticity and the luminance held inside. Then, a light source is determined by drawing a determination table having an outdoor / indoor separation plane formed by a plane or a curved surface that is not parallel to the chromaticity plane.

The difference between the light source estimation apparatus according to this embodiment and the conventional light source estimation determination method using only the sampling chromaticity signal will be described with reference to FIG. In the conventional light source estimation device, since the light source is estimated only by the color difference, withered grass and white fluorescent light under the outdoor light in winter, green and daylight fluorescent light under the outdoor light, etc. are on the chromaticity plane. Since it was expressed at almost the same point, even though it was outdoors, the light source was mistakenly estimated to be indoors. As a method of improving this, it has been attempted to supplementarily add a process of determining that the aperture is outdoors when the aperture is a certain value or more.

However, in this method, the separation plane that separates the space determined to be outdoors by the diaphragm is limited to the threshold plane parallel to the chromaticity plane, as shown in FIG.
In such a method, if a threshold value is set on the basis of yellow, which has many relatively bright objects, outdoor judgment cannot be performed for green, which has low brightness. On the contrary, if the threshold value is determined based on green, indoors will be judged to be outdoor for yellow. For this reason, from the judgment that the information for estimating the light source is more in color than in luminance, it is usually said that the threshold value is set high and the aperture cannot be fully utilized. According to the configuration of the present invention,
As shown in FIG. 2B, it is not necessary to make the separation surface for separating the indoors and outdoors parallel to the chromaticity plane, and the separation surface can be formed by any curved surface in the three-dimensional space composed of chromaticity and luminance. Can be configured.

As described above, according to the present embodiment, it is possible to set the brightness of the subject, which is the judgment boundary between indoor and outdoor, for each color. Can give a small threshold value, and can give a large threshold value for yellow and the like, which has many high-luminance subjects, so that the determination accuracy inside and outside can be improved, and a light source estimation device with high estimation accuracy of the light source can be realized.

Note that the signal used as the aperture value signal need not be the f value if it is set so as to be consistent with the calculation rule in the subject brightness calculation circuit 14. Although not described here intentionally, a timing generator is provided as necessary to synchronize the input timing of the aperture signal with the timing of the average luminance calculation so that the aperture and the image signal correspond accurately. It can also be configured to time the circuit. In addition, the present invention can be used not only in video cameras, but also in general image pickup devices such as electronic still cameras with photoelectric conversion. The same applies to the following examples.

(Second Embodiment) A second embodiment of the present invention will be described below with reference to the drawings. In the second embodiment of the present invention, the light source determination circuit 153 of the first embodiment is constructed by a neural network simulation means.

As for the neural network simulating means, learning vector quantization (hereinafter abbreviated as LVQ), a multilayer perceptron using an error backpropagation method as a learning algorithm, an associatron, and the like, average sampling chromaticity, average subject brightness, and light source If you can learn the relationship between them, you can use what you think is appropriate. For details of neural network simulation means such as LVQ, multi-layer perceptron, and associatron, refer to Reference 1 (PDP model, DE Lamelhardt et al., 2 persons, translated by Shunichi Amari, 1989), Reference 2.
(Basic theory of neural networks, Kaoru Nakano and 7 others,
(1990), so I will not go into detail here.

FIG. 3 is a block diagram showing the configuration of the light source determination circuit 153 constructed by using the neural network simulation means of the second embodiment of the present invention. In FIG. 3, reference numeral 31 is a judgment processing means constituted by a neural network simulation means for receiving an average sampling chromaticity signal and an average sampling subject luminance signal and outputting a light source estimation value signal, and 32 is from the judgment processing means 31. The light source fixing means obtains the estimated light source signal from the light source estimated value signal.

The operation of the light source determination circuit configured as described above when the multilayer perceptron is used as the neural network simulation means will be described. First, the judgment processing means 31 constituted by the neural network simulation means receives the average sampling chromaticity signal and the average sampling subject luminance signal, and judges whether each light source such as an outdoor fine weather, a fluorescent lamp, and a halogen is 1, or not, A light source estimated value signal expressed by 0 (it is configured to output a value of about 0.9, 0.1 when a neural network simulating means such as a multilayer perceptron is used) is output. Is configured.

That is, the neural network simulation unit constituting the judgment processing unit 31 uses the average sampling chromaticity signal and the average sampling subject luminance signal input as the input signals, and the images in which the signals are generated are taken as the teacher signals. A decision processing unit using a neural network simulating means by giving a light source at this time and performing learning based on the error back propagation method so that the error between the estimation result of the decision processing means 31 and the teacher signal becomes small. The person 31 empirically learns how to estimate the light source based on the color and brightness of the subject like a human, and makes an accurate judgment.

However, when an analog function such as a sigmoid function is used as the output function of the neural network simulation means such as the multi-layer perceptron, the light source estimation value signal for each light source is as follows: Is given as a fluorescent lamp = 0.7 and a light source is halogen = 0.4, so that it is not always an appropriate expression as the estimated light source signal as described in the above embodiment. Therefore, the light source fixing means 32 shapes the light source estimated value signal from the determination processing means 31. Examples of output shaping include (1) finding the output with the maximum value.

In the above example, the light source is a fluorescent lamp = 0.7 (2) The signal accuracy is judged by whether the value is equal to or more than a threshold value. In the above example, it is determined whether 0.7> (threshold value). judge.

(3) If the threshold value is exceeded, the light source is output as an estimated light source signal. In the above example, when (threshold value) = 0.5, "the light source is a fluorescent lamp" is output.

(4) In other cases, "undecidable" is output. You can think of the process. In addition to this, depending on whether or not the difference between the maximum value and the second largest value is larger than a certain value, it may be replaced with the processing for determining the signal accuracy, or this may be added to the above determination. ..

When the judgment processing section 31 is constituted by the neural network simulating means, it is usually sufficient to use the learned neural network simulating means, so that the learning function may be omitted when the device is incorporated. ..

Further, in the present embodiment, the case where the multilayer perceptron is adopted as the neural network simulating means has been described. However, as described at the beginning, this shows the relationship among the average sampling chromaticity, the average subject brightness and the light source. Any neural network simulating means may be used as long as it can be learned.

[0028]

As described above, the present invention provides a light source estimation device,
An indoor / outdoor structure configured by allowing a color-separated image signal and an aperture value signal to be input, and having a plane or a curved surface that is not parallel to the chromaticity plane in a three-dimensional space composed of the aperture value signal, chromaticity, and luminance A determination table having a separation surface is provided inside, and the light source can be determined by drawing the table according to the points designated by the average sampling chromaticity and the average sampling subject luminance signal from the average value calculation circuit.

With this configuration, when the light source is estimated, the chromaticity and the luminance of the subject are treated equally, so that it is not necessary to take the separation plane for separating the indoors and outdoors as parallel to the chromaticity plane.
The separation surface can be formed by an arbitrary curved surface in a three-dimensional space composed of chromaticity and luminance. This makes it possible to set the brightness of the subject, which is the judgment boundary between indoors and outdoors, for each color. For example, it is possible to give a small threshold value to green with low brightness and a large threshold value to yellow with high brightness. This makes it possible to improve the accuracy of outdoor / indoor determination, and thus realize a light source estimation device with high light source estimation accuracy.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a light source estimation device according to an embodiment of the present invention.

FIG. 2A is an image diagram of an indoor / outdoor separation plane set in a chromaticity / luminance space for explaining the principle of operation of a conventional light source estimation apparatus. FIG. 2B is a principle of operation of the light source estimation apparatus of the present invention. Image of indoor / outdoor separated curved surface formed in chromaticity / luminance space for explaining

FIG. 3 is a block diagram showing the configuration of a light source determination circuit configured by using a neural network simulation means in a second embodiment of the present invention.

[Description of Reference Signs] 11 Sampling circuit 12 Aperture value input terminal 13 Signal conversion circuit 14 Subject brightness calculation circuit 15 Light source estimation unit 151 Signal evaluation circuit 152 Average value calculation circuit 153 Light source determination circuit 31 Determination processing means 32 Light source fixing means

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Toshikazu Fujioka 3-10-1 Higashisanda, Tama-ku, Kawasaki-shi, Kanagawa Matsushita Giken Co., Ltd.

Claims (4)

[Claims] 1. A sampling circuit for obtaining image data suitable for light source estimation from a color-separated image signal, an aperture value input terminal for receiving an aperture value signal of a video camera, and at least the sampled image data. A signal conversion circuit that calculates a sampling luminance signal, a subject luminance calculation circuit that calculates a sampling subject luminance from an aperture value signal and a sampling luminance signal, and a light source estimation unit that estimates a light source from image data and a sampling subject luminance signal. A light source estimation device provided. 2. The signal conversion circuit converts the sampled image data into a sampling chromaticity signal and a sampling luminance signal, and the light source estimation unit estimates the light source from the sampling chromaticity signal and the sampling subject luminance signal. Light source estimation device. 3. The light source estimation unit evaluates whether the signal is good or bad, mainly based on the sampling luminance signal and the sampling chromaticity signal and the sampled subject luminance signal, and the sampling color determined as good data by the signal evaluation circuit. An average value calculation circuit that calculates each average value using the chromaticity signal and the sampled subject luminance signal, and a plane or curved surface that is not parallel to the chromaticity plane in a three-dimensional space composed of chromaticity and luminance. It has a judgment table having an outdoor / indoor separation plane inside, and consists of a judgment circuit for judging the light source by drawing the table according to the points specified by the average sampling chromaticity and the average sampling subject luminance signal from the average value calculation circuit. The light source estimation device according to claim 2. 4. The light source estimation device according to claim 3, wherein the determination circuit comprises a determination processing means constituted by a neural network simulation means and a light source fixing means for shaping a light source estimation value signal of the determination processing means.