JP4061638B2 - Image processing apparatus and method, recording medium, and program - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image processing apparatus and method, a recording medium, and a program, and more particularly to an image processing apparatus and method, a recording medium, and a program that are suitable for application to an apparatus that handles image data including luminance levels.
[0002]
[Prior art]
Cameras that handle color images, both digital and analog, such as video cameras and still cameras, are widespread, but the colors captured by these cameras differ from those seen by the human eye. There is a case. This is because, for example, there is a light source with a strong red color or a light source with a strong green color due to the spectral distribution of the light source, and the color balance of the reflected light from the subject that has received light from the light source is lost. However, in a digital camera or the like using a light receiving element, the cause of sensing the color of the light source is an example. As a result, there are cases where the image is reddish or greenish as a whole.
[0003]
To prevent this, the digital camera has a function to adjust the color balance so that the color of the subject image captured by the light receiving element is adjusted to the same color as the subject seen by the human eye. ing. Such color balance adjustment is generally called white balance adjustment in the sense that the color under a white light source with appropriate color balance is reproduced. This is performed using the intensity ratio of the three primary colors of R (red), G (green), and B (blue) under a light source.
[0004]
[Problems to be solved by the invention]
When the luminance signal is generated using the R signal, the G signal, and the B signal whose white balance has been adjusted as described above, the luminance signal is calculated based on the following expression, for example.
Y = Cr x (R x Rw) + Cg x (G x Gw) + Cb x (B x Bw)
In the above equation, Y is the value of the luminance signal, Rw, Gw, and Bw are values of the white balance amplifier gain, and Cr, Cg, and Cb are coefficients of predetermined values, respectively.
[0005]
From the above formula, when Rw, Gw, Bw (value of white balance amplifier gain of each signal) is changed, even if the subject is exactly the same, in other words, the value of each signal of R, G, B It can be seen that the value of the luminance level Y may change even if there is no change.
[0006]
By the way, processing such as changing the white balance at a high speed before recording is generally performed before recording in order to avoid recording while changing the white balance when taking a still image. Is called. In addition, when a light source such as a strobe is supported, it is necessary to record while changing the white balance according to the color temperature of the light source during light emission.
[0007]
In such a case, that is, when the white balance is changed, as described with reference to the above formula, the luminance level confirmed by the user with the viewfinder before recording and the luminance of the recorded image, as described with reference to the above formula There is a problem that a difference occurs in the level, and an image unintended by the user is recorded.
[0008]
The present invention has been made in view of such a situation, and an object thereof is to acquire an image intended by a user by correcting a luminance signal.
[0009]
[Means for Solving the Problems]
An image processing apparatus according to the present invention includes an imaging unit that captures an image of a subject, a first storage unit that stores respective data of an R signal, a G signal, and a B signal of an image captured by the imaging unit; Data stored in the first storage means, first calculation means for calculating the first luminance level based on the white balance gain, and data of the first luminance level calculated by the first calculation means. When the second storage means to store and the recording of the image picked up by the image pickup means are instructed, the data stored by the first storage means is read, the read data and the white balance at that time Second calculation means for calculating the second luminance level based on the gain, first luminance level stored by the second storage means, and second luminance calculated by the second calculation means With bell, so that the luminance level of the image to be recorded is the same as the first luminance level, setting the gain of AGC, the charge accumulation time due to electronic shutter adjustment, or, the light amount adjustment performed by the iris And control means for controlling any one of them.
[0010]
The image processing method according to the present invention includes a first image capturing step for capturing an image of a subject, and first storage for controlling storage of respective data of the R signal, the G signal, and the B signal of the image captured by the processing of the image capturing step. A storage control step, a first calculation step for calculating a first luminance level based on the data whose storage is controlled in the processing of the first storage control step, a white balance gain, and a process of the first calculation step The second storage control step for controlling the storage of the data of the first luminance level calculated in step 1, and the processing of the first storage control step when recording of the image captured in the processing of the imaging step is instructed The second calculation step of reading out the data whose storage is controlled in step S2 and calculating the second luminance level based on the read data and the white balance gain at that time , By using the first luminance level storage in the processing of the second storage control step it is controlled, the second luminance level calculated in the processing of the second calculation step, the luminance level of the image to be recorded Control step for controlling any one of the setting of the gain of AGC, the adjustment of the charge accumulation time by the electronic shutter, or the adjustment of the light amount performed by the iris so as to be equal to the first luminance level. It is characterized by that.
[0011]
The recording medium program of the present invention controls the storage of each of the imaging step for capturing an image of a subject and the R signal, G signal, and B signal of the image captured in the processing of the imaging step. Storage control step, data for which storage is controlled in the processing of the first storage control step, a first calculation step for calculating a first luminance level based on the white balance gain, and a first calculation step A second storage control step for controlling storage of data of the first luminance level calculated in the processing, and when recording of an image captured in the processing in the imaging step is instructed, the first storage control step Second calculation for reading data whose storage is controlled by processing and calculating a second luminance level based on the read data and the white balance gain at that time Using a step, a first luminance level storage in the processing of the second storage control step is controlled, the second luminance level calculated in the processing of the second calculation step, the image to be recorded A control step for controlling any one of the setting of the gain of AGC, the adjustment of the charge accumulation time by the electronic shutter, or the light amount adjustment performed by the iris so that the luminance level becomes the same as the first luminance level ; It is characterized by including.
[0012]
The program according to the present invention is a first storage control that controls storage of respective data of an imaging step for capturing an image of a subject and R signal, G signal, and B signal of the image captured in the processing of the imaging step. A first calculation step for calculating the first luminance level based on the step, the data whose storage is controlled in the process of the first storage control step, the white balance gain, and the calculation in the process of the first calculation step. A second storage control step for controlling the storage of the first brightness level data, and when the recording of the image captured in the processing of the imaging step is instructed, the processing is stored in the processing of the first storage control step. A second calculation step of reading out the controlled data and calculating a second luminance level based on the read data and the white balance gain at that time By using the first luminance level storage in the processing of the second storage control step is controlled, the second luminance level calculated in the processing of the second calculation step, the luminance level of the image to be recorded A control step for controlling one of the AGC gain setting, the charge accumulation time adjustment by the electronic shutter, or the light amount adjustment performed by the iris so as to be the same as the first luminance level. It is made to perform.
[0013]
In the image processing apparatus, method, and program according to the present invention, the R, G, B signal data, the R, G, B signal data, and the white balance gain among the image data of the imaged subject are calculated. The brightness level to be stored is stored. When an image recording is instructed, a new luminance level is calculated using the stored R, G, B signal data and the white balance gain at that time. The stored brightness level and the new brightness level are used, and the AGC gain setting and the charge accumulation time by the electronic shutter are set so that the brightness level of the recorded image is the same as the stored brightness level . The light amount adjustment performed by the adjustment or the iris is controlled.
[0014]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a diagram showing the configuration of an embodiment of an image processing apparatus to which the present invention is applied. An image processing apparatus 10 illustrated in FIG. 1 is an apparatus that is provided in a digital video camera or a still camera and corrects a color of a captured image (video).
[0015]
An image (light) of a subject is received by a CCD (Charge Coupled Devices) 13 through a lens 11 and an iris 12 of the image processing apparatus 10. The lens 11 condenses the input light, the light amount of the collected light is adjusted by the iris 12, and an image is formed by the CCD 13.
[0016]
The CCD 13 converts the intensity of the received light into an electric signal. At the time of this conversion, the charge accumulation time in the CCD 13 can be controlled by the electronic shutter function. Each pixel constituting the CCD 13 is provided with a color filter. Due to the effect of the color filter, received light is converted into an R (red) signal, a G (green) signal, and a B (blue) signal. And are respectively supplied to an S / H (Sampling / Hold) circuit 14.
[0017]
The S / H circuit 14 samples an input electric signal based on a predetermined sampling frequency, and temporarily holds the sampled value. The held value (signal) is supplied to an AGC (Automatic Gain Control) circuit 15. The AGC circuit 15 converts the input signal to an appropriate signal level for subsequent signal processing, and supplies the signal to an A / D (Analog / Digital) conversion circuit 16. The A / D conversion circuit 16 converts the input RGB color analog signal into a digital signal and outputs the digital signal to a WB (white balance) amplifier 17 and a controller 18.
[0018]
The controller 18 controls each unit of the image processing apparatus 10. In particular, the following explanation will be given here assuming that control regarding white balance and luminance level Y is performed. As will be described in detail later, the controller 18 temporarily stores data for controlling the luminance level Y in the memory 20 and reads it as necessary.
[0019]
The WB amplifier 17 adjusts the white balance by multiplying each input digital signal of R, G, B by a gain (Gr, Gg, Gb) based on an instruction from the controller 18, and a color difference matrix This is supplied to the circuit 19.
[0020]
Here, the white balance will be briefly described. In the white balance, the WB gain is set so that the R signal, the G signal, and the B signal when a white subject is imaged are R = G = B. For example, when papers that humans recognize as white are compared to images taken under clear sky and images taken under cloudy skies without white balance processing, the white paper in those images is , They are not recognized as the same whiteness. That is, white is not necessarily captured as a color that is necessarily recognized as white, depending on shooting conditions and the like.
[0021]
In view of this, white balance adjustment is basically performed on the signal of the imaged image so that white is recognized as white, regardless of the situation of the imaged image.
[0022]
The color difference matrix circuit 19 generates a color difference signal using the R signal, the G signal, and the B signal whose white balance has been adjusted. The generated color difference signals are an RY signal and a BY signal. The color difference matrix circuit 19 also generates and outputs a luminance level Y.
[0023]
The output from the color difference matrix circuit 19 is supplied to the display / recording control unit 21. The display / recording control unit 21 controls display of the display unit 22 and recording of image data on the recording medium 23. The display unit 22 is composed of, for example, an LCD (Liquid Crystal Display) or the like, and displays an image of a captured subject. The user can refer to the image displayed on the display unit 22.
[0024]
The recording medium 23 may be detachable from the image processing apparatus 10 or may be built therein. An example of the recording medium 23 is a Memory Stick (trademark).
[0025]
The operation unit 24 is provided with a shutter button or the like that is operated when the user wants to record the image displayed on the display unit 22 on the recording medium 23, and an instruction from the user is given by the image processing apparatus 10. It is provided so that it may be supplied to.
[0026]
The luminance level Y generated by the color difference matrix circuit 19 in the image processing apparatus 10 described above is calculated by the following equation (1).
Y = 0.3 (R x Gr) + 0.6 (G x Gg) + 0.1 (B x Gb) (1)
In Expression (1), R, G, and B represent R, G, and B signal values, respectively, and Gr, Gg, and Gb represent white balance amplifier gains, respectively. Therefore, (R × Gr), (G × Gg), and (B × Gb) are the values of the R signal, the G signal, and the B signal, respectively, with the white balance adjusted, and are output from the WB amplifier 17. Value.
[0027]
In a system (in this case, the image processing apparatus 10) that generates the luminance level Y based on the expression (1), the user confirms the image of the subject being captured by the display unit 22 that displays the image. After that, when the recording of the image is instructed and the recording is executed, the white balance changes between the confirmation and the recording, and the brightness level of the confirmed image and the brightness level of the recorded image change. There is something that will end up.
[0028]
In this way, if the brightness level changes, an image that the user did not intend is provided to the user, which may cause inconveniences such as giving the user unpleasant feeling. was there. In order to improve such a state, in the present embodiment, the following processing is performed on the luminance level Y.
[0029]
The image processing apparatus 10 holds values of R, G, and B signals when a user confirms an image (before recording) in a memory or the like, and the held values and a point at which recording is actually performed. Based on the equation (1), the brightness level Y ′ at the time of confirmation is calculated using the white balance amplifier gain in, and the calculated brightness level Y ′ is similarly held in the memory or the like.
[0030]
Immediately before the image is recorded, the white balance amplifier gain at the time of recording the image is calculated, and the values of the R, G, B signals held at the time of confirmation and the white balance amplifier gain at the time of recording are used for recording. Luminance level Y is obtained. Then, the ratio between the luminance level Y ′ and the luminance level Y is obtained, and the AGC circuit 15 is set during recording so that Y = Y ′.
[0031]
Here, the description will be made on the assumption that the AGC circuit 15 is set, but the luminance level may be corrected by controlling an electronic shutter provided in the iris 12 or the CCD 13.
[0032]
Such processing will be further described with reference to the flowchart of FIG. The processing based on the flowchart of FIG. 2 is mainly executed by the controller 18. In the process based on the flowchart shown in FIG. 2, the image processing apparatus 10 is turned on, imaging of the subject is started, and the captured image is displayed on the display unit 22 (displayable state). Is done when.
[0033]
In step S11, a white balance (WB) gain is calculated. In step S12, it is determined whether or not an instruction to record the captured image is given. When the user wants to record a captured image on the recording medium 23, he / she operates a button for instructing recording, such as a shutter button provided on the operation unit 24. The controller 18 determines whether or not the recording is instructed by determining whether or not the operation unit 24 is operated.
[0034]
If it is determined in step S12 that recording is not instructed, the process proceeds to step S13, and the data of the R signal, G signal, and B signal of the image captured at that time is stored in the memory 20, respectively. . The controller 18 stores the signal output from the A / D conversion circuit 16 in the memory 20.
[0035]
In step S14, the brightness level Y ′ is calculated using the WB gain set at that time. This calculation is performed by substituting the value at that time into Expression (1). The calculated luminance level Y ′ is stored in the memory 20 in step S15. As described above, the memory 20 stores the data of the R signal, the G signal, the B signal, and the luminance level Y ′ before the image is recorded.
[0036]
These stored data are updated at predetermined intervals. That is, the data stored in the memory 20 is also updated by repeating the processes of steps S11 to S15 at predetermined intervals.
[0037]
On the other hand, if it is determined in step S12 that recording of image data of a captured image has been instructed, the process proceeds to step S16. In step S16, R signal, G signal, and B signal data are read from the memory 20. This read data is data acquired from an image before recording is instructed.
[0038]
In step S17, the luminance level Y is calculated by substituting the read data and the value of the WB gain at that time into equation (1). The luminance level Y is the luminance level of the image for which recording is instructed. In step S18, Y ′ / Y is calculated using the luminance level Y calculated in step S17 and the luminance level Y ′ stored in the memory 20 in step S15.
[0039]
In this way, by calculating the ratio between the luminance level Y ′ before recording and the luminance level Y at the time of recording, how much the luminance level Y at the time of recording varies with respect to the luminance level Y ′ at the start of recording. Can ask.
[0040]
The value of Y ′ / Y calculated in this way is used, and the process of step S19 is executed. In step S19, the gain value of the AGC circuit 15 is set to the value calculated in step S18 (Y '/ Y times) . Then, the gain of the AGC circuit 15 is controlled based on the set gain. Thus, the gain level of the AGC circuit 15 is set and controlled, so that the luminance level at the time of recording becomes the same as the luminance level before the start of recording.
[0041]
Since the brightness level is the same before and after the recording operation, the user refers to the image displayed on the display unit 22, and the user can select between the image desired to be captured and the actually recorded image. It is possible to prevent the occurrence of such a difference in brightness that the user feels uncomfortable.
[0042]
For example, when recording an image after suddenly shaking the camera in a state where the subject's light source color temperature is different (from the sun to the shade, or vice versa), avoid recording in a state where the color temperature is shifted. Therefore, sometimes the white balance response speed is increased. In such a situation, in the system for creating the brightness level Y from R, G, B after the white balance amplifier using the expression (1), the brightness level of the image confirmed by the user with the viewfinder and the recorded image It is conceivable that there will be a difference in the brightness level.
[0043]
However, if the gain of the AGC circuit 15 using the luminance level is set as in the present embodiment, the difference between the luminance level of the image confirmed by the user with the viewfinder and the luminance level of the recorded image is corrected. Since the recorded image is recorded at a luminance level equivalent to that of the confirmation image, it is possible to prevent the user from feeling uncomfortable due to a change in the color tone of the recorded image.
[0044]
Also, when shooting in response to momentary light source changes, such as when recording still images when using a digital camera, etc., the camera or strobe lighting device performs dimming processing so that the recorded image has the optimal brightness level. However, when the white balance changes in accordance with the color temperature of the strobe light, the brightness level after dimming does not become the intended one.
[0045]
However, in the present embodiment, as described above, the luminance level change due to the white balance fluctuation is corrected, so that even in such a case, it is possible to perform image recording at an appropriate luminance level.
[0046]
The image data of the image whose luminance level has been corrected in this way is recorded on the recording medium 23 in step S20. Such processing is repeatedly performed while the image processing apparatus 10 captures an image of the subject.
[0047]
In the above-described embodiment, it has been described that the luminance level is corrected by setting and controlling the gain of the AGC circuit 15. However, the light amount adjustment performed by the iris 12 is controlled or provided in the CCD 13. The luminance level may be corrected by controlling the adjustment of the charge accumulation time by the electronic shutter.
[0048]
In other words, in the above-described series of processing, each processing of the flowchart shown in FIG. 2 may be executed by a circuit configured in hardware, or configured and executed in software. You may do it. When a series of processing is executed by software, various functions can be executed by installing a computer in which the programs that make up the software are installed in dedicated hardware, or by installing various programs. For example, it is installed from a recording medium in a general-purpose personal computer or the like.
[0049]
Before describing the recording medium, a general-purpose personal computer will be briefly described. FIG. 3 is a diagram illustrating an internal configuration example of a general-purpose personal computer. A CPU (Central Processing Unit) 111 of the personal computer executes various processes according to a program stored in a ROM (Read Only Memory) 112. A RAM (Random Access Memory) 113 appropriately stores data and programs necessary for the CPU 111 to execute various processes. The input / output interface 115 is connected to an input unit 116 including a keyboard and a mouse, and outputs a signal input to the input unit 116 to the CPU 111. The input / output interface 115 is also connected to an output unit 117 including a display and a speaker.
[0050]
In addition, a storage unit 118 constituted by a hard disk or the like and a communication unit 119 that exchanges data with other devices via a network such as the Internet are connected to the input / output interface 115. The drive 120 is used when data is read from or written to a recording medium such as the magnetic disk 131, the optical disk 132, the magneto-optical disk 133, and the semiconductor memory 134.
[0051]
As shown in FIG. 3, the recording medium is distributed to provide a program to the user separately from the personal computer, and includes a magnetic disk 131 (including a flexible disk) on which the program is recorded, an optical disk 132 (CD- Consists of package media including ROM (Compact Disc-Read Only Memory), DVD (including Digital Versatile Disc), magneto-optical disc 133 (including MD (Mini-Disc) (registered trademark)), or semiconductor memory 134 In addition, it is configured by a hard disk including a ROM 112 in which a program is stored and a storage unit 118 that are provided to the user in a state of being pre-installed in a computer.
[0052]
In this specification, the steps for describing the program provided by the medium are performed in parallel or individually in accordance with the described order, as well as the processing performed in time series, not necessarily in time series. The process to be executed is also included.
[0053]
Further, in this specification, the system represents the entire apparatus constituted by a plurality of apparatuses.
[0054]
【The invention's effect】
As described above, according to the image processing apparatus, method, and program of the present invention, among the captured image data of the subject, R, G, B signal data, R, G, B signal data, and white When the brightness level calculated from the balance gain is stored and image recording is instructed, the stored R, G, B signal data and the white balance gain at that time are used to create a new brightness. Calculate the level. The stored brightness level and the new brightness level are used, and the AGC gain is set and the electric charge is accumulated by the electronic shutter so that the brightness level of the recorded image is the same as the stored brightness level. Since the time adjustment or the light amount adjustment performed by the iris is controlled, even in situations where the brightness level changes suddenly, there is no difference between the image confirmed by the user at the time of imaging and the actually recorded image. In addition, it is possible to prevent such a change in color tone that causes discomfort to the user.
[Brief description of the drawings]
FIG. 1 is a diagram showing a configuration of an embodiment of an image processing apparatus to which the present invention is applied.
FIG. 2 is a flowchart for explaining the operation of a controller 18;
FIG. 3 is a diagram illustrating a medium.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 10 Image processing apparatus, 11 Lens, 12 Iris, 13 CCD, 14 S / H circuit, 15 AGC circuit, 16 A / D conversion circuit, 17 WB amplifier, 18 Controller, 19 Color difference matrix circuit, 20 Memory, 21 Display and recording Control unit, 22 display unit, 23 recording medium, 24 operation unit

Claims (4)

Imaging means for capturing an image of a subject;
First storage means for storing respective data of the R signal, G signal, and B signal of the image captured by the imaging means;
First calculation means for calculating a first luminance level based on the data stored in the first storage means and a white balance gain;
Second storage means for storing data of the first luminance level calculated by the first calculation means;
When the recording of the image captured by the imaging unit is instructed, the data stored in the first storage unit is read out, and the data is read out based on the read-out data and the white balance gain at that time. Second calculating means for calculating a luminance level of 2;
Using the first brightness level stored by the second storage means and the second brightness level calculated by the second calculation means, the brightness level of the recorded image is the first brightness level. Control means for controlling any one of AGC gain setting, charge accumulation time adjustment by an electronic shutter, or light amount adjustment performed by an iris so as to be the same as a luminance level. An image processing apparatus. An imaging step for capturing an image of a subject;
A first storage control step for controlling storage of respective data of the R signal, the G signal, and the B signal of the image captured in the processing of the imaging step;
A first calculation step for calculating a first luminance level based on the data whose storage is controlled in the processing of the first storage control step and a white balance gain;
A second storage control step for controlling storage of data of the first luminance level calculated in the processing of the first calculation step;
When the recording of the image captured in the process of the imaging step is instructed, the data whose storage is controlled in the process of the first storage control step is read, and the read data and the current time A second calculation step of calculating a second luminance level based on the white balance gain;
Using the first luminance level whose storage is controlled by the processing of the second storage control step and the second luminance level calculated by the processing of the second calculation step , A control step for controlling any one of the AGC gain setting, the charge accumulation time adjustment by the electronic shutter, or the light amount adjustment performed by the iris so that the luminance level becomes the same as the first luminance level. An image processing method comprising: and. An imaging step for capturing an image of a subject;
A first storage control step for controlling storage of respective data of the R signal, the G signal, and the B signal of the image captured in the processing of the imaging step;
A first calculation step for calculating a first luminance level based on the data whose storage is controlled in the processing of the first storage control step and a white balance gain;
A second storage control step for controlling storage of data of the first luminance level calculated in the processing of the first calculation step;
When the recording of the image captured in the process of the imaging step is instructed, the data whose storage is controlled in the process of the first storage control step is read, and the read data and the current time A second calculation step of calculating a second luminance level based on the white balance gain;
Using the first luminance level whose storage is controlled by the processing of the second storage control step and the second luminance level calculated by the processing of the second calculation step , A control step for controlling any one of the AGC gain setting, the charge accumulation time adjustment by the electronic shutter, or the light amount adjustment performed by the iris so that the luminance level becomes the same as the first luminance level. A recording medium on which a computer-readable program is recorded. An imaging step for capturing an image of a subject;
A first storage control step for controlling storage of respective data of the R signal, the G signal, and the B signal of the image captured in the processing of the imaging step;
A first calculation step for calculating a first luminance level based on the data whose storage is controlled in the processing of the first storage control step and a white balance gain;
A second storage control step for controlling storage of data of the first luminance level calculated in the processing of the first calculation step;
When the recording of the image captured in the process of the imaging step is instructed, the data whose storage is controlled in the process of the first storage control step is read, and the read data and the current time A second calculation step of calculating a second luminance level based on the white balance gain;
Using the first luminance level whose storage is controlled by the processing of the second storage control step and the second luminance level calculated by the processing of the second calculation step , A control step for controlling any one of the AGC gain setting, the charge accumulation time adjustment by the electronic shutter, or the light amount adjustment performed by the iris so that the luminance level becomes the same as the first luminance level. A program that causes a computer to execute and.

Images