JP2007174338A - Imaging apparatus and imaging method thereof - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accomplish an imaging apparatus and imaging method thereof in which image processing such as white balance processing can be speedily performed. <P>SOLUTION: In a driving system where pixel data of a CCD 3 are read while being divided into four fields, first of all, a field (n) is set to 0 (S1), pixel data of the set field (n) are read out (S2) and when the pixel data of the field (n) are completely read (S3, Y), the field (n) is made increment (S4). If the field (n) is neither field 2 (S5, N) nor field 4, processing is returned to S2. If field (n)=field 2 is judged in S5, on the other hand, WB measurement processing is started based on pixel data of field 0 and field 1 (S7) and processing is returned to S2. If field (n)=field 4 is judged in S6, on the other hand, a WB gain is set on the basis of a result of the WB measurement (S8), and color processing such as WB processing is started (S9). <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an imaging apparatus and an imaging method thereof, and more particularly to an imaging apparatus that performs image processing quickly and an imaging method thereof.

In recent years, in a digital camera, one frame of image data obtained from an image sensor such as a CCD is read, and after performing a WB (white balance) measurement process based on the read image data, the result of the measurement process is displayed. Based on this, image processing such as white balance processing for adjusting WB (white balance) gain is performed, and then compression processing is performed to perform recording and storage. This WB (white balance) measurement process measures the degree of influence on the color of the entire subject due to the difference in the light source or the like, and obtains the white balance coefficient with the entire pixel obtained from the CCD as the object of calculation.
In addition to the white balance processing, the image processing may include YUV conversion, gamma conversion, pixel interpolation, various filter processing, etc. Among these image processing, for example, pixel interpolation, various filter processing, etc. When determining the pixel value of a single pixel, an operation including the pixel values of a plurality of surrounding pixels is performed, and depending on the processing contents, a peripheral pixel that is 5 pixels or more away from two pixels may be the target of the calculation. .
The normal WB (white balance) measurement process starts after all pixels are read out from the CCD. In order to shorten this process, the CCD image data is read out in two fields. After the field is read out, before the reading of the two fields is completed, the WB measurement process is started, and the white balance coefficient is calculated using the read out image data of one field and the image data of the two fields currently read out. As a result, a technique has also appeared that aims to shorten the white balance processing by ending the calculation of the white balance coefficient at an early stage (Patent Document 1).

JP Patent Publication No. 2003-87803

However, conventionally, since all the image data of one frame obtained by the CCD is read and transferred, the WB measurement is completed at the end of the WB measurement, and thus the image processing end time such as the white balance processing is delayed. There was a problem that it became late.
Further, according to the technique described in Patent Document 1, the white balance coefficient can be obtained earlier than before by starting the calculation of the white balance coefficient before the reading of the image data of the two fields is completed. However, since the white balance coefficient can only be obtained after all the image data of one frame obtained by the CCD has been read out, it is limited to advancing the image processing time such as white balance processing. was there.
Further, the entire image processing time including image processing other than the white balance processing cannot be efficiently reduced.

  Therefore, the present invention has been made in view of such conventional problems, and an object thereof is to provide an imaging apparatus and an imaging method thereof that can perform image processing such as white balance processing more quickly.

In order to achieve the above object, an image pickup apparatus according to the first aspect of the present invention includes an image pickup device that photoelectrically converts light of a subject;
A plurality of fields are provided so as to be uniformly distributed over the entire pixel region of the image sensor, and reading means for sequentially reading out pixel data for each field;
WB measurement means for calculating a white balance coefficient reflecting color information of the entire pixel area based on pixel data read by the reading means;
White balance means for performing white balance processing for adjusting the color of the pixel data read from the reading means based on the white balance coefficient calculated by the WB measuring means;
Color process means for sequentially executing a plurality of image processing including white balance processing by the white balance means in a series of flows on pixel data sequentially read from the reading means;
With
The WB measuring means is
Of the plurality of fields, when the pixel data of the field necessary for color restoration is read by the reading means, the calculation of the white balance coefficient is started without waiting for the reading of the pixel data of the remaining fields. ,
The color process means includes
After the white balance coefficient is calculated by the WB measuring unit, the execution of a plurality of image processes according to the series of flows is started.

Also, for example, as described in claim 2, the color process means includes:
When pixel data to be subjected to a plurality of image processes according to the series of flows is not read out by the reading unit, a plurality of image processes are performed in a series of flows after the pixel data is read out by the reading unit. You may make it perform sequentially.

Also, for example, as described in claim 3, the color process means includes:
When pixel data to be subjected to a plurality of image processes according to the series of flows is not read out by the readout unit, a plurality of image processes are sequentially executed in a series of streams in synchronization with the readout cycle of the readout unit. You may make it make it.

Also, for example, as described in claim 4, the color process means includes:
The execution of a plurality of image processes according to the series of flows may be started when the pixel data of all fields are read by the reading unit.

Further, for example, as described in claim 5, the color process means includes:
Including image processing that requires pixel data of a plurality of pixels around the pixel in order to determine the pixel data of one pixel, the necessary peripheral pixels are distributed over all of the plurality of fields. May be.

For example, as described in claim 6, the WB measuring means is
The white balance coefficient may be calculated using only the minimum necessary pixel data of one or a plurality of fields in which all the RGB color information is provided.

For example, as described in claim 7, the WB measuring unit is
After the reading means finishes reading the pixel data of one or more fields necessary for color restoration, the calculation of the white balance coefficient is started without waiting for reading of the pixel data of the remaining fields. You may do it.

For example, as described in claim 8, the WB measuring unit is
The reading of the white balance coefficient may be started when the reading means starts reading of the field when the pixel data of one or more fields necessary for color restoration are gathered. .

For example, as described in claim 9, the WB measuring unit is
An integration unit that calculates an integration value of the pixel data may be included, and the white balance coefficient may be calculated based on the integration value calculated by the integration unit.

Further, for example, as described in claim 10, the integration means includes:
When the pixel data for which the integral value is to be calculated is not read by the reading unit, the integral value of the pixel data may be calculated after the pixel data is read by the reading unit. .

For example, as described in claim 11, the integrating means includes
When the pixel data for which the integral value is to be calculated is not read by the reading unit, the integrated value of the pixel data may be calculated in synchronization with the reading cycle of the reading unit.

In order to achieve the above object, an imaging method according to a twelfth aspect of the present invention provides a plurality of fields so as to be evenly distributed and arranged over the entire pixel area of an imaging element that photoelectrically converts light of a subject. A readout step of sequentially reading out pixel data into
A WB measurement step of calculating a white balance coefficient reflecting the color information of the entire pixel region based on the pixel data read by the reading step;
A white balance step for performing white balance processing for adjusting the color of the pixel data read from the read step based on the white balance coefficient calculated by the WB measurement step;
A color process step for sequentially executing a plurality of image processes including a white balance process by the white balance step with respect to pixel data sequentially read from the read step;
With
The WB measurement step includes
The calculation of the white balance coefficient is started without waiting for the pixel data of the remaining fields to be read when the pixel data of the field necessary for color restoration is read by the reading step among the plurality of fields. ,
The color process step includes
After the white balance coefficient is calculated in the WB measurement step, execution of a plurality of image processes according to the series of flows is started.

According to the first aspect of the present invention, an image sensor that photoelectrically converts light of a subject and a plurality of fields are provided so as to be evenly distributed over the entire pixel region of the image sensor, and a pixel is provided for each field. A reading means for sequentially reading data, a WB measuring means for calculating a white balance coefficient reflecting the color information of the entire pixel area based on the pixel data read by the reading means, and calculated by the WB measuring means White balance means for performing white balance processing for adjusting the color of the pixel data read from the reading means based on a white balance coefficient, and pixel data sequentially read from the reading means by the white balance means A color process that sequentially executes multiple image processing including white balance processing in a series of flows. And the WB measuring means reads out pixel data of the remaining fields when the pixel data of the field necessary for color restoration is read by the reading means among the plurality of fields. The calculation of the white balance coefficient is started without waiting, and the color process means starts execution of a plurality of image processes according to the series of flows after the white balance coefficient is calculated by the WB measurement means. Therefore, even during reading of pixel data, WB measurement can be started early.
In addition, the overall processing time including the white balance processing can be shortened, and the execution of the plurality of image processing can be started at an early stage, so that the photographing and recording operation can be performed quickly.

  According to a second aspect of the present invention, when the pixel data to be subjected to a plurality of image processing according to the series of flows is not read by the reading means, the color processing means reads the image data as the reading data. Since a plurality of image processes are executed in a series of flow after being read by the means, a plurality of image processes including a white balance process are performed even while the pixel data is being read. It is possible to reduce the overall processing time of the plurality of image processes including the weight balance process, and to start the execution of the plurality of image processes at an early stage.

  According to a third aspect of the present invention, when the pixel data to be subjected to a plurality of image processing according to the series of flows is not read by the reading means, the color processing means reads the reading cycle of the reading means. Since a plurality of image processes are executed in a series of flows, a plurality of image processes including a white balance process can be performed even while pixel data is being read out. In addition, it is possible to reduce the processing time of the entire plurality of image processes including the weight balance process, and to start the execution of the plurality of image processes at an early stage.

  According to a fourth aspect of the present invention, the color process means starts execution of a plurality of image processes according to the series of flows when the pixel data of all fields are read by the reading means. As a result, it is possible to quickly perform shooting and recording operations that can start the execution of a plurality of image processes at an early stage.

  According to a sixth aspect of the present invention, the WB measuring means calculates the white balance coefficient using only the minimum necessary pixel data of one or a plurality of fields in which all the RGB color information is provided. As a result, the calculation amount is small, and the WB measurement can be completed quickly.

  According to the seventh aspect of the present invention, the WB measuring unit is configured to perform pixel reading of the remaining fields after the reading unit completes reading of pixel data of one or more fields necessary for color restoration. Since the calculation of the white balance coefficient is started without waiting for the data reading, the calculation of the white balance coefficient can be started even during the reading of the pixel data, and the WB measurement is finished quickly. be able to.

  According to the eighth aspect of the present invention, the WB measuring unit starts reading of a field when pixel data of one or more fields necessary for color restoration are gathered by the reading unit. In addition, since the calculation of the white balance coefficient is started, the calculation of the white balance coefficient can be started even when the pixel data of the field necessary for color restoration is being read, and the WB measurement can be performed. Can be completed quickly.

  According to a ninth aspect of the present invention, the WB measurement unit includes an integration unit that calculates an integration value of pixel data, and calculates a white balance coefficient based on the integration value calculated by the integration unit. Therefore, the white balance coefficient can be calculated with high accuracy.

  According to a tenth aspect of the present invention, when the pixel data for which the integration value is to be calculated is not read by the reading means, the integrating means reads the pixel data by the reading means. Since the integration value of the pixel data is calculated from the white balance coefficient, the calculation of the white balance coefficient can be started even during the reading of the pixel data of the field necessary for color restoration, and the WB measurement can be performed quickly. Thus, a plurality of image processing by a series of flows can be started at an early stage.

  According to an eleventh aspect of the present invention, when the pixel data for which an integral value is to be calculated is not read by the reading means, the integrating means synchronizes with the reading cycle of the reading means and outputs the pixel data. Since the integral value is calculated, white balance coefficient calculation can be started even during readout of pixel data of a field necessary for color restoration, and WB measurement can be finished quickly. Thus, a plurality of image processing by a series of flows can be started at an early stage.

  According to the eleventh aspect of the present invention, the image pickup apparatus of the present invention can be realized by causing the image pickup apparatus to perform the method described above.

Hereinafter, the present embodiment will be described in detail with reference to the drawings as an example in which the imaging apparatus of the present invention is applied to a digital camera.
[Embodiment]
A. Configuration of Digital Camera FIG. 1 is a block diagram showing a schematic electrical configuration of a digital camera 1 that implements the imaging apparatus of the present invention.
The digital camera 1 includes a taking lens 2, a CCD 3, a vertical driver 4, a TG (timing
generator) 5, unit circuit 6, DMA controller (hereinafter referred to as DMA) 7, control unit 8, memory 9, DMA 10, WB measurement unit 11, control unit 12, DMA 13, color process processing unit 14, DAM 15, control unit 16, A DMA 17, a display unit 18, a DMA 19, a compression unit 20, a DMA 21, a flash memory 22, and a bus 23 are provided.

The photographing lens 2 projects the light of the subject onto the CCD 3, and the CCD 3 is scan-driven by the vertical driver 4, and photoelectrically converts the intensity of the light of the subject projected at a fixed period to obtain an imaging signal (image). Data) to the unit circuit 6. Here, since the CCD 3 has a Bayer array color filter, the projected light of the subject is photoelectrically converted into the amount of light of each color of RGB values. The operation timings of the vertical driver 4 and the unit circuit 6 are controlled by the control unit 8 via the TG 5.
Here, the CCD reading method is a driving method (four-field reading method) in which all pixels of the CCD are read in four fields.
The field division here is a division in which each field is evenly distributed and arranged in the entire pixel area of the CCD, and a division method for assigning each field in units of horizontal lines as in this embodiment. In addition, a vertical line unit or a block unit may be used. Further, in this embodiment, in order to divide into four fields and to obtain all RGB color information, at least two fields are required. However, a configuration in which all RGB color information is obtained in one field is also possible. Good.

FIG. 2 is a diagram illustrating a state of pixel data read by the four-field reading method.
For example, as shown in FIG. 2, the CCD pixels are 8 × 8 pixels, and FIG. 2A shows the state of the pixels read out in the field 0. The shadow area portion in FIG. 2 represents a pixel that has not yet been read out.

When the reading of the field 0 is completed, the reading of the field 1 is started, and FIG. 2B shows the state of the pixels read out in the case of the field 1.
When field 1 is read, field 2 is read, and field 3 is read.
As shown in FIG. 2D, since there is no shadow area portion, reading of the field 3 completes reading of all pixels, that is, reading of one frame is completed.

  A TG 5 is connected to the unit circuit 6, a CDS (Correlated Double Sampling) circuit that holds the imaging signal output from the CCD 3 by correlated double sampling, and an AGC (AGC (automatic gain adjustment) for the imaging signal after the sampling Automatic gain control) circuit and an A / D converter that converts the analog image pickup signal after the automatic gain adjustment into a digital signal. The image pickup signal of the CCD 3 passes through the unit circuit 6 and is then transferred to the Bayer data by the DMA 7. Is stored in the memory 9.

  The control unit 8 is constituted by a microcomputer including a CPU, a ROM storing an operation program executed by the CPU, and a RAM used as a work memory. The control unit 8 is connected to the CCD via the TG 5. The drive and unit circuit 6 and the DMA 7 are controlled.

The DMA 10 reads the image data of the Bayer data stored in the memory 9 and outputs the read image data to the WB measurement unit 11.
The WB measurement unit 11 integrates the pixel value for each color (R, G, B) of the sent Bayer data, calculates a white balance coefficient, and sends the result to the control unit 12.
The control unit 12 includes a microcomputer including a CPU, a ROM that stores an operation program executed by the CPU, and a RAM that is used as a work memory. The control unit 12 includes the DMA 10 and the WB measurement unit 11. To control.

The DMA 13 reads the image data of the Bayer data stored in the memory 9 and outputs the read image data to the color process processing unit 14.
The color process processing unit 14 performs image processing on image data, and includes a WB gain unit, a gamma conversion unit, a YUV conversion unit, a pixel interpolation unit, a YUV conversion unit, various filter processing units, and the like.
The WB gain unit controls the gain of each color in accordance with the gain of each color set by the control unit 16, that is, performs white balance adjustment processing. The gamma conversion unit converts a γ value of image data, and the YUV conversion unit converts image data of Bayer data into image data of luminance / color difference signals. The pixel interpolating unit interpolates intermediate pixels when enlarging the image or changing the resolution, and the filter processing unit performs noise removal, contour enhancement, and the like.

  Normally, these image processes are performed independently, but in this color process processing unit 14, in order to perform these plural image processes efficiently and quickly, the image data sequentially output from the DMA 13 is processed. Thus, a plurality of image processes are sequentially performed in a series of flows. The order in which the DMA 13 reads out the image data and outputs it to the color process processing unit 14 depends on the contents of each image process processed by the color process processing unit 14. That is, when these image processes include, for example, an image process that requires a pixel value of a peripheral pixel two pixels away from the pixel when determining the pixel value of a certain pixel, the vertical 5 The color process processing is sequentially performed in units of a square area of 25 pixels in total by 25 pixels. Then, the processing is sequentially performed while moving the square area to be subjected to color process processing from the left to the right in the entire image, and when the processing of the square area at the right end is finished, the position of the square area is shifted down by one step and again. The processing is sequentially performed while moving from left to right until the processing of the square area at the lower end is completed. When performing the color process processing in this order, the DMA 13 sequentially reads out newly required image data portions and outputs them to the color process processing unit 14. The moving direction of the square area in this case may be a method of moving the top and bottom first in addition to the method of moving the left and right first.

The DMA 15 causes the memory 9 to sequentially store image data portions after the series of image processing is performed by the color process processing unit.
The control unit 16 is constituted by a microcomputer including a CPU, a ROM storing an operation program executed by the CPU, and a RAM used as a work memory. The control unit 12 includes a DMA 13 and a color process processing unit. 14, the DMA 15 is controlled, and the gain of the WB gain unit of the color process processing unit 14 is set based on the WB measurement result (white balance coefficient) sent from the control unit 12.

The DMA 17 reads the image data of YUV data stored in the memory 8 and outputs it to the display unit 18.
The display unit 18 includes a color LCD and its drive circuit, and displays an image of the image data output from the DMA 12.

The DMA 19 outputs YUV image data or compressed image data stored in the memory 9 by still image shooting to the compression unit 15, or the image data compressed or decompressed by the compression unit 20 in the memory 9. Or remember.
The compression unit 20 is a part that performs compression / decompression of image data (for example, compression / decompression of JPEG format).
The DMA 21 reads the compressed image data stored in the memory 9 and records the compressed image data in the flash memory 22 or stores the compressed image data recorded in the flash memory 22 in the memory 9.
The DMA 17, the display unit 18, the DMA 19, the compression unit 20, and the DMA 21 are controlled by a control unit (not shown).

B. Next, the first embodiment will be roughly described in comparison with the prior art.
According to the prior art, WB measurement is performed after reading (transfer) of pixel data (pixel data for one frame) of all fields is completed, and a color process such as WB (white balance) processing is performed according to the measurement result. Since the processing was being performed, the start of the color process was awaited until the end of the WB measurement even after the reading of the field 3 was completed.

On the other hand, in the first embodiment, the RGB pixel data necessary for the WB measurement is prepared at the end of the reading of the field 1, and therefore, by starting the WB measurement at the end of the reading of the field 1, that is, By performing the WB measurement using only the pixel data of the field 0 and the field 1, the WB measurement is finished at an early stage, and when the transfer of the pixel data of all the fields is completed, the color process processing is started immediately to thereby save time. It is to shorten.
In other words, by performing WB measurement using pixel data of the minimum necessary field for WB measurement, the WB measurement is completed before the transfer of all pixel data is completed, and the color process processing is executed at an early stage. It is.

FIG. 3A shows a time chart of pixel data reading, WB measurement, and color process processing according to the prior art in four-field reading, and FIG. 3B shows pixel data of the first embodiment. Is a time chart of reading, WB measurement, and color process processing.
As can be seen from a comparison between FIGS. 3A and 3B, in the first embodiment, since the color process processing is performed simultaneously with the end of reading of the field 3, the WB measurement and the color process processing are early. You can see that it is done.

C. Operation of digital camera 1

Next, the operation of the digital camera 1 in the embodiment will be described with reference to the flowchart of FIG.
First, when the operation of reading the pixels of the CCD 3 is started, the control unit 8 sets the field n to 0 (step S1). That is, the control unit 8 stores 0 in the field count information storage area of the RAM in the control unit 8.
Then, the control unit 8 starts a process of reading out the pixel data of the set field n (step S2), and the read out pixel data is stored in the memory 9 via the DMA 7. That is, since the control unit 8 reads out the field corresponding to the number stored in the field count information storage area, here, the pixel data of the field 0 (since n = 0) is read out.
Next, the control unit 8 determines whether or not the reading of the field n has been completed (step S3).

If it is determined in step S3 that the reading of the field has not ended, the process stays in step S3 until the reading ends. If it is determined in step S3 that the reading of the field has ended, the control unit 8 increments the field n (n = +1), that is, 1 is added to the number stored in the field count information storage area.
Next, the control unit 8 determines whether or not n = 2 (step S5). This determination is made based on whether or not the number stored in the field count information storage area is two.
If it is determined in step S5 that n = 2 is not true, the controller 8 determines whether n = 4 (step S6), and if it is not n = 4, the control unit 8 returns to step S2.

On the other hand, if it is determined in step S5 that field n = field 2, WB measurement is started (step S7), and the process returns to step S2. At this time, the control unit 8 sends information indicating that the writing of the pixel data up to the field 1 is completed to the control unit 12, and the control unit 12 receives the information to start the WB measurement. 11 is controlled. That is, the DMA 10 reads out the pixel data of the field 0 and the field 1 stored in the memory 9 according to the control signal of the control unit 12 and outputs the pixel data to the WB measurement unit 11, and the WB measurement 11 receives the transmitted pixel. When the integral value of the data is calculated and the integral values of the transmitted pixel data of field 0 and field 1 are all calculated, the white balance coefficient is calculated based on the calculated integral value.
At this time, the control unit 12 does not read all the pixels in the field 0 and the field 1 but reads them out in order to increase the speed of the WB measurement processing.
When the control unit 12 acquires the WB measurement result (white balance coefficient) from the WB measurement unit 11, the control unit 12 outputs the acquired result to the control unit 16.

If it is determined in step S6 that the field n = 4, the WB gain is set based on the WB measurement result (white balance coefficient). At this time, the control unit 8 sends information indicating that all of the pixel data of one frame has been read to the control unit 16, and the control unit 16 receives the information based on the WB measurement result sent from the control unit 12. Then, the gain of the WB gain unit of the color process processing unit 14 is set (step S8).
Note that the control unit 16 may set the gain of the WB gain unit when the WB measurement result is sent from the control unit 12.

  Next, the control unit 16 controls the DMA 13 and the color process processing unit 14 to start color process processing such as WB processing (step S9). That is, the DMA 13 sequentially reads out all the pixel data from the field 0 to the field 3 stored in the memory 9 according to the control signal of the control unit 16 in a block unit of a square area of 5 vertical pixels × 5 horizontal pixels. The data is output to the color process unit 14, and the color process processing unit 14 starts execution of color process processing in which image processing is sequentially performed in units of the sent blocks.

D. As described above, in the first embodiment, when pixel data (image data) of field 0 and field 1 are read from the CCD 3 and stored in the memory 9, only the pixel data of field 0 and field 1 are used. Since the WB measurement process is started, the WB measurement process can be completed early.
In addition, since the WB measurement process can be completed before the reading of the field 3 is completed, a color process such as a white balance process can be started when the reading of the field 3 is completed. It can be shortened.

[Second Embodiment]
Next, a second embodiment will be described.
In the first embodiment, the color process is started after the reading of the field 3 is completed. However, in the second embodiment, after the WB measurement is completed, the reading of the field 3 is performed. Color process processing is started without waiting for completion. That is, as described above, when a series of color process processes are performed while sequentially moving the square area to be processed in the entire image, for example, the square area is composed of 5 vertical pixels × 5 horizontal pixels. Then, at least until the reading of the field 3 is started, all necessary pixels are not arranged in the square area at any position. However, even if the reading of the field 3 is not completed, if the image data for at least the first five pixels of the field 3 is read, the color process processing for the square area corresponding to this portion can be executed. If the reading of the field 3 is advanced by one pixel, the square area to be processed can be advanced by one pixel.

E. Operation of Digital Camera 1 In the second embodiment, the imaging apparatus of the present invention is realized by using the digital camera 1 having the same configuration as that shown in FIG. Before describing the operation of the digital camera 1, the outline of the second embodiment will be described.
FIG. 3C shows a time chart of image data reading, WB measurement, and color process processing according to the second embodiment.
As can be seen from FIG. 3C, when the WB measurement is completed, the color process processing is started.

At this time, since all the pixels for one frame have not yet been read from the CCD 3 at the start of the color process processing, the reading operation from the memory 9 of the DMA 13 does not exceed the writing operation to the memory 9 of the DMA 7. As described above, the control unit 8 sends write pixel number information to the control unit 16, and the control unit 16 reads pixel data from the memory 9 in accordance with the sent pixel number information.
Hereinafter, the operation of the digital camera 1 according to the second embodiment will be described with reference to the flowchart of FIG.

First, when the operation of reading the pixels of the CCD 3 is started, the control unit 8 sets the field n to 0 (stores 0 in the field count information storage area) and the field n (here, n = 0) in step S31. Therefore, the process of reading out the pixel data of field 0) is started (step S32). The read pixel data is stored in the memory 9 via the DMA 7.
Next, the control unit 8 determines whether or not the field currently being read is field 3 (step S33).

If it is determined in step S33 that the currently read field is not field 3, the control unit 8 determines whether or not reading of the read field n has been completed (step S34).
If it is determined in step S34 that the reading is not completed, the process stays in step S34 until the process is completed. If it is determined that the reading is completed in step S34, the process proceeds to step S35, and the control unit 8 increments the field n (n = + 1).

Next, the control unit 8 determines whether n = 2 (step S36).
If it is determined in step S36 that n = 2 is not true, the process returns to step S32, and reading of the pixel data of the field n corresponding to the number stored in the field count information storage area is started.
On the other hand, if it is determined in step S36 that n = 2, the WB measurement process is started (step S37), and the process returns to step S32. At this time, the control unit 8 sends information indicating that the writing of the pixel data up to field 1 to the memory 9 has been completed to the control unit 12, and the control unit 12 receives the information to start the WB measurement process. The DMA 10 and the WB measuring unit 11 are controlled. That is, the DMA 10 reads out the pixel data of the field 0 and the field 1 stored in the memory 9 according to the control signal to the control unit 12 and outputs the pixel data to the WB measurement unit 11, and the WB measurement 11 receives the transmitted pixel. When the integral value of the data is calculated and the integral values of the transmitted pixel data of field 0 and field 1 are all calculated, the white balance coefficient is calculated based on the calculated integral value.

On the other hand, if it is determined in step S33 that the currently read field is field 3, the control unit 8 starts processing to sequentially output the pixel numbers Lw of the pixels in field 3 written in the memory 9 to the control unit 16. (Step S38). That is, the control unit 8 outputs to the control unit 16 a pixel number such as the pixel number Lw of the pixel written in the memory 9 (for example, what number in which line of the three fields).
Next, the control unit 12 determines whether or not the WB measurement is completed (step S39). If the WB measurement is not completed, the control unit 12 remains in step S39 until the completion, and if completed, the process proceeds to step S40.

In step S40, the WB gain is set based on the WB measurement result (white balance coefficient). At this time, when the WB measurement is completed, the control unit 12 outputs the WB measurement result to the control unit 16, and the control unit 16 sets the WB gain based on the output measurement result.
Next, the control unit 16 controls the DMA 13 and the color process processing unit 14 to start color process processing such as WB processing (step S41).
That is, the DMA 13 sequentially reads all the pixel data of the fields 1 to 3 stored in the memory 9 in accordance with the control signal of the control unit 16 in units of blocks of a square area of 5 pixels by 5 pixels. The color process processing unit 14 starts execution of color process processing in which image processing is sequentially performed in units of the transmitted blocks.

Next, the control unit 16 determines whether or not the pixel number Lr of the pixel data of the field 3 that the DMA 13 is about to read from the memory 9 is less than or equal to the pixel number Lw written in the memory 9 sent from the control unit 16. A determination is made (step S42). Thereby, it is possible to determine whether or not all the pixel data of the block unit of 5 × 5 pixels to be read from the memory 9 is stored in the memory 9.
Since this 5 × 5 pixel block unit is composed of the pixels of field 0 to field 3, when the reading of field 3 is started, the pixels necessary for the block unit are not prepared, but the reading of field 3 is performed. For example, image processing can be performed sequentially for each block having necessary pixels.

If it is determined in step S42 that the pixel number Lr is equal to or smaller than the pixel number Lw, the control unit 16 reads out pixel data in block units of a square area of 5 vertical pixels × 5 horizontal pixels via the DMA 13 and sends it to the color process processing unit 14. Then, the color process processing unit 14 executes color process processing for performing image processing in units of the sent blocks (step S43), and proceeds to step S45.
On the other hand, if it is determined in step S42 that the pixel number Lr is not less than the pixel number Lw, the control unit 16 temporarily stops the color process processing and the reading operation from the memory 9 of the DMA 13 (step S44), and returns to step S42. This is because the pixel data of the block that the DMA 13 is about to read is not yet stored in the memory 9.
That is, when the pixel number Lr is less than or equal to the pixel number Lw, the write operation to the memory 9 of the DMA 7 and the read operation from the memory 9 of the DMA 13 are performed in synchronization.
When the process proceeds to step S45, it is determined whether or not the color process processing for the pixel data of all fields has been completed. If it is determined that the process has not been completed, the process returns to step S42.

F. As described above, in the second embodiment, when the pixel data (image data) of field 0 and field 1 are read from the CCD 3 and stored in the memory 9, the WB measurement process starts and the WB measurement process ends. Then, since color process processing such as white balance processing is performed, the color process processing can be completed earlier, and the time from imaging to color process processing collection can be further shortened.

[Third Embodiment]
Next, a third embodiment will be described.
In the first embodiment, the WB measurement is started after the reading of the field 1 is completed. In the third embodiment, the WB measurement is started when the reading of the field 0 is completed. Is. In the third embodiment, it goes without saying that WB measurement is performed using pixel data of only the minimum fields (field 0 and field 1) necessary for WB measurement.

G. Operation of Digital Camera 1 In the third embodiment, the imaging apparatus of the present invention is realized by using the digital camera 1 having the same configuration as that shown in FIG. Before describing the operation of the digital camera 1, an outline of the third embodiment will be described.
FIG. 3D shows a time chart of image data reading, WB measurement, and color process processing according to the second embodiment.
As can be seen from FIG. 3D, when the reading of the field 0 is completed, the WB measurement is started.

At this time, since the pixel data of the field 1 has not yet been read from the CCD 3 at the start of the WB measurement, the reading operation from the memory 9 of the DMA 10 does not exceed the writing operation of the memory 9 of the DMA 7. The control unit 8 sends the write pixel number information to the control unit 12, and the control unit 12 reads the image data from the memory 9 according to the sent pixel number information.
The operation of the digital camera 1 according to the third embodiment will be described below with reference to the flowchart of FIG.

When the operation of reading the pixel data of the CCD 3 is started, in step S51, the control unit 8 sets the field n to 0 (stores 0 in the field count information storage area), and the field n (here, n = 0). , The process of reading out pixel data of field 0) is started (step S52). The read pixel data is stored in the memory 9 via the DMA 7.
Next, the control unit 8 determines whether or not the reading of the field n being read has ended (step S53).

If it is determined in step S53 that the reading is not completed, the process stays in step S53 until it is completed. If it is determined in step S53 that the reading is completed, the process proceeds to step S54, and the control unit 8 increments n (n = + 1). .
Next, the controller 8 determines whether or not n = 1 (step S55).
If it is determined in step S55 that n = 1 is not true, the controller 8 determines whether n = 4 (step S56), and if it is not n = 4, the control unit 8 returns to step S52.

On the other hand, if it is determined in step S55 that n = 1, WB measurement processing is started (step S57), and the process returns to step S52. At this time, the control unit 8 sends information indicating that the writing of the pixel data of the field 0 is completed to the control unit 12, and the control unit 12 receives the information to start the WB measurement process. 12 is controlled.
That is, the DMA 10 sequentially reads out the pixel data stored in the memory 9 according to the control signal of the control unit 12 in the line order of the field, that is, the first line, the second line,. When the reading of the field 0 is finished, the first line, the second line,... Of the field 1 are sequentially read, and the read pixel data is output to the WB measuring unit 11, which is sent to the WB measuring unit 11. WB measurement processing is performed based on the received pixel data.

At this time, when the WB measurement process is started, the control unit 8 performs a process of sequentially outputting the pixel number Lw of the field 1 written in the memory 9 to the control unit 12. That is, the control unit 8 outputs to the control unit 12 a pixel number such as the pixel number of the pixel written in the memory 9 (for example, what number on the line in the field 1). The pixel number Lw is output until all the pixel data of the field 1 are output.
In the WB measurement process, the control unit 12 determines whether or not the pixel number Lr that the DMA 10 is about to read from the memory 9 is less than or equal to the pixel number Lw written in the memory 9 sent from the control unit 8. Only when the pixel number Lr is less than or equal to the pixel number Lw, the pixel with the pixel number Lr is read and output to the WB measurement. When the pixel number Lr is not less than or equal to the pixel number Lw, The reading operation and integration value calculation are temporarily stopped. This is because when Lr is not equal to or less than Lw, the pixel data of the pixel to be read from the memory 9 is not stored in the memory 9.
That is, when the pixel number Lr is equal to or smaller than the pixel number Lw, the writing operation to the memory 9 of the DMA 7 and the reading operation from the memory 9 of the DMA 10 are performed in synchronization.

Since the pixel data of the field 0 is already stored in the memory 9, it is needless to say that the pixel number of the field 1 is the object of determination here. That is, when the DMA 10 reads the pixel data of the field 0, the pixel number Lr is determined to be equal to or less than the pixel number Lw.
Further, when all the integral values of the transmitted pixel data of the field 0 and the field 1 are calculated, the WB measurement unit 11 calculates a white balance coefficient based on the calculated integration value, and sends the result to the control unit 16. Needless to say, output.

On the other hand, if it is determined in step S56 that n = 4, the control unit 16 sets the gain of the WB gain unit of the color process processing unit 14 based on the transmitted WB measurement result (white balance coefficient). (Step S58). At this time, the control unit 8 sends information indicating that all of the pixel data of one frame has been read to the control unit 16, and the control unit 16 receives the information based on the WB measurement result sent from the control unit 12. Thus, the gain is set. Note that the control unit 16 may set the gain of the WB gain unit when the WB measurement result is sent from the control unit 12.
Next, the control unit 16 controls the DMA 13 and the color process processing unit 14 to start color process processing such as WB processing (step S59).

H. As described above, in the third embodiment, when the pixel data (image data) in the field 0 is read from the CCD 3 and stored in the memory 9, the WB measurement process is started. The WB measurement can be finished before the data is read, and a color process such as white balance processing can be started when the reading of the field 3 is finished.

[Modification]
The above-described embodiments can be modified as follows.
(1) In each of the above embodiments, the pixels of the CCD 3 are read in four fields, but each RGB color is read out in an integer m field of 1 to n-1 for an integer n of 2 or more. May be divided into arbitrary n fields and read out.

(2) In each of the above embodiments, the WB measurement unit is provided in hardware. However, the WB measurement process may be performed by software.
(3) Moreover, in each said embodiment, although the measurement object was set as WB measurement, another thing may be sufficient. In other words, any other measurement may be used as long as it is a measurement for performing image processing on the image data obtained by the CCD 3.
(4) In the above embodiments, WB measurement is performed based on the image data obtained from the CCD 3, and white balance processing is performed based on the result. WB measurement may be performed on the obtained image data, and white balance processing may be performed based on the result.

(5) Further, the second embodiment and the third embodiment may be combined.
That is, for example, when all of the image processing included in the color process processing can be started when the pixel data of field 0 and field 1 are ready, as shown in FIG. When the pixel data of 0 is read and the pixel data of the field 0 is stored in the memory 9, the WB measurement process is started. After the reading of the field 1 is completed, the color process process is started immediately after the WB measurement process is completed. You may do it.
In this case, the control unit 8 sends the writing pixel number information to the control unit 12 and the control unit 16 so that the WB measurement process and the color process process do not exceed the write operation to the memory 9 of the DMA 8. The control unit 16 reads image data from the memory 9 in accordance with the sent pixel number information.

  (6) In the second embodiment and the third embodiment, the control unit 8 outputs the pixel number Lw written in the memory 9 to the control unit 12 and the control unit 16. The line numbers written in the memory 9 in units of lines instead of in units of pixels may be output to the control unit 12 or the control unit 16.

(7) In the second embodiment, when the reading of the pixel data in the field 3 is started, the writing pixel number Lw is output to the control unit 16 (step S38). The writing pixel number Lw may be sequentially output from the time when the data is read. That is, the writing pixel number Lw may be output in all readings of the fields 0 to 3.
Also in the third embodiment, when the reading of the field 0 is completed and the WB measurement process is started (step S57), the control unit 8 outputs the write pixel number Lw to the control unit 12. However, the writing pixel number Lw may be output to the control unit 12 from the start of reading of the field 0.

(8) Moreover, you may make it combine each said embodiment and said modification (1) thru | or (5) arbitrarily.
(9) Furthermore, the digital camera 1 in the above embodiment is not limited to the above embodiment, but is a mobile phone with a camera, a PDA with a camera, a personal computer with a camera, an IC recorder with a camera, or a digital video camera. In short, any device that can perform image processing on image data may be used.

1 is a block diagram of a digital camera 1 according to an embodiment of the present invention. It is a figure which shows the mode of the pixel data read by 4 field read-out system. 2 is a time chart of pixel data reading, WB measurement, and color process processing. It is a flowchart which shows operation | movement of the digital camera of 1st Embodiment. It is a flowchart which shows operation | movement of the digital camera of 2nd Embodiment. It is a flowchart which shows operation | movement of the digital camera of 3rd Embodiment.

Explanation of symbols

1 Digital Camera 2 Shooting Lens 3 CCD
4 Vertical driver 5 TG
6 Unit circuit 7 DMA
8 Control unit 9 Memory 10 DMA
11 WB measurement unit 12 Control unit 13 DMA
14 Color process processing unit 15 DMA
16 Control unit 17 DMA
18 Display 19 DMA
20 Compression unit 21 DMA
22 Flash memory 23 Bus

Claims (12)

An image sensor that photoelectrically converts the light of the subject;
A plurality of fields are provided so as to be uniformly distributed over the entire pixel region of the image sensor, and reading means for sequentially reading out pixel data for each field;
WB measurement means for calculating a white balance coefficient reflecting color information of the entire pixel area based on pixel data read by the reading means;
White balance means for performing white balance processing for adjusting the color of the pixel data read from the reading means based on the white balance coefficient calculated by the WB measuring means;
A color process unit that sequentially executes a plurality of image processes including a white balance process by the white balance unit with respect to pixel data sequentially read from the read unit;
With
The WB measuring means is
Of the plurality of fields, when the pixel data of the field necessary for color restoration is read by the reading means, the calculation of the white balance coefficient is started without waiting for the reading of the pixel data of the remaining fields. ,
The color process means includes
An image pickup apparatus, wherein after the white balance coefficient is calculated by the WB measurement unit, execution of a plurality of image processes according to the series of flows is started. The color process means includes
When pixel data to be subjected to a plurality of image processes according to the series of flows is not read out by the reading unit, a plurality of image processes are performed in a series of flows after the pixel data is read out by the reading unit. The imaging apparatus according to claim 1, wherein the imaging apparatus is sequentially executed. The color process means includes
When pixel data to be subjected to a plurality of image processes according to the series of flows is not read out by the readout unit, a plurality of image processes are sequentially executed in a series of streams in synchronization with the readout cycle of the readout unit. The imaging apparatus according to claim 2, wherein: The color process means includes
The imaging apparatus according to claim 1, wherein execution of a plurality of image processes according to the series of flows is started when pixel data of all fields is read by the reading unit. The color process means includes
Including image processing that requires pixel data of a plurality of pixels around the pixel in order to determine pixel data of one pixel, and that the necessary peripheral pixels are dispersed in all of the plurality of fields. The image pickup apparatus according to claim 1, wherein the image pickup apparatus is characterized. The WB measuring means is
6. The image pickup apparatus according to claim 1, wherein the white balance coefficient is calculated using only pixel data of one or a plurality of minimum fields in which all color information of RGB is gathered. . The WB measuring means is
After the reading means finishes reading the pixel data of one or more fields necessary for color restoration, the calculation of the white balance coefficient is started without waiting for reading of the pixel data of the remaining fields. The imaging apparatus according to claim 1, wherein the imaging apparatus is an imaging apparatus. The WB measuring means is
The white balance coefficient calculation is started when the reading means starts reading a field when pixel data of one or more fields necessary for color restoration are gathered. The imaging device according to claim 1. The WB measuring means is
9. The imaging apparatus according to claim 8, further comprising an integration unit that calculates an integration value of the pixel data, and calculating a white balance coefficient based on the integration value calculated by the integration unit. The integration means includes
When the pixel data for which an integral value is to be calculated is not read by the reading means, the integral value of the pixel data is calculated after the pixel data is read by the reading means. The imaging device according to claim 9. The integration means includes
11. The integrated value of pixel data is calculated in synchronization with a reading cycle of the reading means when pixel data for which an integral value is to be calculated is not read by the reading means. The imaging device described. A step of reading out pixel data sequentially for each field by providing a plurality of fields so as to be uniformly distributed over the entire pixel region of the image sensor that photoelectrically converts the light of the subject;
A WB measurement step of calculating a white balance coefficient reflecting the color information of the entire pixel region based on the pixel data read by the reading step;
A white balance step for performing white balance processing for adjusting the color of the pixel data read from the read step based on the white balance coefficient calculated by the WB measurement step;
A color process step for sequentially executing a plurality of image processes including a white balance process by the white balance step with respect to pixel data sequentially read from the read step;
With
The WB measurement step includes
The calculation of the white balance coefficient is started without waiting for the pixel data of the remaining fields to be read when the pixel data of the field necessary for color restoration is read by the reading step among the plurality of fields. ,
The color process step includes
An imaging method, comprising: starting execution of a plurality of image processes according to the series of flows after the white balance coefficient is calculated in the WB measurement step.

Images