JP2002199275A - Digital camera and image pickup method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a digital camera and an image pickup method that can reduce a temporal limit in cases where the capturing of an image at a high-speed such as a consecutive shot is required. SOLUTION: The digital camera is provided with an image pickup element 2 that picks up an image of an object, an analog/digital conversion section 3 that applies analog/digital conversion to an output signal of the image pickup element 2, a buffer memory section 4 that temporally stores video data subjected to analog/digital conversion, and a signal processing section 20 that applies various signal processing to video data read from the buffer memory section 4 to generate a luminance signal and a chrominance signal. The signal processing section 20 has a first processing mode where a software program conducts all processing items and a second processing mode where nonlinear processing items such as gamma correction are conducted by a hardware unit and the remaining processing items are conducted by the software program, a control section 17 allows the signal processing section 20 to execute the second processing mode when high-speed processing such as a consecutive shot is required and allows the signal processing section 20 to execute the first processing mode in the normal photographing thereby automatically relieving the load of the software processing.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a digital camera and an image pickup method which do not cause a time limit in continuous shooting performance when high-speed image capturing is performed during continuous shooting such as continuous shooting.

[0002]

2. Description of the Related Art Conventionally, digital still cameras have the advantages of displaying and viewing photographed images in real time, and being able to easily process images by taking them into a personal computer or the like, and are rapidly spreading. Further, a camera having a high-speed continuous shooting function capable of continuous shooting has been developed.

FIG. 10 shows a block diagram of a conventional digital still camera. As shown in FIG. 10, an image incident from the photographing lens 1 is photoelectrically converted by an imaging device 2 such as a CCD having a color filter arranged on a light receiving surface into a video signal, and is digitally converted by an A / D conversion unit 3. You. The digitized video signal is taken into a buffer memory unit 4 such as an SDRAM as one image data. The image data taken into the buffer memory unit 4 is
It is output to the γ conversion unit 5 configured in the flow 18 of the software processing by the CPU calculator. In the γ conversion unit 5,
The gradation conversion is performed so as to correct the non-linear characteristic (so-called γ characteristic) of the output signal with respect to the input (brightness) of the image sensor 2. The gradation-converted video signal is a high-frequency luminance signal (YH)
And a YH generator 6 for separating and generating RGB signals.
It is supplied to the GB separation unit 8. The YH generation unit 6 performs sampling generation of YH pixels corresponding to each of a predetermined number of pixels of the color filter array on the image sensor (CCD) 2, and extracts a high-frequency luminance component. In the RGB separation unit 8,
R, G, and B pixels corresponding to each of a predetermined number of pixels of the color filter array on the image sensor 2 are sampled and generated.
RGB color signals are generated. The output of the YH generator 6 is H
The signal passes through the PF 7 to extract only high frequency components. The output signals of the respective colors of the RGB separation unit 8 are passed through LPFs 9, 10, and 11, and the outputs of the LPFs 9, 10, and 11 output a color signal (C) and an YL generation unit 12 that generates a luminance signal (YL) in a low frequency range. It is output to the C generation unit 13 that generates it. The YL generation unit 12 uses the RGB signals to obtain a predetermined equation (YL = 0.59G +
(0.30R + 0.11B), a low-frequency luminance signal YL is output, added to the YH signal by the adder 14, and the luminance signal Y is output to the data compression unit 15 such as a JPEG unit. . The C generator 13 converts the RB signal and the YL signal into color signals Cr and Cb by an operation represented by a predetermined formula (Cr = R-YL, Cb = B-YL), and switches the Cr and Cb signals to the color signal C by time-division switching. (C = Cr, Cb, Cr, Cb...) And outputs the result to the data compression unit 15. The data compression section 15 receives the Y and C signals as input, generates compressed image data, and outputs the compressed image data to the media section 16. The media unit 16 is a recording medium such as a flash ROM, and image data is written to the media unit 16. The controller 17 controls the above series of operations.

In the above configuration, the video signal of the subject imaged by the image sensor 2 is temporarily stored in the buffer memory unit 4, and the necessary signal processing is performed by the signal processing unit 18 constituted by software using a CPU arithmetic unit. And then stored in the buffer memory unit 4 again.
Data compression such as PEG is performed and recorded on the media unit 16.

However, when high-speed image capture is required, such as during continuous shooting, if there is no capacity to store the data in the buffer memory unit, it takes a long processing time to write the media in the subsequent stage. There is a drawback that the restriction is caused.

On the other hand, with respect to a digital camera that performs signal processing in the camera by software processing such as a CPU, Japanese Patent Application Laid-Open Nos. 7-131748 and 200
There is one described in JP-A-138855.

[0007] Japanese Patent Application Laid-Open No. Hei 7-131748 discloses that an image memory, a CPU working memory, and a buffer memory in an electronic still video camera are used as one memory for various purposes. Depending on the system configuration, such as the transfer speed and the total capacity of the built-in memory,
It is described that a PU uses a memory suitable for the system by selectively setting each memory allocation amount. In addition, it is described that the area setting in the memory is changed between single shooting and continuous shooting during single shooting and continuous shooting. Therefore, effective use of memory is achieved by allocating each memory in one memory for multipurpose use. However, it does not describe handling of processing that requires a particularly long time, such as nonlinear processing.

[0008] JP-A-2000-138855 discloses that
There is described a digital camera capable of reducing a memory capacity required for continuous shooting without compressing image data and obtaining a clear image without performing correction or the like on the image data. Specifically, it is described that when a photograph is taken, each image data sequentially output from the photographing sensor during the period is overwritten and recorded in the memory. When continuous shooting is performed, the image data finally recorded in the memory is overwritten during the continuous shooting period, so that when the subject moves, the moved portion is recorded as a locus. Therefore, there is a drawback that a plurality of continuously shot images cannot be taken out and displayed or printed one by one.

[0009]

As described above, in the conventional digital camera, when high-speed image capturing is required, such as during continuous shooting, the memory capacity required for the above-described capturing operation is stored in the buffer memory unit. If not, the processing time until the subsequent stage of writing to the media becomes long, and the continuous shooting performance is time-limited.

In view of the above-mentioned problems, the present invention provides a time-consuming signal processing such as a non-linear processing when a high-speed image capturing such as a continuous shooting is required. It is an object of the present invention to provide a digital camera and an imaging method capable of reducing restrictions.

[0011]

According to a first aspect of the present invention, there is provided a digital camera, comprising: an image pickup device for picking up an image of a subject; an A / D converter for A / D converting an output signal of the image pickup device; A buffer memory unit for temporarily storing the converted video data, and performing various signal processing on the video data read from the buffer memory unit to generate a luminance signal and a chrominance signal; And a second processing mode in which a specific processing of the signal processing is performed by hardware and the remaining processing is performed by software. The signal processing unit includes a signal processing unit capable of switching modes, and a control unit that controls switching between the first and second processing modes in the signal processing unit.

In the first aspect of the present invention, when high-speed processing is required, the second processing mode is used. In the second processing mode, among the processing contents in the signal processing unit,
Performing some processing that requires time, such as non-linear processing, by hardware and reducing the burden on software, improve the overall performance of signal processing,
Processing time can be shortened. When the high-speed processing is not required, the first processing mode is used. In the first processing mode, image quality degradation due to hardware processing can be prevented by performing processing using only software without using hardware.

[0013] According to a second aspect of the present invention, in the digital camera according to the first aspect, the control unit is configured to perform a normal photographing operation,
At the time of continuous shooting or when the shutter is pressed, the signal processing unit detects that the signal processing is being performed, and switches the signal processing unit to the first processing mode at the time of normal photographing. When the signal processing unit is performing signal processing when pressed, the signal processing unit is controlled to switch to the second processing mode.

According to the second aspect of the present invention, high-speed processing is required during continuous shooting or when the shutter interval is short even if the shutter is pressed and the signal processing unit is performing signal processing. Since signal processing is performed and high-speed processing is not so required at the time of normal photographing, signal processing is performed in the first processing mode with little image quality deterioration.

According to a third aspect of the present invention, in the digital camera according to the first aspect, the control unit is configured to control the first unit for determining whether or not the shutter has been pressed, and to determine whether the shutter has been pressed. A second means for determining whether or not a shooting mode is set; a third means for determining whether or not the signal processing unit is performing signal processing when the second means is not in a continuous shooting mode; A fourth means for shifting to the first processing mode when signal processing is not being performed by the means, and a second means for shifting to the first processing mode when the second means is in the continuous shooting mode and when the third means is performing signal processing. And a fifth means for shifting to the processing mode of (1).

In the third aspect of the present invention, the control unit executes the second processing mode when the continuous shooting mode is determined when it is determined that the shutter is pressed or when the signal processing is being performed. When the camera is neither in the continuous shooting mode nor in signal processing (that is, during normal shooting), control is performed so as to execute a first processing mode that does not use hardware.

According to a fourth aspect of the present invention, in the digital camera according to any one of the first to third aspects, the hardware processing in the second processing mode is a non-linear processing such as gamma correction. I do.

According to the fourth aspect of the present invention, the non-linear processing
For example, in a process such as a gamma correction process that requires a large number of conditional statements that do not become mathematical formulas, the burden of software processing on it is large.
The burden on software can be reduced, the overall performance of signal processing can be increased, and the processing time can be reduced.

According to a fifth aspect of the present invention, in the imaging method, an imaging step of imaging a subject with an imaging element, an A / D conversion step of A / D converting an output signal of the imaging element, and performing the A / D conversion. Storing the video data temporarily stored in a buffer memory unit; and performing various signal processing on the video data read from the buffer memory unit to generate a luminance signal and a color signal. A first processing mode in which the processing is performed by software; and a second processing mode in which a specific processing of the signal processing is performed by hardware, and the remaining processing is performed by software. A signal processing step capable of switching a processing mode; and first and second signal processing steps in the signal processing step.
And a control step of switching and controlling the processing mode.

According to a sixth aspect of the present invention, in the imaging method of the fifth aspect, the control step includes a first step of determining whether or not a shutter has been pressed and a continuous shooting when determining that the shutter has been pressed. A second step of determining whether or not the camera is in a mode, a third step of determining whether or not a signal is being processed when the continuous shooting mode is not set in the second step, and a signal being not being processed in the third step A fourth step of sometimes shifting to the first processing mode, and shifting to the second processing mode when the second step is in the continuous shooting mode and when the third step is during signal processing. And a fifth step.

[0021]

Embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a digital camera according to an embodiment of the present invention. In the present embodiment, a digital still camera will be described as an example. FIG.
The same components as those of 0 are denoted by the same reference numerals.

The digital camera according to the embodiment shown in FIG. 1 has a photographic lens 1 on which a subject image is incident, and a color filter arranged on a light receiving surface. An image sensor 2 for converting the image into a signal;
A / D converter 3 for A / D converting the output signal of image sensor 2
A buffer memory unit 4 for temporarily storing video data A / D-converted by the A / D conversion unit 3, and a buffer memory unit 4
A first processing mode in which all kinds of signal processing are performed by software by performing various kinds of signal processing on video data read from the A second processing mode in which a specific processing of the signal processing is performed by hardware, and the remaining processing is performed by software, and the first and second processing modes are switchable. A data compression unit 15 such as JPEG for inputting the luminance signal Y and the chrominance signal C from the signal processing unit 20 to generate compressed image data, and a flash memory such as a smart media. The media unit 16 for recording the compressed image data and the respective units described above are controlled. In particular, control for switching the first and second processing modes in the signal processing unit 20 is performed. And it is configured to include a possible control unit 17, a. Note that reference numeral 18A in the signal processing unit 20
(Refer to FIG. 1) shows a part (function) realized by the flow of software processing by the CPU computing unit.

The control unit 17 detects a normal photographing operation and a case where the signal processing unit 20 is performing signal processing when the shutter interval is short and the signal processing unit 20 is performing signal processing during continuous shooting or when the shutter is pressed. Is switched to the first processing mode at the time of photographing, and when the signal processing unit 20 is performing signal processing during continuous shooting or when the shutter is pressed, the signal processing unit 20 is switched to the first processing mode.
Can be controlled to switch to the second processing mode.

The embodiment shown in FIG. 1 differs from the conventional example shown in FIG. 10 in that, besides the γ conversion section 5 constituted by software, the γ conversion section constituted by hardware is provided after the buffer memory section 4. A part 22 is newly provided, and this γ
The conversion unit 22 and the γ conversion unit 5 can be appropriately switched and used by software.

The operation of each block constituting FIG. 1 will be described below. 2 to 8 are supplementary diagrams for explaining the operation of each block in FIG.

As shown in FIG. 1, an image incident from a photographing lens 1 is photoelectrically converted by a color image pickup element 2 having a color filter arranged on a light receiving surface into an image signal.
The signal is converted into a digital signal by the D converter 3. The digitized video signal is fetched as one image data into a buffer memory unit 4 composed of an SDRAM or the like. The image data taken into the buffer memory unit 4 is
In addition to being output to the γ conversion unit 5 configured in the software processing flow by the CPU calculator 18A, the output is also output to the γ conversion unit 22 configured by hardware.

As shown in FIG. 2, in the arithmetic processing by the CPU arithmetic unit 18A, first, nonlinear processing such as γ correction is performed, and then other linear processing is performed.

Both the γ conversion section 5 and the γ conversion section 22 have input / output characteristics as shown in FIG. The input / output characteristics shown in FIG. 3 indicate correction data that can be called an inverse γ characteristic for correcting a non-linear characteristic (a so-called γ characteristic) of an output signal with respect to an input (brightness) of the image sensor 2.

The image data read from the buffer memory unit 4 is input to the γ conversion unit 5 and the γ conversion unit 22, respectively, and each γ conversion output is input to the input terminals a and b of the switch 21 as the switching means. You. Switching between the input terminals a and b of the switch 21 is performed by a switching instruction SW from the control unit 17.

The control unit 17 detects whether a normal photographing operation, a continuous shooting operation, or when the CPU operation processing flow 18A in the signal processing unit 20 is performing signal processing when the shutter is pressed. A switch instruction SW is generated, the γ conversion unit 5 is selected by the switch 21 at the time of normal photographing, and the switch 21 is selected at the time of continuous shooting or when the shutter is pressed and the CPU operation processing flow 18A is performing signal processing. The γ conversion unit 22 is selected.

After the video signals whose gradations have been converted by the γ conversion units 5 and 22 pass through the switch 21, the YH generation unit 6 for generating a high frequency luminance signal (YH) and the RGB separation for separating and generating the RGB signals. Output to the unit 8. YH generator 6
Then, when the color filter array of the image sensor 2 is as shown in FIG. 4 (here, R00, R01, R20, R21 are R (red) pixels, G00, G01, G10, G11, G20, G21, G30, G31
Is a G (green) pixel, and B10, B11, B30, and B31 are B (blue) pixels). For example, for each of the four pixels at the center in FIG. make use of,
Sampling as shown in FIG. 5 is performed, and a high-frequency luminance component (YH) is extracted. Note that FIG. 4 shows a filter array of 4 (horizontal) × 4 (vertical) pixels for simplicity of description. FIG. 5 also shows four pixels other than the central part.
Sampling is performed in the same manner as described above, and a high-frequency luminance component (YH) is generated.

In the RGB separation section 8, for example, each of the four pixels at the center of FIG.
Are sampled as in R, G, and B, and RGB color signals are generated. The output of the YH generator 6 is the HPF 7
Then, only high-frequency components are extracted through high-pass filter characteristics as shown in FIG. The output signal of each color of the RGB separation unit 8 is supplied to an LPF 9 having a low-pass filter characteristic as shown in FIG.
The outputs of the LPFs 9, 10, and 11 are output to an YL generator 12 for generating a luminance signal (YL) in a low frequency range and a C generator 13 for generating a color signal (C).

In the YL generation unit 12, the RGB signals are used to calculate the mixture ratio shown in the following equation (1). YL = 0.59G + 0.30R + 0.11B (1) The low-frequency luminance signal YL is output from the YL generation unit 12. Is output, added to the YH signal by the adder 14, and output as a luminance signal Y to the data compression unit 15.

The C generator 13 converts the R and B signals and the YL signal into color signals Cr and Cb by the operation shown in the following equation (2), where Cr = R-YL, Cb = B-YL (2) The signals Cr and Cb are time-divisionally switched to color signals C shown in the following equation (3), and C = Cr, Cb, Cr, Cb... (3) The color signals C are output to the data compression unit 15.

The data compression section 15 receives the Y and C signals as input, generates compressed image data, and outputs it to the media section 16. The compressed image data is written to the media unit 16.
The controller 17 controls the above series of operations.

Next, a control operation performed by the control unit 17 regarding the signal processing in the signal processing unit 20 will be described with reference to a flowchart of FIG.

First, in step S1, it is determined whether or not the shutter has been pressed. If it is determined that the shutter has been pressed, it is determined in step S2 whether the continuous shooting mode has been set. If it is determined in S2 that the mode is not the continuous shooting mode, the process proceeds to step S3, and it is determined whether or not signal processing is being performed. When the signal is not being processed in step S3,
In other words, if neither the continuous shooting mode nor the signal processing is being performed, the process proceeds to step S4, and all the signal processing units 20 are shifted to the software processing mode (first processing mode). When the continuous shooting mode is set in step S2 and the signal processing is being performed in step S3, the process proceeds to step S5, and the mode is shifted to a processing mode by hardware processing and software processing (second processing mode). In this way,
The control unit 17 can execute the first processing mode and the second processing mode in the signal processing unit 20 by switching them by software.

In the above embodiment, in the signal processing section for converting image data of an image pickup device such as a CCD into a luminance signal and a chrominance signal, only a non-linear processing portion which has conventionally been processed by software with a CPU arithmetic unit is used. Into hardware,
The processing time in the CPU computing unit is shortened. The above method is used at the time of continuous shooting or when the shutter interval is short and signal processing is being performed. At the time of normal shooting, hardware is not used, and software processing is performed only by the CPU computing unit.

The restriction of high-speed performance during continuous shooting or when the shutter time interval is short is reduced by substituting the time-consuming nonlinear processing among the processing contents of the CPU arithmetic unit with external hardware. Becomes possible. In addition, during normal shooting, by switching to a method that does not use external hardware, it is possible to prevent deterioration in image quality due to hardware processing.

In the embodiment described above, reference numeral 18
The functional blocks of the respective units indicated by A are realized by the processing flow of the software of the CPU computing unit, but the present invention is not limited to this, and may be configured by actual electronic circuit elements.

[0041]

As described above, according to the present invention, when high-speed operation such as continuous shooting is required, in the signal processing for processing the video signal from the image sensor, the non-linear processing portion is implemented by hardware. By performing the other processing by software, the load of the software processing can be reduced and the processing speed can be increased, and the time limitation such as continuous shooting can be reduced. In addition, during normal shooting, by switching to processing using only software, it is possible to prevent deterioration in image quality due to hardware processing.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a digital camera according to an embodiment of the present invention.

FIG. 2 is a diagram showing a configuration of a calculation process of a CPU calculator in FIG. 1;

FIG. 3 is a diagram showing input / output characteristics of a γ converter in FIG. 1;

FIG. 4 is a diagram showing an example of a color filter array when a CCD is used as an image sensor.

FIG. 5 is a diagram showing YH pixels generated by sampling in the YH generation unit in FIG. 1 corresponding to each pixel constituting the central block in FIG. 4;

FIG. 6 is a diagram showing R, G, and B pixels generated by sampling corresponding to each pixel constituting the central block in FIG. 4 in the RGB separation unit in FIG. 1;

FIG. 7 is a diagram showing input / output characteristics of the HPF in FIG. 1;

FIG. 8 is a diagram showing input / output characteristics of the LPF in FIG. 1;

FIG. 9 is a flowchart showing a control operation performed by a control unit regarding signal processing of the signal processing unit in FIG. 1;

FIG. 10 is a block diagram showing a conventional digital camera.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 2 ... Image sensor 3 ... A / D conversion part 4 ... Buffer memory part 5 ... γ conversion part (software processing) 17 ... Control part 20 ... Signal processing part 21 ... Switch (switching means) 22 ... γ conversion part (hardware processing) )

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) H04N 5/92 H04N 5/91 J // H04N 101: 00 5/92 H

Claims (6)

[Claims] An imaging device for imaging a subject; an A / D conversion unit for A / D converting an output signal of the imaging device; a buffer memory unit for temporarily storing the A / D converted video data; A first processing mode in which various signal processing is performed on the video data read from the buffer memory unit to generate a luminance signal and a chrominance signal, wherein the signal processing is performed by software; A second processing mode in which a specific processing is performed by hardware and the remaining processing is performed by software, and a signal processing unit capable of switching between the first and second processing modes; A digital camera, comprising: a control unit that controls switching between first and second processing modes in the unit. 2. The control unit detects whether the signal processing unit is performing signal processing during normal shooting and during continuous shooting or when a shutter is pressed, and detects the signal processing during normal shooting. Switching the signal processing unit to the first processing mode, and controlling the signal processing unit to switch to the second processing mode when the signal processing unit is performing signal processing during continuous shooting or when a shutter is pressed. The digital camera according to claim 1, wherein A first means for determining whether or not a shutter has been pressed; a second means for determining whether or not a continuous shooting mode has been performed when it has been determined that the shutter has been pressed; Third means for determining whether or not the signal processing unit is performing signal processing when the second means is not in the continuous shooting mode; and the first processing mode when the signal processing is not being performed by the third means. And a second unit which is in the continuous shooting mode by the second unit and when the signal processing is being performed by the third unit.
5. A digital camera according to claim 1, further comprising: a fifth unit for shifting to the processing mode. 4. The digital camera according to claim 1, wherein the hardware processing in the second processing mode is a non-linear processing such as a gamma correction. 5. An image pickup step of picking up an image of a subject with an image pickup device, an A / D conversion step of A / D converting an output signal of the image pickup device, and a temporary buffer memory unit for storing the A / D converted video data. And performing various kinds of signal processing on the video data read from the buffer memory unit to generate a luminance signal and a chrominance signal, wherein the signal processing is performed by software. A signal processing apparatus having a processing mode and a second processing mode in which a specific processing of the signal processing is performed by hardware and the remaining processing is performed by software, and is capable of switching between the first and second processing modes. And a control step of switching and controlling the first and second processing modes in the signal processing step. 6. The control step includes: a first step of determining whether a shutter has been pressed; a second step of determining whether a continuous shooting mode has been performed when the shutter has been pressed; A third step of judging whether or not the signal processing is being performed when the second step is not in the continuous shooting mode; and a fourth step of shifting to the first processing mode when the signal processing is not being performed in the third step. And shifting to the second processing mode in the continuous shooting mode in the second step and during signal processing in the third step.
The imaging method according to claim 5, further comprising the steps of: