JP4281161B2 - Electronic camera - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an electronic camera that realizes efficient signal processing by dynamically switching a signal processing path of image data according to whether or not RAW data is used.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, electronic cameras that can select the following two types of data formats when recording captured image data are known.
(1) Lossy compressed data such as JPEG that has undergone a complete image processing (2) Raw data from the image sensor (so-called RAW data)
[0003]
The former lossy compressed data has the advantage that it has a relatively small code amount and can store a large number of images on an external recording medium such as a memory card. Also, it is a general-purpose recording format that can be decrypted by general-purpose image browsing software and printed and displayed as it is.
[0004]
On the other hand, the latter RAW data is image data faithful to the output signal of the image sensor, and is a data recording format on the premise of external processing. Since this RAW data does not undergo much irreversible gradation conversion and data compression, the amount of information such as the number of quantization bits is large, and the dynamic range as image information is wide. Therefore, it is strong in data processing and has an advantage that a fine gradation component is hardly lost. For this reason, this type of RAW data is a data format that is particularly suitable for printing and design applications that require advanced data processing and high quality.
[0005]
FIG. 4 is a block diagram of an electronic camera disclosed in Japanese Patent Laid-Open No. 11-112932 and capable of selecting the above two types of data recording formats.
Hereinafter, the operation of the conventional example will be described with reference to FIG.
[0006]
<When recording general lossy compressed data>
First, a case where general lossy compressed data is recorded will be described.
A photographing lens 86 is attached to the electronic camera 85. The image sensor 87 photoelectrically converts a light image formed through the photographing lens 86 to generate image data.
The image data generated in this way is linearly quantized via the A / D converter 88 and converted to RAW data of about 12 to 16 bits. This RAW data is temporarily stored in the image memory 89.
[0007]
The notch LPF circuit 90 reads RAW data from the image memory 89 and removes a luminance step due to the color filter. The APC processing circuit 91 performs edge enhancement on the luminance signal Y (or G) from which the luminance step has been removed. The γ conversion circuit 92 executes γ correction on the luminance signal Y (or G).
On the other hand, the interpolation LPF circuit 94 reads RAW data from the image memory 89, and performs interpolation processing and low-pass processing of complementary color signals. The color conversion MAT circuit 95 converts this complementary color signal to generate a color signal RB. The γ conversion circuit 96 performs γ correction on the color signal RB.
[0008]
The luminance / color difference signal generation circuit 93 performs color difference conversion on these signals to generate luminance color difference data (Y, Cb, Cr) for each 8 bits.
The compression processing circuit 97 performs image compression (DCT conversion / quantization / encoding) on the luminance color difference data (Y, Cb, Cr) to generate lossy compressed data. The lossy compressed data generated in this way is recorded on an external recording medium 98 such as a memory card.
[0009]
<< When recording RAW data >>
Next, a case where RAW data is recorded will be described.
The image data captured by the image sensor 87 is converted into RAW data of about 12 to 16 bits via the A / D converter 88 and is temporarily stored in the image memory 89.
The RAW data stored in the image memory 89 in this manner is recorded on the external recording medium 98 after being subjected to lossless compression or the like.
With the above operation, in the conventional example, the above two types of data recording formats can be appropriately selected.
[0010]
[Problems to be solved by the invention]
Usually, an electronic camera requires extra image processing time compared to a silver halide camera. Therefore, in order to make the feeling of use of the electronic camera the same as that of a silver salt camera, it is strongly demanded to shorten the image processing time as much as possible.
By the way, in the above-described conventional example, an operation of reading and writing RAW data through the image memory 89 is always inserted. Therefore, there is a tendency that the signal processing of the lossy compressed data is delayed by the amount of time for reading and writing RAW data.
[0011]
Further, in the above conventional example, relatively complicated processing circuits centering on pixel value matrix calculation are concentrated in the dotted line range P shown in FIG. Since these processing circuits handle raw data with a large number of quantized bits as they are, the circuit configuration of the processing circuit tends to be complicated, and the time required for signal processing tends to increase.
[0012]
Accordingly, an object of the present invention is to provide an electronic camera capable of further reducing the signal processing time while allowing the above two types of data formats to be selected as in the conventional example.
[0013]
[Means for Solving the Problems]
Hereinafter, means for solving the problem will be described while associating the reference numerals of the embodiments. Note that the correspondence here is for reference and does not limit the present invention.
[0014]
The electronic camera of the present invention includes an image sensor (11) and first signal processing means (13) for performing at least A / D conversion signal processing on a video signal generated by the image sensor to convert it into digital image data. , 15, 16), second signal processing means (17) for performing irreversible signal processing on the image data converted through the first signal processing means, and an image memory capable of temporarily storing the image data In the electronic camera equipped with (23), signal path switching between the two signal processing means is performed according to the following two operation modes, and irreversible signal processing is performed by the RAW data (second signal processing means). If the RAW data is not necessary, the following high-speed mode is selected and executed, and the RAW data is operated. Characterized by comprising an operation control means for selecting and executing an original mode following (21,22,30) if the.
[0015]
(1) “High-speed mode”: A mode in which a series of signal processing is continuously executed by giving the output of the first signal processing means to the second signal processing means and by synchronously operating the two signal processing means ( 2) "Original mode" ··· The output of the first signal processing means is temporarily stored in the image memory, the image data read from the image memory is given to the second signal processing means, and the two signal processing means are operated at individual timings. Mode to let [0016]
In the above configuration, an operation control unit is newly provided to dynamically switch the signal path between the first signal processing unit and the second signal processing unit according to the operation mode selection.
In the high-speed mode, the operation control means first gives the output of the first signal processing means to the second signal processing means, and sets the signal path so that the two signal processing means can process the image data in a pipeline manner. . As a result, the signal processing is synergistically speeded up based on the following two speedup actions.
[1] The time required to read / write RAW data to / from the image memory 89 in the conventional example (FIG. 4) becomes unnecessary.
[2] Since the two signal processing means operate synchronously, signal processing is executed in real time between the two signal processing means without delay.
[0017]
On the other hand, the operation control means sets the signal path so as to give the output of the first signal processing means to the second signal processing means via the image memory in the original image mode. As a result, RAW data (here, image data before being subjected to irreversible signal processing by the second signal processing means) remains in the image memory. Therefore, this RAW data can be used later.
[0018]
As described above, according to the present invention, it is possible to use RAW data by selecting an original image mode as necessary, and it is possible to reliably reduce the signal processing time by selecting a high-speed mode.
[0019]
The operation control means, the "storage area in the image memory is provided to store the output of the first signal processing means in the original mode (23C)", in the high-speed mode, saves the image data processing step It may also be used as a buffer area for
[0020]
Normally, in the original image mode, since RAW data is stored, a relatively large storage area must be secured on the image memory. Such a storage area is not required in the high-speed mode and is in an idle state.
Therefore, the operation control means uses the storage area in the idle state in the high-speed mode as a temporary save area for the image data before the medium recording is completed, so as to efficiently use the storage area.
As a result, in the high-speed mode, a large amount of image data being processed can be saved, and new imaging can be started in advance without waiting for completion of all signal processing.
As described above, according to the present invention, it is possible to obtain an effect such as further shortening the photographing interval of the electronic camera while effectively using the image memory in conjunction with the switching of the operation mode.
[0021]
Further, the operation control means, the RAW data of stripping in the image memory (storage or recording medium) external output may be.
[0022]
In the above configuration, when the user does not intend to use the RAW data, the high speed mode is selected as a result, and the signal processing is automatically speeded up.
On the other hand, when the user tries to use the RAW data, the original image mode is appropriately selected, and the RAW data can be used.
[0023]
Further, the operation control means, in the original mode, the operation clock of the second signal processing means, may be set faster than the operation clock of the first signal processing means.
[0024]
Usually, in the high-speed mode, the two signal processing units perform a synchronous operation in synchronization with the signal output of the image sensor, and therefore the operation clock cannot be increased without permission.
However, in the original picture mode, an image memory is arranged between the two signal processing means, and both operate at individual timings. Therefore, in the original picture mode, the operation clock of the second signal processing means can be set independently.
Therefore, in the above configuration, the operation control unit sets the operation clock of the second signal processing unit at a higher speed than the operation clock of the first signal processing unit in conjunction with the signal path switching in the original image mode.
As a result, with the second signal processing means alone, the signal processing time is shortened in the original image mode than in the high speed mode. As a result, it is possible to speed up signal processing as much as possible in the original image mode.
[0025]
The second signal processing means, "irreversible gradation conversion", may be a means for performing at least one of the "irreversible pixel thinning."
[0026]
In the high speed mode of the present invention, two signal processing means are connected in a pipeline manner. Therefore, like the circuit in the dotted line range P of the conventional example (FIG. 4), a circuit that presupposes a complicated reference of the image memory 89 or a circuit that performs complicated image compression uses the second signal for performing pipeline processing. Not very suitable as a processing means.
[0027]
On the other hand, “irreversible gradation conversion such as γ correction” and “irreversible pixel thinning such as color difference thinning” are simple in processing operation because processing is performed in units of one pixel or adjacent pixels. Particularly suitable for pipeline-type signal processing in high-speed mode.
[0028]
Further, these irreversible signal processes are executed with a relatively simple circuit configuration, so that they are resistant to the speeding up of the operation clock as described above , and are suitable for speeding up the original picture mode as much as possible.
[0029]
Furthermore, in these irreversible signal processes, the amount of image data information is reduced. Therefore, the image processing circuit located in the subsequent stage can handle a smaller amount of information. Therefore, it is suitable for simplifying the circuit configuration of the image processing circuit, which is likely to become a bottleneck for signal processing, and for further increasing the image processing speed.
[0030]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described below with reference to the drawings.
This embodiment is an embodiment of the electronic camera 10 corresponding to the inventions described in claims 1 to 5.
[0031]
FIG. 1 is a configuration block diagram of the electronic camera 10.
In FIG. 1, a photographing lens 10 a is attached to the electronic camera 10. The light receiving surface of the image sensor 11 is disposed in the image space of the photographing lens 10a. The image pickup device 11 is supplied with control pulses for controlling accumulation, discharge, readout, and the like of signal charges from the timing generator 12.
Image data output from the image sensor 11 is input to the image signal processor 14 via the A / D converter 13. The timing generator 12 supplies an operation clock φA to the A / D converter 13 and the image signal processor 14.
[0032]
The image signal processor 14 includes a plurality of operation units as described below.
Signal level correction unit 15
・ White balance correction unit 16
・ Γ correction part 17
Color interpolation unit 18
Color difference conversion unit 19
・ JPEG compression unit 20
-Mode control unit 21
[0033]
Image data output from the image signal processor 14 is input to the CPU 22. The CPU 22 transmits operation mode setting information to the mode control unit 21 in the image signal processor 14. The CPU 22 supplies two types of operation clocks φB and φC to the image signal processor 14.
[0034]
The electronic camera 10 is provided with an image memory 23 for temporarily storing image data. The image signal processor 14 and the CPU 22 each access the image memory 23 via a dedicated data bus.
The electronic camera 10 is provided with a monitor 25 for displaying a monitor image. The monitor 25 is connected to the CPU 22 via the monitor display circuit 24.
[0035]
Further, the electronic camera 10 is provided with a card interface 26 to which the memory card 27 can be attached and detached. The card interface 26 is connected to the CPU 22.
The electronic camera 10 is provided with a data terminal 29 for exchanging data with an external device. The data terminal 29 is connected to the CPU 22 via the interface 28.
Further, the electronic camera 10 is provided with a mode setting button 30 and other operation members. The switch outputs of these operation members are supplied to the CPU 22.
[0036]
《Explanation of high-speed mode operation》
Hereinafter, the operation of the electronic camera 10 in the high speed mode will be described.
First, the user operates the mode setting button 30 to set whether or not RAW data is required. This setting information is transmitted to the mode control unit 21 via the CPU 22.
[0037]
The mode control unit 21 switches the signal path of the image signal processor 14 to a signal path corresponding to the high-speed mode when it is operated that RAW data is unnecessary.
FIG. 2 is an explanatory diagram showing such a signal path corresponding to the high-speed mode.
As shown in FIG. 2, the mode control unit 21 connects the A / D conversion unit 13, the signal level correction unit 15, the white balance correction unit 16, and the γ correction unit 17 in a pipeline shape.
[0038]
After that, the mode control unit 21 supplies the operation clock φA of the timing generator 12 to the signal level correction unit 15, the white balance correction unit 16, and the γ correction unit 17, and sets these processing units to operate synchronously. To do.
In such a state, image data is output from the image sensor 11. This image data is linearly quantized by the A / D converter 13 and converted into digitized image data of about 12 to 16 bits.
[0039]
The digitized image data is sequentially output to the γ correction unit 17 after sequentially passing through a signal level correction unit 15 that performs clamp correction and gain correction and a white balance correction unit 16 that performs white balance correction.
The γ correction unit 17 performs γ correction on the image data, and reduces the number of quantization bits of the image data to about 8 bits and outputs the result.
[0040]
A series of signal processing so far is executed in real time in units of pixels in synchronization with the operation clock φA supplied from the timing generator 12.
The output of the γ correction unit 17 (non-linear processing data of about 8 bits) is temporarily stored in the storage area 23A in the image memory 23.
At this time, the mode control unit 21 expands the storage capacity of the storage area 23 </ b> A by allocating a “RAW data storage area 23 </ b> C” used in the original image mode described later to a part of the storage area 23 </ b> A. As a result, it is possible to save a plurality of frames of image data during processing in the storage area 23A. The electronic camera 10 can start the signal processing of the next frame without waiting for the completion of the signal processing by this saving operation.
[0041]
The color interpolation unit 18 appropriately reads out image data from the storage area 23A, executes color interpolation processing by local pixel calculation, and aligns the three RGB color components for all pixels. The color difference conversion unit 19 sequentially converts the RGB components into color difference data including luminance Y and color differences Cr and Cb.
The processing of the color interpolation unit 18 and the color difference conversion unit 19 here is executed by the operation clock φB supplied from the CPU 22.
[0042]
The color difference data (Y, Cb, Cr) converted in this way is temporarily stored in the storage area 23B in the image memory 23.
In order to preview the captured image at this stage, the monitor display circuit 24 reads out the color difference data (Y, Cb, Cr) in the storage area 23B via the CPU 22 and displays it on the monitor 25.
[0043]
The JPEG compression unit 20 reads out the color difference data (Y, Cb, Cr) from the storage area 23B, and executes irreversible image compression (DCT conversion / quantization / encoding) in synchronization with the operation clock φB. The irreversibly compressed image data is recorded in the memory card 27 via the CPU 22 and the card interface 26.
Depending on the set value of the compression rate, the CPU 22 may read the color difference data (Y, Cb, Cr) directly from the storage area 23B and record it on the memory card 27 via the card interface 26.
With the above operation, the high-speed mode processing is completed.
[0044]
《Explanation of original mode operation》
On the other hand, when the user operates via the mode setting button 30 that RAW data is required, the mode control unit 21 switches the signal path of the image signal processor 14 to a signal path corresponding to the original image mode.
FIG. 3 is a diagram showing a signal path corresponding to the original picture mode.
[0045]
As shown in FIG. 3, the mode control unit 21 sets a signal path so as to give the output of the white balance correction unit 16 to the γ correction unit 17 via the image memory 23.
Thereafter, the mode control unit 21 supplies the operation clock φA of the timing generator 12 to the signal level correction unit 15 and the white balance correction unit 16.
[0046]
On the other hand, the mode control unit 21 switches the operation clock of the γ correction unit 17 to an operation clock φC that is faster than the operation clock φA.
In such a state, image data is output from the image sensor 11. This image data is linearly quantized by the A / D converter 13 and converted into digitized image data of about 12 to 16 bits.
[0047]
The digitized image data passes through the signal level correction unit 15 and the white balance correction unit 16 in this order, and is temporarily stored in the storage area 23C in the image memory 23 as RAW data of about 12 to 16 bits.
The γ correction unit 17 performs γ correction while reading out the RAW data from the storage area 23C in synchronization with the high-speed operation clock φC, and outputs it as non-linear processing data of about 8 bits.
[0048]
The nonlinear processing data of about 8 bits is temporarily stored in the storage area 23A in the image memory 23.
The color interpolation unit 18 appropriately reads out image data from the storage area 23A, executes color interpolation processing by local pixel calculation, and aligns the three RGB color components for all pixels. The color difference conversion unit 19 sequentially converts the RGB components into color difference data including luminance Y and color differences Cr and Cb.
[0049]
The color difference data (Y, Cb, Cr) converted in this way is temporarily stored in the storage area 23B in the image memory 23.
The JPEG compression unit 20 executes image compression (DCT conversion / quantization / encoding) in synchronization with the operation clock φB while appropriately reading out the color difference data (Y, Cb, Cr) from the storage area 23B. The irreversibly compressed image data is recorded in the memory card 27 via the CPU 22 and the card interface 26.
[0050]
Depending on the set value of the compression rate, the CPU 22 may read the color difference data (Y, Cb, Cr) directly from the storage area 23B and record it on the memory card 27 via the card interface 26.
On the other hand, RAW data remains as it is in the storage area 23C. The CPU 22 reads this RAW data and outputs it to the interface 28. The interface 28 outputs this RAW data to the outside via the data terminal 29.
Thus, the operation of the original image mode is completed.
[0051]
<< Effects of the embodiment >>
As described above, in the present embodiment, the mode control unit 21 dynamically switches the signal path inside the image signal processor 14. As a result, in the original image mode, the RAW data is saved in the storage area 23C, and the RAW data can be used later.
[0052]
On the other hand, in the high-speed mode, the signal processing is speeded up synergistically by the following two speed-up actions.
[1] The reading / writing of RAW data with respect to the image memory 23 is omitted.
[2] A series of signal processing (imaging device 11 → A / D conversion unit 13 → signal level correction unit 15 → white balance correction unit 16 → γ correction unit 17 → storage area 23A) is processed in real time in synchronization with the operation clock φA. To do.
[0053]
As a result, signal processing up to γ correction can be almost completed in the high-speed mode within the time required to save the RAW data in the image memory 23 in the original image mode.
[0054]
In the present embodiment, the idle storage area 23C is effectively used as a part of the storage area 23A in the high-speed mode. As a result, it is possible to use the storage area 23A having an increased capacity as a signal processing saving area, and it is possible to remarkably shorten the shooting interval in the high-speed mode (particularly during continuous shooting).
In the present embodiment, the mode control unit 21 switches the operation clock of the γ correction unit 17 from φA to φC at high speed in conjunction with the signal path switching to the original image mode. Accordingly, it is possible to shorten the time required for γ correction as much as possible even in the original image mode.
[0055]
<< Additional items of embodiment >>
In the above-described embodiment, the case where the RAW data is externally output in the original image mode has been described, but the present invention is not limited thereto. For example, in the original image mode, the CPU 22 (operation control means) may store the RAW data in the recording medium as it is or in a reversible compressed state.
[0056]
In the above-described embodiment, the storage area 23C in the idle state in the high-speed mode is effectively used as a part of the storage area 24A and used as a buffer area for nonlinear processing data of about 8 bits. However, the present invention is not limited to this. In general, the storage area 23C in an idle state in the high-speed mode may be used as a save area for image data in the process (for example, compressed image data before recording on a memory card or image data being compressed). Even with such a configuration, it is possible to start shooting the next frame without waiting for completion of processing of the image data.
[0057]
Furthermore, in the above-described embodiment, in the original image mode, the output of the white balance correction unit 16 is recorded in the image memory 23 as RAW data. However, the present invention is not limited to this. In general, RAW data faithful to the original image may be used as long as the image data is subjected to irreversible signal processing (reduction of quantization bit number, gradation conversion, pixel thinning, etc.). it can. Therefore, the output of the A / D conversion unit 13, the output of the signal level correction unit 15, or the signal immediately after black level correction may be recorded in the image memory 23 as RAW data.
[0058]
In the above-described embodiment, the γ correction unit 17 that performs irreversible gradation conversion is shown as an example of the second signal processing unit, but the present invention is not limited to this. For example, a signal processing unit that performs irreversible pixel thinning may be used as the second signal processing means.
[0059]
【The invention's effect】
According to the present invention, since the operation control means for dynamically switching the signal path between the first signal processing means and the second signal processing means is newly provided, the signal processing time is shortened in the high-speed mode, and the original picture is displayed. In the mode, RAW data can be used reliably.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of an electronic camera 10 corresponding to the present invention.
FIG. 2 is a diagram illustrating a signal path in a high-speed mode.
FIG. 3 is a diagram illustrating a signal path in an original image mode.
4 is a configuration diagram of a conventional electronic camera 85. FIG.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 10 Electronic camera 11 Image pick-up element 12 Timing generator 13 A / D conversion part 14 Image signal processor 15 Signal level correction part 16 White balance correction part 17 γ correction part 18 Color interpolation part 19 Color difference conversion part 20 JPEG compression part 21 Mode control part 22 CPU
23 Image memory 26 Card interface 27 Memory card 28 Interface 29 Data terminal 30 Mode setting button

Claims (5)

An image sensor;
First signal processing means for performing at least A / D conversion on the video signal generated by the image sensor and converting it to digital image data;
Second signal processing means for performing irreversible signal processing on the image data converted through the first signal processing means;
In an electronic camera equipped with an image memory capable of temporarily storing image data,
In accordance with the operation modes (1) and (2) below, the signal path is switched between the two signal processing means, and RAW data (before irreversible signal processing is performed by the second signal processing means) When the RAW data is operated as unnecessary, the high speed mode (1) is selected and executed. When the RAW data is operated as ( 2. An electronic camera comprising operation control means for selecting and executing the original image mode of 2) .
(1) High-speed mode: A mode in which a series of signal processing is continuously executed by giving the output of the first signal processing means to the second signal processing means and causing the two signal processing means to operate synchronously. (2) Original image mode: The output of the first signal processing means is stored in the image memory, the image data read from the image memory is given to the second signal processing means, and the two signal processing means are individually provided. Mode to operate at timing The electronic camera according to claim 1,
The operation control means includes
The “storage area on the image memory prepared for storing the output of the first signal processing means in the original picture mode” is used as a buffer area for saving image data in the process in the high speed mode. An electronic camera characterized by that. The electronic camera according to claim 1 or 2,
The operation control means includes
An electronic camera characterized in that RAW data stored in an image memory is externally output (or stored in a recording medium). The electronic camera according to any one of claims 1 to 3,
The operation control means includes
An electronic camera characterized in that, in the original picture mode, the operation clock of the second signal processing means is set faster than the operation clock of the first signal processing means. The electronic camera according to any one of claims 1 to 4,
The electronic camera, wherein the second signal processing means is means for performing at least one of “irreversible gradation conversion” and “irreversible pixel thinning”.

Images