JP2004241813A - Imaging apparatus and digital image processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus and a digital image processing method without the need of a line buffer for color interpolation processing. <P>SOLUTION: The imaging apparatus is provided with a digital image processing section (10) for converting an image signal from a CCD (1a), wherein a plurality of color filters are arranged according to the arrangement of providing RGB 3-color signals by two lines, into image data and applies image processing to the digital image data. The digital image processing section (10) includes: an A/D conversion section (2); a color interpolation processing section (3); a horizontal resize processing section (4); and a nonvolatile memory (5). The A/D conversion section (2) converts an image signal outputted from the CCD (1a) into digital image data. The color interpolation processing section (3) performs color interpolation processing on the basis of the digital data by one line from the A/D conversion section (2). The horizontal resize processing section (4) applies reduction processing in a horizontal direction to output data from the color interpolation processing section (3). The nonvolatile memory (5) stores output data from the horizontal resize processing section (4). <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image pickup apparatus and a digital image processing method, and more specifically, an image pickup apparatus that converts an image signal output from an image pickup element in which a plurality of color filters are arranged into digital image data, and the digital image processing thereof Regarding the method.
[0002]
[Prior art]
For example, a digital device including an RGB single-plate solid-state image pickup device (for example, a charge coupled device (CCD) type image pickup device (hereinafter simply referred to as “image pickup device” or “CCD”)) having a 2 × 2 pixel cycle. In an imaging apparatus such as a camera, a digital scanner, etc., since only one of the three primary colors (that is, RGB: red, green, and blue) corresponding to the color of the color filter can be obtained from one pixel in the imaging element, Color generation processing is required for generation.
[0003]
This color interpolation processing has been conventionally performed by a product-sum operation using horizontal and vertical filters (see, for example, Patent Document 1). Since the image signal from the CCD is digitally converted and then output to the image processing system for each horizontal line, in a general color interpolation process, a number of line buffers corresponding to the number of taps in the vertical direction are provided. is necessary.
[0004]
[Patent Document 1]
JP 2000-23174 (page 3-8, Fig. 1)
[0005]
Here, a color interpolation processing operation (that is, a conventional digital image processing method) of a conventional imaging apparatus will be described with reference to FIGS. FIG. 4 is a system block diagram schematically showing a main part of an imaging apparatus such as a conventional digital camera or digital scanner.
[0006]
As shown in FIG. 4, the conventional imaging device converts an image signal from a CCD 11a provided in an imaging unit 11 that images a subject via an optical system device 17 such as a lens into digital image data and performs processing. As a constituent element, a digital image processing unit (image processing circuit) 20 is provided. The digital image processing unit 20 includes an A / D conversion unit 12, a line buffer 13, a color interpolation processing unit 14, a horizontal resize processing unit 15, and a nonvolatile memory (that is, an output memory) 16. ing.
[0007]
The image signal output from the CCD 11 a is converted into digital image data by the A / D converter 12. Digital image data output from the A / D converter 12 is stored in the line buffer 13 line by line. The color interpolation processing unit 14 performs color interpolation processing based on a plurality of lines of image data stored in the line buffer 13.
[0008]
FIG. 5 is a schematic diagram conceptually showing a procedure of color interpolation processing by the digital image processing unit 20 shown in FIG. In this example, since linear interpolation processing is performed using 3 × 3 pixels, the image data is stored in a three-stage line buffer 13 for each line, and color interpolation processing is executed when three lines are prepared. .
[0009]
The image data that has been subjected to the color interpolation processing by the color interpolation processing unit 14 is further subjected to horizontal reduction processing by the horizontal resize processing unit 15. Then, the image data subjected to the reduction processing is written and recorded in the nonvolatile memory 16 and used for further vertical interpolation processing and the like.
[0010]
As described above, the conventional digital image processing unit 20 requires the line buffer 13 that can hold image data for lines necessary for color interpolation processing. Thus, a digital image processing unit that requires a line buffer for color interpolation processing cannot obtain all three colors of RGB with one line of output image signal, such as an image sensor using a so-called Bayer array for a color filter array. It is used in an imaging device provided with a type of imaging device.
[0011]
[Problems to be solved by the invention]
Therefore, for example, in the case of a conventional digital camera provided with the above-mentioned “image pickup element that cannot obtain all three colors of RGB by the output image signal for one line”, the above-mentioned display is performed for through display, postview display, etc. Thus, a digital image processing unit having a line buffer capable of holding image data for lines necessary for color interpolation processing must be used.
[0012]
However, through display, post-view display, and the like are simple image displays for grasping an imaging state with a small image display device (for example, an LCD (that is, Liquid Crystal Display) or the like) disposed in a digital camera. Despite the fact that strict color interpolation processing is not necessary for the generation of the display image, it is necessary to employ a corresponding line buffer with the recent increase in the number of pixels of the image sensor, so the scale of the image processing circuit The increase of unavoidable.
[0013]
The increase in the scale of the image processing circuit is very disadvantageous in reducing the power consumption of the digital image processing unit and realizing high-speed image processing.
[0014]
The present invention has been made in view of the above-described circumstances, and an object thereof is to provide an imaging apparatus and a digital image processing method that can reduce the scale of an image processing circuit without requiring a line buffer for color interpolation processing. There is.
[0015]
[Means for Solving the Problems]
In order to achieve the above-described object, an imaging apparatus according to the present invention includes:
An image pickup apparatus including an image pickup element in which a plurality of color filters are arranged in an arrangement in which image signals of RGB three colors are obtained by two lines,
An A / D converter that converts an image signal output from the image sensor into digital image data;
A color interpolation processing unit that performs color interpolation processing based on the digital image data for one line from the A / D conversion unit;
A horizontal resize processing unit for performing horizontal reduction processing on the output data of the color interpolation processing unit;
A storage unit for storing output data of the horizontal resize processing unit;
It is characterized by having.
[0016]
In order to achieve the above-described object, the digital image processing method of the present invention is as described in claim 2.
A digital image processing method for performing image processing on digital image data based on an image signal from an imaging device in which a plurality of color filters are arranged in an array in which an image signal of RGB three colors is obtained by two lines,
A color interpolation processing step for performing color interpolation processing based on the digital image data for one line;
A horizontal resize processing step for performing horizontal reduction processing on the output data of the color interpolation processing step;
A storage step for storing output data of the horizontal resize processing step;
It is characterized by including.
[0017]
According to the first and second aspects of the present invention, color interpolation processing is performed for each line of image data instead of performing color interpolation processing based on digital image data for a plurality of lines. Since the reduction process is performed and the image data for one line subjected to the color interpolation process and the horizontal reduction process is stored, the line buffer conventionally required for the color interpolation process can be deleted. Therefore, the amount of memory necessary for digital image processing can be reduced, and the scale of the image processing circuit can be reduced. By reducing the scale of the image processing circuit, it is possible to reduce costs, increase the image processing speed by reducing the number of memory accesses, and reduce power consumption.
[0018]
The present invention has been briefly described above. Further, details of the present invention will be further clarified by reading embodiments of the invention described below with reference to the accompanying drawings.
[0019]
DETAILED DESCRIPTION OF THE INVENTION
DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments according to the invention will be described in detail with reference to the drawings. FIG. 1 is a system block diagram schematically showing a main part of an imaging apparatus according to the present invention.
[0020]
As shown in FIG. 1, the imaging apparatus of the present invention converts an image signal from a single-plate CCD 1a provided in an imaging unit 1 that images a subject via an optical system device 6 including a lens or the like into digital image data. A digital image processing unit (image processing circuit) 10 is provided as a component to be converted and processed. The digital image processing unit 10 can be configured by an image processor or the like, and includes an A / D conversion unit 2, a color interpolation processing unit 3, a horizontal resize processing unit 4, and a nonvolatile memory (that is, an output memory) as a storage unit. 5).
[0021]
In the present embodiment, the CCD 1a is a “type in which all three colors of RGB cannot be obtained with an output image signal for one line” using a so-called Bayer color filter array, and the digital image processing unit 10 according to the present invention includes: It is preferable to be used in an imaging device such as a digital camera or a digital scanner having the imaging unit 1 having such a CCD 1a.
[0022]
The image signal output from the CCD 1a is converted into digital image data by the A / D converter 2. The digital image data output from the A / D conversion unit 2 is subjected to color interpolation processing by the color interpolation processing unit 3 for each line, and further, the horizontal resize processing unit 4 performs horizontal reduction processing (that is, Resize processing). Then, the image data subjected to the color interpolation processing and the resizing processing are written and recorded in the nonvolatile memory 5 and used for further vertical interpolation processing and the like.
[0023]
FIG. 2 is a schematic diagram conceptually showing the procedure of color interpolation processing by the digital image processing unit 10 shown in FIG. First, the color interpolation processing unit 3 performs color interpolation processing on R and G of RG image data on the first line, and the image data subjected to the color interpolation processing (that is, RG data for each pixel) is horizontal. The resize processing unit 4 performs the reduction process in the horizontal direction, and the non-volatile memory (that is, the output memory) 5 stores the image data after the reduction process.
[0024]
Next, with respect to the GB image data of the second line, only the B image data is subjected to color interpolation processing by the color interpolation processing unit 3, and the image data subjected to the color interpolation processing (that is, B data for each pixel). ) Is reduced in the horizontal direction by the horizontal resize processing unit 4, and the image data after the reduction processing is stored in a nonvolatile memory (that is, output memory) 5. As described above, the color interpolation processing for the three RGB colors is performed, and the image data reduced in the horizontal direction is written and recorded in the nonvolatile memory 5.
[0025]
FIG. 3 is a timing chart showing an example of processing operation of image data and control signals by the imaging apparatus shown in FIG. First, color interpolation processing is enabled for the image data of the first line, and R and G image data subjected to color interpolation processing and reduction processing are written in the output memory. Next, color interpolation processing is enabled for B in the image data of the second line, and B image data subjected to color interpolation processing and reduction processing is written to the output memory. Then, when the output of the RGB image data is enabled, it is output to a subsequent processing unit (not shown). A subsequent processing unit (not shown) performs image reduction processing such as vertical interpolation processing of image data. These series of controls are realized by control means (not shown) represented by a CPU (that is, Central Processing Unit).
[0026]
For example, image data of 2000 pixels per line is subjected to horizontal reduction processing to be 640 pixels in the case of VGA display. Conventionally, a plurality of stages of line buffers for 2000 pixels are required, but according to the present embodiment, they can be reduced, and the output memory has a small size (640 pixels in the case of the above example) that matches the display. The scale of the image processing circuit can be reduced.
[0027]
In a conventional imaging apparatus, color interpolation processing is performed after the image data is held in the line buffer and three RGB colors are obtained. In this embodiment, the image data is subjected to color interpolation processing for each line and the horizontal direction is obtained. After executing the reduction processing, it is recorded in the output memory, so that it is possible to delete a line buffer that has been conventionally required. The color interpolation process can use a known technique such as a linear interpolation process, and the horizontal reduction process can use a known technique such as data thinning.
[0028]
In the above description, the combination of a single-plate image pickup device using a so-called Bayer color filter array and the digital image processing unit according to the present invention has been described. However, a single RGB image can be obtained with two lines of output image signals. Needless to say, the digital image processing unit according to the present invention is suitable for combination with a plate-type imaging device. In the case of a digital camera equipped with such an image sensor, a small image display device (for example, an LCD (ie, LCD (ie, LCD)), such as a simple image display such as a through display or a post-view display. If the digital image processing unit according to the present invention performs display image processing that requires an image quality sufficient to grasp the imaging state with a liquid crystal display (Liquid Crystal Display, etc.), an excellent effect can be obtained.
[0029]
【The invention's effect】
As described above, according to the present invention, color interpolation processing is not performed based on digital image data for a plurality of lines, but color interpolation processing is performed for each line of image data, and further reduction processing is performed. Are stored for each line of image data for which the color interpolation process and the horizontal reduction process have been performed, so that a line buffer conventionally required for the color interpolation process can be deleted. Therefore, the amount of memory necessary for digital image processing can be reduced, and the scale of the image processing circuit can be reduced. By reducing the scale of the image processing circuit, it is possible to reduce costs, improve the image processing speed by reducing the number of memory accesses, and reduce power consumption.
[Brief description of the drawings]
FIG. 1 is a system block diagram schematically showing a main part of an imaging apparatus according to the present invention.
FIG. 2 is a schematic diagram conceptually showing a procedure of color interpolation processing by the digital image processing unit shown in FIG. 1;
FIG. 3 is a timing chart showing an example of processing operation of image data and control signals by the imaging apparatus shown in FIG. 1;
FIG. 4 is a system block diagram schematically illustrating a main part of a conventional imaging apparatus.
5 is a schematic diagram conceptually showing a procedure of color interpolation processing by the conventional digital image processing unit shown in FIG.
[Explanation of symbols]
1 Imaging unit 1a CCD
2 A / D conversion unit 3 Color interpolation processing unit 4 Horizontal resize processing unit 5 Non-volatile memory (storage unit)
6 Optical system device 10 Digital image processing unit

Claims (2)

An image pickup apparatus including an image pickup element in which a plurality of color filters are arranged in an array in which image signals of RGB three colors are obtained by two lines
An A / D converter that converts an image signal output from the image sensor into digital image data;
A color interpolation processing unit that performs color interpolation processing based on the digital image data for one line from the A / D conversion unit;
A horizontal resize processing unit for performing horizontal reduction processing on the output data of the color interpolation processing unit;
A storage unit for storing output data of the horizontal resize processing unit;
An imaging apparatus comprising: A digital image processing method for performing image processing on digital image data based on an image signal from an image pickup device in which a plurality of color filters are arranged in an array in which RGB three-color image signals are obtained by two lines,
A color interpolation processing step for performing color interpolation processing based on the digital image data for one line;
A horizontal resize processing step for performing horizontal reduction processing on the output data of the color interpolation processing step;
A storage step for storing output data of the horizontal resize processing step;
A digital image processing method.

Images