JP3954204B2 - Signal processing apparatus and signal processing method thereof - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a signal processing apparatus and a signal processing method for generating display data by thinning out signals from pixels arranged corresponding to each color filter and performing signal processing on data for a supplied signal, and in particular, a color processing method thereof. A Bayer pattern is used for the filter array, which is suitable for use when, for example, a still image captured with hundreds of thousands to millions of pixels and a movie image are switched and displayed in an electronic still camera, a movie camera, or the like. Is.
[0002]
[Prior art]
For example, an electronic still camera, a video camera, or the like (hereinafter, representatively referred to as an electronic still camera) photoelectrically converts light incident from an object scene in a pixel cell including, for example, a solid-state imaging device. . In particular, when a single-plate color filter is used immediately before the image pickup unit, the image pickup unit obtains a signal of only one of the three primary colors R, G, and B corresponding to the color of the color filter from each pixel. I can't. And the image imaged using the signal of each obtained color is displayed on the display part arrange | positioned in the electronic still camera etc. which were mentioned above. In general, a large-sized liquid crystal display or the like is used as the display unit when emphasizing good visibility, and a small liquid crystal display or the like is used as a display device such as an index image or a movie image when power consumption is important. Used as
[0003]
When this small liquid crystal display is used for the display unit, the number of pixels of the display unit is smaller than that of the imaging unit. Therefore, the electronic still camera thins out the obtained signal and supplies it to the display unit. Usually, a Bayer pattern is often used for the color filter array. Since two lines are used to obtain RGB from the characteristics of the color filter array, a line memory is used.
[0004]
In general, there is a concern about resolution degradation due to the thinning of the signal. Therefore, even if the signal is thinned out, for example, the solid-state imaging device disclosed in Japanese Patent Publication No. 8-2587225 reads the horizontal line signal every 1 / P horizontal period by the image sensor driving means and writes it to a plurality of memory units. The horizontal resolution is artificially improved by simultaneously reading out the data of the P horizontal line every horizontal period by the memory driving means and generating the signal by the synthesizing means.
[0005]
In addition, when a Bayer array is used for the color filter array, a color reproducibility in which a false signal is generated becomes a problem because a luminance signal having a correct spectral characteristic cannot be obtained by a line from which the signal is extracted regardless of signal thinning. There is a case. A proposal of a color image pickup apparatus that pays attention to improvement of such false signals is described in, for example, Japanese Patent Laid-Open No. 4-329786. This color image pickup apparatus uses a color filter array in which only the first color filter has an offset sampling structure and the other second and third color filters have a rectangular lattice sampling structure, and the first image filter is used to synchronize signals. The color difference difference between the signal from the pixel of the first color filter and the signal from the second color filter in the same row or column as the pixel of this color filter is taken, and similarly, the signal from the pixel of the first color filter The color difference between the pixel of this color filter and the signal from the third color filter in the same row or the same column is taken, and these color difference signals are subjected to, for example, matrix synthesis and calculation to generate a luminance signal, and spectral characteristics Correction is performed. In solid-state imaging devices using a Bayer array color filter array, attempts have been made to ensure color reproducibility by using various methods in addition to the signal processing described above.
[0006]
[Problems to be solved by the invention]
By the way, as described above, when an image is generated by thinning out signals from a solid-state imaging device using a Bayer array in the horizontal (H) direction, not only is the resolution difficult to obtain, but false colors generated by the image may increase. is there. Further, in the signal processing performed for improving the false color and the image quality, the readout pixels are increased in total by reading out the extracted signals. As a result, this signal processing results in an increase in circuit configuration and an increase in power consumption.
[0007]
Furthermore, when a movie image is displayed using a signal read from a solid-state imaging device that requires several million pixels, thinning processing becomes important. In this case, the high frequency component Y of the luminance signal from the information of all pixels_HIf the thinning process is performed after the pixel is generated corresponding to the pixel, it takes a lot of time.
[0008]
The present invention eliminates the disadvantages of the prior art, performs a pixel thinning process with a simple configuration, suppresses the generation of false colors, and can perform signal processing corresponding to moving image reproduction, and An object of the present invention is to provide a signal processing method.
[0009]
[Means for Solving the Problems]
In order to solve the above-described problems, the present invention provides first, second, and third color filters having different spectral sensitivity characteristics arranged in a Bayer pattern and supplied from pixels arranged corresponding to each color filter. A signal processing device that generates display data by performing signal processing on data for a signal to be processed, and handles a plurality of pixels as one block in a horizontal direction with respect to pixels for a plurality of lines corresponding to a Bayer pattern The block sampling means for sampling the data supplied from each pixel of this block corresponding to each pixel position, and the first and second data in each block using the data of a plurality of pixels obtained by this block sampling means The pixel values of the second and third colors are calculated, and the luminance signal, the color difference signal, or the three primary color signals are displayed on the display data with the calculated pixel values. Characterized by comprising a display data generating means for generating, and control means for controlling the timing of the sampling of the block sampling means as.
[0010]
Here, the block sampling means includes a block dividing means for horizontally dividing a condition that includes at least one of the first, second, and third colors in the block, and an inside of the block divided by the block dividing means. And data extraction means for extracting only data supplied from the pixels. As a result, it becomes possible to easily handle the target of data to be subjected to signal processing.
[0011]
It is desirable that the block dividing means handle the pixels for two lines corresponding to the Bayer pattern collectively in units of two pixels in the horizontal direction, and form a block with a total of four pixels. This defines a block for the Bayer basic pattern.
[0012]
The block sampling means includes a block dividing means for horizontally dividing a condition that includes at least one of the first, second, and third colors in the block, and a block divided by the block dividing means. The block thinning means that thins out every other horizontal direction, and the pixels that are arranged in the column direction located at both the left and right ends in the block thinned out by this block thinning means and the pixels of the blocks adjacent to each side meet the new condition It is advantageous to include re-blocking means for forming the block, and block data extracting means for extracting data from each block in order by making the block of the re-blocking means correspond to the block at the thinning position. In this case, the signal width in each pixel is widened by thinning, and the sampling frequency can be reduced to half that of the conventional frequency.
[0013]
In particular, for example, when a signal is read from a solid-state imaging device having hundreds of thousands to millions of pixels, the control unit outputs the sampling relationship of the block sampling unit and the signal from the pixel in the same relationship. It is preferable to control drive signal generation means for generating a drive timing signal. Thereby, in the case of recording and displaying a signal, for example, the driving signal from the driving signal generating means can be switched and operated in driving the solid-state imaging device. In particular, this pixel decimation can be easily realized by control from the control means in accordance with switching of the movie mode that requires high-speed processing. As a result of this driving, the supplied signal is already a signal corresponding to a block or a signal corresponding to block thinning, so that the configuration of each part can be omitted.
[0014]
The signal processing apparatus of the present invention divides the color filter pattern in the horizontal direction by handling a plurality of pixels as one block in the pixels of the Bayer pattern by the block sampling means in accordance with the control of the control means. The block sampling means samples data supplied from each pixel of this block used for display. By sampling at each pixel position corresponding to this block, the sampling frequency is halved compared to the conventional sampling frequency. The pixel value calculation means calculates the pixel values of the first, second, and third colors for each pixel using the data of the plurality of pixels obtained by the block sampling means. By calculating using signals from the pixels in the block, the pixels are halved when viewed in the horizontal direction. That is, the data handled is halved. When display data is calculated by the display data generation means using the calculated pixels, not only the vertical false color peculiar to the Bayer pattern but also the horizontal false color is suppressed by taking the average of the two diagonal pixels. As a result, the color reproducibility can be improved compared to the conventional image.
[0015]
Further, according to the present invention, first, second and third color filters having different spectral sensitivity characteristics are arranged in a Bayer pattern, and data for signals supplied from pixels arranged corresponding to the respective color filters is used. A signal processing method of a signal processing device that performs display processing to generate signal data, and handles a plurality of pixels as one block in a horizontal direction with respect to pixels for a plurality of lines corresponding to a Bayer pattern, and A block sampling process for sampling data supplied from each pixel of the block corresponding to each pixel position, and the first, second and second data in each block using data of a plurality of pixels obtained in this block sampling process A pixel value calculating step for calculating a pixel value of the third color, and a luminance signal and a color difference signal or a pixel value calculated by the pixel value calculated in the pixel value calculating step; A display data generation step of generating a primary color signal as the display data, and the block sampling step performs sampling of the data based on a timing signal from a timing control step of generating a timing signal of data sampling timing. Features.
[0016]
Here, the blocks are preferably handled in units of two pixels in the horizontal direction with respect to the pixels for two lines corresponding to the Bayer pattern, and the block is preferably formed by a total of four pixels. Among various Bayer patterns, the configuration of blocks handled in a basic Bayer pattern is defined.
[0017]
The block sampling process includes a block dividing process for horizontally dividing a condition that includes at least one of the first, second, and third colors in the block, and pixels in the block divided in the block dividing process. It is desirable to include a data extraction step for extracting only supplied data. When sampling is performed by paying attention to each pixel position in each block by dividing into blocks, every other sampling in the horizontal direction is performed in sampling, so the sampling frequency is halved compared to the conventional sampling frequency.
[0018]
Further, the block sampling process includes a block dividing process for horizontally dividing a condition that includes at least one of the first, second, and third colors in the block, and a block divided by the block dividing process. A block thinning step that thins out every other horizontal direction, and a block that is thinned out in this block thinning step and the pixels arranged in the column direction located at both the left and right ends and the pixels of the block adjacent to each side meet the new condition. And a block data extraction step of extracting data from each block in order by making the block of the reblocking step correspond to the block at the thinning position. In this case, the sampling frequency can be reduced to 1/4 compared with the conventional sampling frequency.
[0019]
The timing control process advantageously controls the generation of a drive timing signal from which signals from the pixels are obtained in the same relationship as the sampling relationship of the block sampling process. By this driving, the supplied signal is already a signal corresponding to a block or a signal corresponding to block thinning, so that the processing steps of each part can be omitted.
[0020]
According to the signal processing method of the signal processing device of the present invention, block division is performed in which the pixels of the Bayer pattern are divided into a plurality of pixels in the horizontal direction and divided into a plurality of pixels in the block sampling process according to the timing signal of the timing control process. The data supplied from each pixel of this block is subjected to sampling processing corresponding to the pixel position. By performing sampling corresponding to each pixel position in the block, the sampling frequency can be reduced to half of the conventional sampling frequency. In the pixel value calculating step, the pixel values of the first, second, and third colors in this block are calculated using the data of the plurality of pixels obtained from the block sampling step. Pixels of the calculated pixel value exist every other pixel in the horizontal direction in the block. When the display data is calculated in the display data generation process using the calculated pixels, the number of pixels in the horizontal direction is halved and not only the vertical false color generated in the image captured by the Bayer pattern but also the horizontal direction False colors and the like can also be suppressed by averaging two diagonal pixels to improve color reproducibility.
[0021]
DETAILED DESCRIPTION OF THE INVENTION
Next, embodiments of a signal processing apparatus and a signal processing method according to the present invention will be described in detail with reference to the accompanying drawings.
[0022]
The signal processing apparatus according to the present invention performs sampling by paying attention to the pixel position in the block, keeps the sampling period longer than the conventional frequency, keeps the frequency to be used low, and generates vertical and horizontal directions in an image captured by the Bayer pattern. It is characterized in that the color reproducibility is improved by suppressing the false color and the like. In this embodiment, the case where the present invention is applied to an electronic still camera will be described with reference to FIGS.
[0023]
As shown in FIG. 1, the electronic still camera 10 includes a camera main body 11 and a memory card unit 20, both of which are integrally connected via a connector 30. The camera body 11 includes an optical system 11a, a color filter array 11b, a CCD image sensor 11c, a preprocessing unit 11d, an A / D conversion unit 11e, a signal processing unit 12, a data compression unit 11f, an input / output interface unit 11g, a data input The memory card unit 20 includes an input / output interface unit 21 and a memory card 22. The unit 11h includes a display unit 11i and a system control unit 13.
[0024]
The optical system 11a includes a photographing lens 110a, a diaphragm (not shown), and the like. The color filter array 11b is a single plate type as described above. The arrangement of this single-plate color filter array will be described with the three primary colors R, G, and B. In this embodiment, the electronic still camera 10 uses a basic Bayer pattern for the arrangement of the color filter array.
[0025]
A case where this pattern is applied will be described. This color filter array 11b is disposed immediately before the imaging surface of the CCD image sensor 11c.
[0026]
In the CCD image sensor 11c, cells of the CCD image sensor 11c corresponding to the elements of the color filter array 11b are arranged in an array as pixels. When incident light from the object field forms an image in this cell, the incident light is photoelectrically converted in this cell. The signal reading is preferably performed by the two-line simultaneous reading method so that the signal processing of the signal processing unit 12 can be performed efficiently.
[0027]
The preprocessing unit 11d amplifies the supplied signal to a desired level and outputs it to the A / D conversion unit 11e. The A / D changing unit 11e converts the analog signal output from the preprocessing unit 11d into a digital signal.
[0028]
As shown in FIG. 2, the signal processing unit 12 includes a still signal processing unit 12A and a movie signal processing unit 12B. In the still signal processing unit 12A, signal processing is performed when displaying a still image with an emphasis on resolution.
[0029]
Although not shown, for example, when there are 1280 pixels in the horizontal direction, the still signal processing unit 12A inputs a signal corresponding to the number of pixels, performs signal processing without losing the number of pixels, and outputs the signal. Output. The movie signal processing unit 12B is provided with a block sampling unit 12a and a display data generation unit 12b. Further, the block sampling unit 12a includes a block dividing unit 120a and a data extracting unit 122a.
[0030]
In the case of signal processing for a basic Bayer pattern, the block dividing unit 12a performs processing so that, for example, pixels for two lines can be regarded as a block in units of two pixels in the horizontal direction. Therefore, in this embodiment, one block is composed of a total of four pixels. The block division unit 12a performs block memory management of pixel data for two lines by dividing the horizontal direction with a counter and assigning block numbers, for example. Further, when two lines are read simultaneously and a representative value described later is calculated, display data is obtained every other line in the vertical direction, and the vertical resolution is lowered. In order to be able to calculate the representative value for each horizontal line even if two lines are read simultaneously, the block memory is divided into blocks while controlling the pixel data read and pixel data write timing for each line using a line memory for four lines. Pixel data is output to the data extraction unit 122a. When the representative value is calculated for each line by supplying the pixel data, an image can be displayed on the display unit 11i without reducing the vertical resolution.
[0031]
Further, a configuration may be adopted in which division processing is simply performed by a plurality of first-in first-out (FIFO) memories that collectively handle four pixels. By this division, processing for each block can be facilitated as will be described later.
[0032]
The data extraction unit 122a extracts a signal corresponding to each pixel position in the block. Each pixel position is the block position P₀₀, P₀₁, P_Ten, P₁₁Sampling is performed corresponding to the four positions. Therefore, although not shown, the data extraction unit 122a has four sample and hold circuits. By sampling with a fixed pixel position, every other pixel is sampled at the same pixel position in an adjacent block (for example, G₀₀, G₀₂, G₀₄,...), When sampling each pixel one by one (for example, G₀₀, R₀₁, G₀₂, R_03,..)), The sampling frequency can be reduced to half the frequency. The sampled data is supplied to the representative value calculation unit 120b of the display data generation unit 12b.
[0033]
The display data generation unit 12b includes a representative value calculation unit 120b, a YC conversion unit 122b, and an output selection unit 124b. The representative value calculation unit 120b defines any one pixel in the block as the representative pixel of the block, and calculates the RGB pixel value of this pixel based on this definition. In order to perform this calculation, the representative value calculation unit 120b includes an arithmetic circuit including at least an adder and a 1/2 multiplier or an adder and a divider. Further, the YC converter 122b receives the luminance signal Y and the color difference signal C._b, C_rIs calculated. For this calculation, there are a multiplier (not shown) storing three coefficients for RGB and an adder for adding the multiplication results. Color difference signal C_b, C_rEach have one adder for calculation. Color difference signal C_bIn this case, the adder adds and inputs the B pixel value calculated by the representative value calculation unit 120b from one side, and subtracts and inputs the luminance signal Y calculated as described above from the other side, and outputs it. Ask for. Similarly color difference signal C_rIs also calculated.
[0034]
The display data generation unit 12b includes a representative value calculation unit 120b and a YC conversion unit 122b, so that it can cope with RGB output and YC output. However, the display unit 11i displays in either one of the output formats. Accordingly, the mode and the like are set for the user to display on the system control unit 13 via the data input unit 11h. The system control unit 13 supplies a control signal reflecting the user's intention to the output selection unit 124b. The output selection unit 124b selects and outputs a desired output format under the control of the system control unit 13. The output selection unit 124b is naturally unnecessary when only the representative value calculation unit 120b is provided. This is because the YC conversion process requires the representative value calculation unit 120b and the YC conversion unit 122b.
[0035]
Note that the YC converted signal is corrected by aperture correction, which is one of the signal adjustments in the YC conversion unit 122b, so that, for example, effects such as contour enhancement and coring are obtained, and saturation is improved. Color gain adjustment may be performed so that
[0036]
The electronic still camera 10 will be briefly described with respect to the configuration of the main body 11 used in the recording mode. The data compression unit 11f performs compression processing for reducing the number of bits of the luminance data Y, color difference data (B-Y), and (R-Y) supplied from the signal processing unit 12. For example, Huffman coding or differential PCM is used as a data compression method. As a result, the data compression unit 11f performs processing for reducing the amount of data and outputs it to the input / output interface unit 11g.
[0037]
Although not shown, the system control unit 13 includes a system controller that controls the electronic still camera 10, a timing signal generation unit, and an address control unit. The system controller also controls a timing signal generation unit and an address control unit to control the electronic still camera 10. The timing signal generator generates a signal for adjusting the driving and operation timing of the electronic still camera 10 and a control signal, and supplies them to each unit as shown in FIG. The address control unit is advantageous for use in controlling the buffer memory unit 12a in which the address is involved in the signal processing unit 12. These controls also control the data and address of the memory card 22 of the memory card unit 20 via the input / output interface unit 11g, the connector 30, and the input / output interface unit (I / F) 21. In particular, the system controller 13 uses the control signal 11n to drive the CCD image sensor 11c so that signals from the pixels can be obtained in the same output format as the pixel data sampling process in the signal processor 12. The signal generator 14 is controlled (see FIG. 2). The drive signal generator 14 outputs to the CCD image sensor 11c a drive signal for two-line simultaneous readout for updating the signal from the pixel line by line.
[0038]
The data input unit 11h sends the mode selection signal operated by the user, setting of a shutter / release button (both not shown), and information on pressing instructions to the system control unit 13 as a mode selection signal 11m. The modes operated by the user include still shooting, movie shooting, RGB output, YC output, and the like, and a desired mode can be selectively performed on the electronic still camera 10 from these modes.
[0039]
Next, the operation of the electronic still camera 10 will be briefly described. Signals obtained from each pixel of the CCD image sensor 11c are obtained simultaneously, for example, for two lines corresponding to the Bayer pattern color filter array 11b. Since the red and blue filters corresponding to the second color and the third color are only on one side, respectively, it is known that two lines are required for the calculation of RGB.
[0040]
When the signal of each pixel is read out in the horizontal direction in the movie mode, the signal processing unit 12 blocks the pixel unit by the block dividing unit 120a. The data extraction unit 122a uses the pixel position P₀₀, P_TenFor every other signal, pixel readout is performed. As a result, sampling is conventionally performed at the same frequency as the clock signal, but sampling is performed at half the clock signal.
[0041]
The data RGB obtained by sampling is supplied to the representative value calculation unit 120b. The representative value calculation unit 120b calculates the pixel position (P₀₀, P₀₁, P_Ten, P₁₁), For example, pixel position P₀₀ PR of the three primary colors RGB₀₀, PG₀₀, PB₀₀Is calculated. This calculation is representative value PR₀₀, PG₀₀, PB₀₀Substituting the pixel value of the color at each pixel position and the calculation using the pixel value with respect to (1)
[0042]
[Expression 1]
PR₀₀= P₀₁
PG₀₀= (P₀₀+ P₁₁)/twenty one)
PB₀₀= P_Ten
To do. Specifically, when calculating by putting the pixel value in equation (1),
[0043]
[Expression 2]
PR₀₀= R₀₁
PG₀₀= (G₀₀+ G₁₁)/twenty two)
PB₀₀= B_Ten
Is obtained.
[0044]
Using this calculated representative value, the luminance signal PY at the representative position₀₀And color difference signal PC_r00, PC_b00Is calculated by the YC converter 122b. This calculation is calculated using equation (3)
[0045]
[Equation 3]
PY₀₀= 0.3 * PR₀₀+ 0.59 * PG₀₀+ 0.11 * PB₀₀
C_r00= PR₀₀-PY₀₀                                               ... (3)
C_b00= PB₀₀-PY₀₀
Obtained by. This representative position corresponds to every other pixel when the pixels of the color filter array are viewed in the horizontal direction.
[0046]
The supplied pixel data is temporarily stored in a memory for a plurality of lines, and a representative value is calculated for each line based on pixel data obtained by performing writing and reading control on these memories. After calculating the representative value related to the line, when calculating the representative value of the line immediately below the line, for example, the pixel position (P_Ten, P₁₁, P₂₀, P_{twenty one}) As a new block and the representative position is P₁₁ Set to. For example, pixel position P₁₁ PR of the three primary colors RGB₁₁, PG₁₁, PB₁₁Is calculated. This calculation is representative value PR₁₁, PG₁₁, PB₁₁Substituting the pixel value of the color at each pixel position and the calculation using the pixel value with respect to
[0047]
[Expression 4]
PR₁₁= P_{twenty one}
PG₁₁= (P₁₁+ P₂₀)/twenty four)
PB₁₁= P_Ten
To do. Specifically, when calculating by putting the pixel value in equation (4),
[0048]
[Equation 5]
PR₁₁= R_{twenty one}
PG₁₁= (G₁₁+ G₂₀)/twenty five)
PB₁₁= B_Ten
Is obtained.
[0049]
Using this calculated representative value, the luminance signal PY at the representative position₁₁And color difference signal PC_r11, PC_b11Is calculated by the YC converter 122b. This calculation is calculated using equation (6)
[0050]
[Formula 6]
PY₁₁= 0.3 * PR₁₁+ 0.59 * PG₁₁+ 0.11 * PB₁₁
C_r11= PR₁₁-PY₁₁                                               ... (6)
C_b11= PB₁₁-PY₁₁
Obtained by. The display data (that is, pixel data) calculated in this way is temporarily stored in a memory (not shown) but then displayed on the display unit 11i under the control of the system control unit 13. At this time, the memory capacity is half that of the memory capacity when all the pixels are displayed. Further, when the thinning process is performed line by line in the vertical direction and displayed, the processing of equations (4) to (6) becomes unnecessary.
[0051]
Next, a modification of this embodiment of the electronic still camera 10 will be described with reference to the block diagram of FIG. 3 and the timing chart of FIG. In this modification, the configuration of the block sample unit 12a is partially changed. In this case, the block sample unit 12a includes a block dividing unit 120a, a block thinning unit 124a, a block generating unit 126a, and a block correspondence extracting unit 128a as shown in FIG.
[0052]
Since the block division unit 120a uses the same configuration as that of the above-described embodiment, detailed description thereof is omitted. The block thinning unit 124a thins out every other read block. In the case of processing in correspondence with blocks, the thinning-out process may be configured to handle every other block as described above. Also, the thinning process is performed by switching between the counter and the selection switch for each pixel so as to repeat the process of outputting two pixels and not outputting the next two pixels when thinning is performed while paying attention to the pixels in the horizontal direction. May be. The block thinning unit 124a supplies the processed pixel data to the block generation unit 126a and the block correspondence extraction unit 128a, respectively.
[0053]
The block generation unit 126a generates a new block different from the skipped block using the pixels of the block remaining after the thinning process. Basically, the components of the block to be generated are the same configuration as the remaining blocks, and a total of four pixels. This newly generated block is, for example, a block (P₀₀, P₀₁, P_Ten, P₁₁) And block (P₀₄, P₀₅, P₁₄, P₁₅) Are generated from the remaining two blocks. When the generated block is expressed using each pixel position of the remaining block, the element is (P₀₁, P₀₄, P₁₁, P₁₄). When this new block is expressed by taking a part of the arrangement of the color filter array 11b shown in FIG. 4A as an example, the configuration of the pixel of the block is specifically (R₀₁, G₀₄, G₁₁, B₁₄)
[0054]
Since it is necessary to select and take out the constituent elements in this way, the block generation unit 126a arranges the memory for temporarily storing each block and the block adjacent to each other in the horizontal direction (not shown), and sets two pixels in the column direction once. Is provided with a selection unit for performing selection to be thinned out. This memory contains elements of 2 rows and 4 columns formally by storing 2 blocks. The selection unit thins out two rows and one column located at both ends of these elements. In other words, it can be said that only the elements of 2 rows and 2 columns located at the center are extracted. The block generation unit 126a outputs the generated block data to the block correspondence extraction unit 128a.
[0055]
The block correspondence extraction unit 128a extracts pixel data while placing importance on the pixel position. At this time, the block correspondence extracting unit 128a performs data extraction for a block remaining after block thinning and data extraction for a newly generated block. Considering these two types of blocks, for example, eight sample hold circuits are provided and data extraction is performed at different timings. The sampling timing in the other type of block is shifted by 3/4 period from the sampling timing in one type of block.
[0056]
Each pixel data obtained in this way is supplied to the representative value calculation unit 120b of the display data generation unit 12b. The configuration used for calculation is basically the same. However, the pixel values assigned to the equations are different. This point will be described in detail in the subsequent operation description. The newly generated block is used for calculating the representative value at the representative position for the block thinned out by the representative value calculation unit 120b.
[0057]
Next, the operation of the modified example of the electronic still camera 10 to which the present invention is applied will be briefly described with reference to FIG. The color filter array 11b is a basic Bayer pattern and is arranged as shown in FIG. Although not shown, the CCD image sensor 11c reads two lines from the horizontal transfer path, for example, at the rising edge of the clock signal shown in FIG. The read pixel signals are read for each line in the relationship shown in FIG. The thinning process is performed after the block division process. Further, when attention is paid to the pixel position of the block, a signal (that is, pixel data) shown in FIG. 4D is obtained. Since the block surrounded by the broken line in the pattern of the color filter array 11b in FIG. 4 (a) is shown, these signals are respectively P₀₀ G at the position corresponding to₀₀, G_04, G₀₈, G_012,..., P₀₁ R at the position corresponding to₀₁, R_05, R₀₉, R_013,..., P_Ten B at the position corresponding to_Ten, B_14, B₁₈, B_112,..., P₁₁ G at the position corresponding to₁₁, G_15, G₁₉, G_111,Is output in the relationship
[0058]
In consideration of sampling at a time when a signal from a pixel that is an element of each remaining block is read out, for example, the data extraction is performed at time T for the first block.₁become. Data extraction for the next horizontal block is time T₂become. The period during this period is four times longer than the period shown in FIG. 4B, which is the same as the conventional sampling period for each pixel. That is, the sampling frequency is 1/4. Based on the pixel data supplied in this way, luminance signal Y, color difference signal C_r, C_bIs calculated, a representative value of each block is generated. FIG. 4F shows a state where these signals are represented in one time domain for convenience. The calculation of the representative value in this case is the same as in the above-described embodiment.
[0059]
On the other hand, a newly generated block is called a reblock because it is a reconstructed block. If the representative value of the thinned block is calculated using this re-block, it is not possible to extract all the pixel values at once. However, the fastest possible pixel value extraction is the pixel G₀₄, B₁₄I.e., time T_ThreeIt is. Time T_ThreeThus, the pixel data being re-blocked is extracted. Representative position P of the block thinned out using this pixel data₀₂ The representative value of is calculated. That is, G in Fig. 4 (a)₀₂ Is a position corresponding to. Representative position P₀₂ The representative value of Eq. (7)
[0060]
[Expression 7]
PR₀₂= P₀₁
PG₀₂= (P₁₁+ P₀₄) / 2 (7)
PB₀₂= P₁₄
Is calculated using Specifically, when calculating by putting the pixel value in equation (7),
[0061]
[Equation 8]
PR₀₂= R₀₁
PG₀₂= (G₁₁+ G₀₄) / 2 (8)
PB₀₂= B₁₄
Is obtained.
[0062]
Using this calculated representative value, the luminance signal PY at the representative position₀₂And color difference signal PC_r02, PC_b02Is calculated by the YC converter 122b. This calculation is based on the assumption that the representative value assigned to Equation (6) is P₀₂ become.
[0063]
[Equation 9]
PY₀₂= 0.3 * PR₀₂+ 0.59 * PG₀₂+ 0.11 * PB₀₂
C_r02= PR₀₂-PY₀₂                                               ... (9)
C_b02= PB₀₂-PY₀₂
Obtained by. In this way, the data of the blocks that are thinned out from the remaining blocks despite the thinning out is calculated. As a result, sampling can be done at 1/4 the sampling frequency compared with the case where all pixels are read out in the horizontal direction. Further, when the display data (that is, pixel data) is stored once in the memory before being supplied to the display unit 11i as in the previous embodiment, the memory capacity at this time is calculated for all conventional pixels. Naturally, it can be halved compared to the case.
[0064]
When this display data is displayed on the display unit 11i, the occurrence of a false color that becomes a problem in the Bayer pattern is calculated by the above-described arithmetic processing, and thus can be reduced. Thus, display data can be provided so that even a display unit 11i that displays a small index image such as a so-called thumbnail image can be sufficiently viewed while suppressing a decrease in vertical resolution. Become.
[0065]
Also in this modification, the system controller 13 controls the operation of the drive signal generator 14 with the control signal 11n according to the mode selection. The drive signal generator 14 supplies a drive signal supplied as a timing signal to the CCD image sensor 11b. At this time, the drive signal is a signal that causes the CCD image sensor 11b to simultaneously read out the two lines thinned out in this block. With this drive signal, the CCD image sensor 11b can supply a signal equivalent to that obtained by the block division to the signal processing unit 12 via each unit. By this driving, the signal processing configuration can be omitted.
[0066]
With this configuration, it is possible to improve the false color generated in the Bayer pattern without impairing the vertical resolution. In addition, since the operation can be performed while suppressing the sampling frequency, the operation in the movie mode (that is, the moving image reproduction) can be performed with low power consumption. Therefore, when applied to a battery-driven device, the circuit configuration can be realized with a simple configuration, and is very effective from the viewpoint of power consumption. This effect can be further enhanced by performing thinning.
[0067]
In the embodiment described above, the CCD image sensor is used as the image data supply means. However, the present invention is not limited to this means. For example, a C-MOS (Complementary-Metal Oxide Semiconductor) type imaging means is used. It goes without saying that the same effect as described above can be obtained.
[0068]
【The invention's effect】
As described above, according to the signal processing apparatus of the present invention, the color filter pattern is divided in the horizontal direction by treating a plurality of pixels as one block in the pixels of the Bayer pattern by the block sampling unit according to the control of the control unit, The block sampling means samples data supplied from each pixel of this block used for display. By sampling at each pixel position corresponding to this block, the sampling frequency is halved compared to the conventional sampling frequency. The pixel value calculation means calculates the pixel values of the first, second, and third colors for each pixel using the data of the plurality of pixels obtained by the block sampling means. By calculating using signals from the pixels in the block, the pixels are halved when viewed in the horizontal direction. When display data is calculated by the display data generation means using the calculated pixels, not only false colors generated in the vertical direction peculiar to Bayer patterns but also false colors in the horizontal direction are suppressed by the average of two diagonal pixels. be able to. Thereby, color reproducibility can be improved from the conventional image. As described above, the operation in the movie mode (that is, the moving image reproduction) can be performed with low power consumption by operating with the sampling frequency being suppressed. When applied to a battery-driven device, the circuit configuration can be realized with a simple configuration and is very effective from the viewpoint of power consumption. This effect can be further enhanced by performing thinning.
[0069]
Further, according to the signal processing method of the signal processing device of the present invention, according to the timing signal to be supplied, block division is performed in which the pixels of the Bayer pattern are divided into a plurality of lines in the horizontal direction and divided into a plurality of pixels and treated as blocks. Data supplied from each pixel of this block is subjected to sampling processing corresponding to the pixel position. By performing sampling corresponding to each pixel position in the block, the sampling frequency can be reduced to half of the conventional sampling frequency. The pixel value is calculated by calculating the pixel values of the first, second, and third colors in this block using data of a plurality of pixels obtained by sampling. Pixels of the calculated pixel value exist every other pixel in the horizontal direction in the block. When the display data is calculated using the calculated pixels, the number of pixels in the horizontal direction is halved and not only the false color in the vertical direction generated in the image captured by the Bayer pattern but also the false color in the horizontal direction, etc. The color reproducibility can be improved by suppressing the average of two diagonal pixels. By operating at a reduced sampling frequency, the operation in the movie mode (that is, moving image reproduction) can be performed with low power consumption. When applied to a battery-driven device, the circuit configuration can be realized with a simple configuration and is very effective from the viewpoint of power consumption. This effect can be further enhanced by performing thinning.
[Brief description of the drawings]
FIG. 1 is a schematic block diagram when a signal processing apparatus of the present invention is applied to an electronic still camera.
2 is a block diagram illustrating a schematic configuration of a signal processing unit and a connection relationship between each unit of the signal processing unit and a system control unit in the electronic still camera of FIG. 1; FIG.
FIG. 3 is a block diagram showing a schematic configuration of a modification of a block sampling unit in the signal processing unit of FIG. 2;
4 is a timing chart illustrating a schematic operation of an electronic still camera using a signal processing unit configured as a modification of FIG. 3;
[Explanation of symbols]
10 Electronic still camera
11 Camera body
12 Signal processor
13 System controller
14 Drive signal generator
20 Memory card section
11b color filter array
11c CCD image sensor
12a Block sampling unit
12b Display data generator
120a Block division
122a Data extraction unit
124a Block thinning section
126a Block generator
128a block compatible extractor
120b Representative value calculator
122b YC converter

Claims (6)

Color filters of first, second, and third colors having different spectral sensitivity characteristics are arranged in a Bayer pattern, and signal processing is performed on data for signals supplied from pixels arranged corresponding to each color filter and displayed. A signal processing device for generating data, the device comprising:
A block that handles a plurality of pixels in a horizontal direction as a single block for pixels corresponding to a plurality of lines corresponding to the Bayer pattern, and samples data supplied from each pixel of the block in correspondence with each pixel position Sampling means;
The pixel values of the first, second, and third colors in each block are calculated using the data of a plurality of pixels obtained by the block sampling means, and the luminance signal and the color difference signal or Display data generating means for generating three primary color signals as the display data;
Control means for controlling the sampling timing of the block sampling means ,
The block sampling means is a block dividing means for horizontally dividing a condition that includes at least one of the first, second, and third colors in the block;
Block thinning means for thinning every other block divided by the block dividing means in the horizontal direction;
Re-blocking means in which the pixels arranged in the column direction located at both left and right ends in the block thinned out by the block thinning means and the pixels of the blocks adjacent to the sides respectively satisfy the above-mentioned condition to form a new block;
And a block data extracting means for extracting data from each block in order by associating the block of the reblocking means with the block at the thinning position . 2. The apparatus according to claim 1 , wherein the block dividing means treats the pixels for two lines corresponding to the Bayer pattern collectively in units of two pixels in a horizontal direction, and forms a block with a total of four pixels. A characteristic signal processing apparatus.   2. The apparatus according to claim 1, wherein the control unit controls a drive signal generation unit that generates a drive timing signal that outputs a sampling relationship of the block sampling unit and a signal from a pixel in the same relationship. A signal processing device. Color filters of first, second, and third colors having different spectral sensitivity characteristics are arranged in a Bayer pattern, and display is performed by performing signal processing on data for signals supplied from pixels arranged corresponding to each color filter. A signal processing method of a signal processing device for generating data, the method comprising:
A plurality of pixels are handled as one block in the horizontal direction with respect to pixels corresponding to the plurality of lines corresponding to the Bayer pattern, and data supplied from each pixel of the block is sampled corresponding to each pixel position . 1 process,
A second step of calculating a first pixel value of the second and third colors in respective block using the data of a plurality of pixels obtained in the first step,
A third step of generating a luminance signal and a color difference signal or a three primary color signal as the display data with the pixel values calculated in the second output step,
The first step performs sampling of the data based on the timing signal from the fourth step of generating the timing signal of the data sampling timing ,
The first step further includes a fifth step in which a division that satisfies a condition including at least one of the first, second, and third colors in the block is performed in the horizontal direction;
A sixth step of horizontally thinning out every other block divided in the fifth step;
A seventh step in which the pixels arranged in the column direction located at both left and right ends in the block thinned out in the sixth step and the pixels of the blocks adjacent to the sides respectively satisfy the above condition to form a new block;
A signal processing method comprising: an eighth step of extracting data in order from each block by associating the block of the seventh step with the block at the thinning position . The method of claim 4, wherein the block includes wherein the Bayer pattern in handling summarized in two pixels in the horizontal direction with respect to two lines of pixels corresponding to form a block with a total of 4 pixels Signal processing method . 5. The signal processing method according to claim 4 , wherein the fourth step controls generation of a drive timing signal that obtains a signal from the pixel in the same relationship as the sampling relationship in the first step.

Images