JP2002171531A - Method for processing pixel signal of single board color camera - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a method for processing a pixel signal of a single board color camera that can reproduce a pixel signal with a small error to which sophisticated interpolation processing is applicable and enhance a file compression rate in comparison with a data quantity of the pixel signal obtained from a solid-state image pickup element for an achromatic object as well as an object with a high saturation. SOLUTION: The pixel signals from the solid-state image pickup element where a plurality of kinds of very small filters each having a different spectral sensitivity characteristic is applied to each of pixels arranged two-dimensionally in a prescribed repetitive period are separated into pixel signals by each pixel configuring a minimum part of the repetition of the very small filters to obtain a difference signal or a rate signal of other signals with respect to one reference signal. A data compressor generates a data file from the difference signal or the rate signal between the reference signal and the other signals.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a pixel of a single-panel color camera which can reproduce a pixel signal obtained from a solid-state image sensor in a single-panel color camera almost as original and convert it into a file having a small data amount. The present invention relates to a signal processing method.

[0002]

2. Description of the Related Art In recent years, many electronic still cameras, which have become very popular, are constituted by single-chip color cameras that obtain color image signals using only one solid-state image sensor. In a single-chip color camera, R (red), G (green), and B (color) necessary for a color image signal are combined by combining minute color filters having different transmitted lights for each pixel of the solid-state imaging device.
(Blue) signal. Here, as an example of an array of minute color filters, which is the mainstream in electronic still cameras,
A so-called primary color Bayer arrangement is shown in FIG.

In the primary color Bayer array shown in FIG. 2, among the fine filters corresponding to each pixel of the solid-state image pickup device, a fine filter transmitting G is arranged every other pixel in the horizontal direction in every horizontal pixel row, and an odd number The horizontal pixel rows and the even-numbered horizontal pixel rows are arranged in an interleaving relationship in which the horizontal position is shifted. On the other hand, the fine filter transmitting R is disposed at pixels between Gs of odd-numbered horizontal pixel columns, and the fine filter transmitting B is disposed at pixels between Gs of even-numbered horizontal pixel columns. I have. As a result, the overall configuration has a configuration in which a portion of two pixels in the horizontal direction and two pixels in the vertical direction are repeated.

The most general method for obtaining R, G, and B signals from a solid-state image pickup device in which color filters having the arrangement shown in FIG. 2 are combined will be described with reference to the configuration diagram of FIG. In the configuration shown in FIG. 3, the pixel signal obtained from the solid-state imaging device 1 is A / A
The signal is converted into a digital signal by the D converter 2 and then applied to the processor 3. The processor 3 includes a memory 4 for simultaneously extracting the added pixel signals as much as necessary for generating R, G, and B.

Here, for example, a G signal whose pixel position in the horizontal direction is (m + 1) and whose pixel position in the vertical direction is (n + 2) in the arrangement of FIG. 2 is represented as G1 (m + 1, n + 2) (hereinafter, referred to as G1 (m + 1, n + 2)). And similar). At this time, the processor 3 simultaneously extracts the real R (m, n + 2) and R (m + 2, n + 2) from the memory 4 as interpolation signals of the non-real R (m + 1, n + 2), and outputs the average value. I do. Similarly, B (m + 1, n + 1) and B (m + 1, n +) exist as interpolation signals of B (m + 1, n + 2).
3) is fetched from the memory 4 and an operation is performed to output the average value. Further, the horizontal direction (m +
2) For the pixel position in the vertical direction (n + 2), the average value of G2 (m + 2, n + 1) and G2 (m + 2, n + 3) that exist as an interpolation signal of G (m + 2, n + 2) that does not exist, or G1 ( m + 1, n + 2) and G1 (m +
3, n + 2) or the average value of these four pixels is output, and as interpolation signals of non-existent B (m + 2, n + 2), B (m + 1, n + 1), B (m + 3, n +)
1) operates to output an average value of B (m + 1, n + 3) and B (m + 3, n + 3).

As a result, R, G, and B signals are obtained from the processor 3 for all pixel positions, and the total amount of data is three times the pixel signal obtained from the solid-state imaging device.
The R, G, B signals thus obtained are applied to the data compressor 5. The operation of the data compressor 5 may be, for example, general JPEG processing, and a detailed description thereof will be omitted.
That is, the added R, G, and B signals are directly converted or converted into a luminance signal and a chroma signal, and discrete cosine transform (DCT) for each block is performed on each of the signals, followed by quantization and Huffman coding. Put it in a file,
The data is sent to the storage device 6 or the transmission device 7. In JPEG processing, the chroma signal is narrowed to reduce the number of samples in order to combine the R, G, and B signals of the data amount three times as large as the pixel signals of the solid-state imaging device into a file of an appropriate size. Since the data amount of the high frequency component of the luminance signal is reduced, it is known that the reproduced signal is an irreversible conversion having an error from the original signal.

In the above description, the processor 3 interpolates a signal of a color that does not actually exist at a certain pixel position with an average value of signals existing around the pixel. However, JP-A-3-124
190, JP-A-7-236147, and the like, it is stated that a higher quality image can be obtained by performing more advanced interpolation processing. In any of these processes, it is essential that the signal of each pixel be used as it is in the original signal, and a sufficient effect can be obtained even with advanced interpolation processing for pixel signals that have caused errors due to irreversible compression processing. Absent.

In order to perform advanced interpolation processing on a pixel signal having no error from the original signal, the following two methods can be considered.
That is, 1) a method of performing an interpolation process in an electronic still camera before performing irreversible compression for filing, or 2) a personal computer when reproducing an image from a file by filing only by reversible compression To perform advanced interpolation processing. In order to realize the method 1), it is necessary that the electronic still camera has a processing capability capable of withstanding advanced interpolation processing. However, solving this by hardware requires an increase in circuit scale and power consumption. In order to solve the problem by software, it is necessary to increase the processing time and to lengthen the photographing interval.

For these reasons, it is preferable to use the above-mentioned method 2) in order to improve the quality of the reproduced image while reducing the size and power consumption of the electronic still camera. Further, according to this method, the pixel signals of the solid-state imaging device are kept as a file as it is, so that interpolation processing of a new algorithm developed later can be performed at the time of image reproduction.

As an example of a processing method capable of realizing the above method 2) without increasing the amount of data as compared with the pixel signals obtained from the solid-state image sensor, signals of pixels obtained from the solid-state image sensor are converted into R, G, B A method to combine them into a file without separating them into a file, etc.
Japanese Patent Application Laid-Open No. H11-168745 proposes a method of separating only G and B and then interpolating non-existing pixel signals and interpolating only existing pixel signals into a file.
In either method, it is easy to combine the data into a file having a data amount equal to the number of pixels of the solid-state imaging device.
When a multi-pixel device such as a solid-state imaging device having one million pixels or more is used, it is desired to reduce the data amount.

However, if the pixel signals are collected into a file without being separated into R, G, B, etc., when the color filter having the configuration shown in FIG. Changes greatly,
In a subject having high saturation, many high-frequency components are generated. If a signal having many high-frequency components is filed in a form close to reversible with a small error after reproduction, the compression ratio cannot generally be reduced. For example, D which uses the difference from the adjacent pixel as data
The PCM method is clearly disadvantageous in terms of compression ratio because a signal having many high-frequency components always has large data. In addition, the reduction of the data amount of the high-frequency component, which is the basis for increasing the compression ratio by the JPEG method, lowers the resolution with the adjacent pixels, so that advanced interpolation that requires reproduction of a pixel signal with a small error is required. It is difficult to apply when performing processing, and it is not possible to create a file with a high compression ratio even by JPEG.

In the method of separating only the actual pixel signals into files after separating them into R, G, and B, it is possible to prevent the generation of high-frequency components in a high-saturation object, but the output of R, G, and B is reduced. In the case of an achromatic subject having substantially the same size, the separated R, G, and B are filed as three independent equal signals, which is not sufficient in terms of reducing the file capacity.

[0013]

As described above, in the conventional pixel signal processing method for a single-chip color camera, a sufficient compression ratio is required to reproduce a pixel signal with a small error, which enables advanced interpolation processing. There is a problem that it is difficult to make a file in a file. An object of the present invention is to reproduce a pixel signal with a small error, which enables advanced interpolation processing, and to obtain a pixel obtained from a solid-state image sensor not only for a high-saturation subject but also for an achromatic subject. An object of the present invention is to provide a pixel signal processing method for a single-chip color camera capable of increasing a file compression ratio as compared with a signal data amount.

[0014]

According to the present invention, there is provided a pixel signal processing method for a single-chip color camera, comprising: a plurality of pixels arranged two-dimensionally on an imaging surface; When extracting the signals of the plurality of pixels from the solid-state imaging device corresponding to a plurality of different types of microfilters at a fixed repetition cycle to generate a data file, the signal of the plurality of pixels is used for each of the predetermined repetition cycles. Separating the extracted first signal group, and separating at least a second signal group extracted for each of the predetermined repetition periods from a signal obtained by removing the first signal group among the signals of the plurality of pixels; A signal corresponding to a difference between a signal belonging to the first signal group and a signal belonging to the second signal group existing within the cycle is obtained at every constant repetition cycle to generate at least a third signal group. And, and generating a data file using at least the third signal group and the first signal group. If the number of pixels constituting the fixed repetition cycle is two or more, a signal group corresponding to the difference is obtained by the same method as the generation of the second signal group and the third signal group. It may be used to generate a file.

In the pixel signal processing method for a single-chip color camera according to the present invention, a plurality of types of fine filters having different spectral sensitivity characteristics are provided to each of a plurality of pixels arranged two-dimensionally on an imaging surface at a constant repetition period. When extracting the signals of the plurality of pixels from the solid-state imaging device corresponding to the data file and generating a data file, separating the first signal group extracted from the signals of the plurality of pixels at the predetermined repetition cycle, At least a second signal group extracted at each of the predetermined repetition periods is separated from a signal obtained by excluding the first signal group among the signals of the plurality of pixels. A signal corresponding to the ratio between the signal belonging to the first signal group and the signal belonging to the second signal group is obtained to generate at least a fourth signal group, and to generate at least a fourth signal group. And generating a data file using a serial fourth signal group. Note that if the number of pixels constituting the above-mentioned fixed repetition cycle is two or more, the second
A signal group corresponding to each ratio may be obtained in the same manner as in the generation of the signal group and the fourth signal group, and used for generating a data file.

[0016]

According to the pixel signal processing method for a single-chip color camera of the present invention, each pixel signal is converted into a reference signal and a difference signal with respect to the reference signal at each repetition cycle of the minute filter. The difference signal can be expected to be a substantially constant value for a highly saturated subject, and to be close to 0 for an achromatic subject, so the compression ratio of the file can be reduced even by reversible compression such as DPCM. Can be increased. Thus, when a file is reproduced, an original pixel signal having a small error is obtained, and a high quality reproduced image is obtained by performing advanced interpolation processing.

According to another pixel signal processing method for a single-chip color camera of the present invention, each pixel signal is converted into a reference signal and a ratio signal to the reference signal at each repetition period of the minute filter. The ratio signal has a substantially constant value for a highly-saturated subject, and 1 for an achromatic subject.
Therefore, the compression ratio of the file can be increased by reversible compression such as DPCM. As a result, when a file is reproduced, an original pixel signal having a small error is obtained, and a high-quality reproduced image is obtained by performing advanced interpolation processing, as in the case of obtaining a difference signal.

[0018]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A first embodiment of the present invention will be described below with reference to the configuration shown in FIG. In FIG. 1, the functions of the solid-state imaging device 1, the A / D converter 2, and the memory 4 are as follows.
This is similar to the case of the conventional example shown in FIG. Here, in the configuration shown in FIG. 1, the pixel signals of the solid-state imaging device 1 digitized by the A / D converter 2 are added to the processor 8 and the memory 4
, A signal of a total of four pixels, that is, two horizontal pixels and two vertical pixels, which is the minimum repetition of the primary color Bayer array shown in FIG. 2, is simultaneously output. The simultaneously output pixel signals are, for example, R (m, n) and G1 (m +
1, n), G2 (m, n + 1), B (m + 1, n + 1)
And at the next sample point R (m + 2, n), G1
(M + 3, n), G2 (m + 2, n + 1), B (m +
3, n + 1).

For example, R output from processor 8
(M, n), G1 (m + 1, n), G2 (m, n +
The four pixel signals of 1) and B (m + 1, n + 1) are applied to the subtractor 9. R (m, n), G2
(M, n + 1), B (m + 1, n + 1) and G1 (m +
Rd (m, n), G2d
(M, n + 1) and Bd (m + 1, n + 1) are output. That is, Rd (m, n) and G2d (m, n +
1) and Bd (m + 1, n + 1) are as shown in the following equations 1 to 3.

[0020]

(Equation 1)

[0021]

(Equation 2)

[0022]

(Equation 3)

The difference signal Rd (m, n) thus obtained is
G2d (m, n + 1), Bd (m + 1, n + 1) and G1
(M + 1, n) is applied to the data compressor 10 and is independently compressed. At this time, the data compressor 1
Since the number of signals compressed with 0 is equal to the number of pixels constituting the minimum repetition part of the filter array, the number of types of transmitted light of the minute filter constituting the filter is larger in the color filter of FIG.

Here, the increase in the number of signals is caused by the data compressor 1
A slight increase in circuit size due to an increase in the number of channels of 0,
Alternatively, the processing time is increased, but as long as the number of pixels, which is the number of samples of the original signal, is the same, there is no direct disadvantage in the capacity of the file. That is, when the signal is separated into the number of pixels constituting the minimum repetition part, the number of samples per signal decreases even if the number of signals increases, and data is divided into blocks of about 64 pixels like JPEG, for example. The total number of blocks in the case of processing is the same as that in the case of separation for each transmitted light. Further, it is apparent that the use of DPCM is not disadvantageous in terms of the file capacity because the number of all signals does not change.

Further, from the relations of the equations (1) to (3), the difference signals Rd, G2d, and Bd become signals of a constant magnitude even in a highly-saturated object such as green, and generate high-frequency components. Absent. In addition, since the magnitudes of the R, G, and B signals are substantially equal in an achromatic subject, the difference signals Rd, G2d, and Bd can be expected to be sufficiently small. At this time, the data amount of the compressed difference signals Rd, G2d, and Bd can be expected to be sufficiently small even by the DPCM method, for example.

Here, the data compressor 10 compresses G1 and the difference signals Rd, G2d, and Bd, respectively, and combines them into a file, and stores them in a file as in the conventional example shown in FIG. To send out. To play an image from a stored or received file, G1 and Rd, G2d, Bd are played from the file, and Rd, G2d,
The R, G2, and B signals are reproduced by adding the G1 signal at the sample position corresponding to each of the Bd sample signals. If these are rearranged, the pixel signal obtained from the solid-state imaging device can be restored as it is, so that a high-quality reproduced image can be obtained by performing advanced interpolation processing by a personal computer or the like.

FIG. 4 shows a second embodiment of the present invention. 4, the operations of the solid-state imaging device 1, the A / D converter 2, the processor 8, and the memory 4 are the same as those of the first embodiment shown in FIG.
This is the same as the embodiment. The embodiment of FIG. 4 differs from the embodiment of FIG. 1 in that signals of horizontal two pixels and vertical two pixels, which are minimum repetitions simultaneously obtained from the processor 8, are added to the divider 11. .

The divider 11 includes, for example, R (m, n), G
1 (m + 1, n), G2 (m, n + 1), B (m + 1,
(n + 1) is added, a signal Rr (m, n) representing the ratio of R (m, n) to G1 (m + 1, n), G
Signals G2r (m, n + 1) and G1 (m + 1, n) representing the ratio of G2 (m, n + 1) to 1 (m + 1, n)
Br representing the ratio of B (m + 1, n + 1) to
(M + 1, n + 1) is output. That is, Rr (m,
n), G2r (m, n + 1), Br (m + 1, n + 1)
Are as follows.

[0029]

(Equation 4)

[0030]

(Equation 5)

[0031]

(Equation 6)

The ratio signal Rr (m, n) thus obtained,
G2r (m, n + 1), Br (m + 1, n + 1) and G1
(M + 1, n) is applied to the data compressor 10 and is respectively compressed. Here, the ratio signals Rr and Br can be expected to be signals of a constant magnitude even for a highly-saturated subject such as green, for example, and a constant value as long as the color does not change even if the brightness changes. Keep. In addition, since the magnitudes of the R, G, and B signals are substantially equal in an achromatic subject, the ratio signals Rr, Br can be expected to be close to 1. In particular, G2r can be expected to be close to 1 in many cases. At this time, the data amount of the compressed ratio signals Rr, G2r, and Br can be expected to be sufficiently small even by the DPCM method, for example.

The data compressor 10 compresses the data of G1 and the ratio signals Rr, G2r, and Br, respectively, and combines them into a file.
Is the same as in the first embodiment shown in FIG. To reproduce an image from a stored or received file, G1 and Rr, G2r, and Br are reproduced from the file, and the signal at the sample position corresponding to the signal at each sample position of Rr, G2r, and Br is multiplied. R, G
It is only necessary to perform advanced interpolation processing after reproducing the signals of 2 and B, and it is possible to reproduce high-quality images.

Further, in the embodiment of the present invention, a method of obtaining a difference signal or a ratio signal with respect to G1 in FIG. 2 as a reference signal has been described as an example, but G2, R, B
It is apparent that substantially the same result can be expected even when another signal is used as a reference signal. Further, in the embodiment of the present invention, the case of using the color filter of the primary color type Bayer array shown in FIG. 2 is taken as an example. However, the present invention separates the signal into the minimum number of pixels of the repetition part, and It is essential to generate a file by using a difference signal or a ratio signal of one signal with respect to another signal. Therefore, the embodiment of the present invention is not limited to the case where the minimum repeated portion of the filter array is two horizontal pixels and two vertical pixels. Furthermore, the embodiment of the present invention is not limited to the case where the transmitted light of the micro filter used for the filter is R, G, B.

[0035]

As described above, according to the pixel signal processing method for a single-chip color camera of the present invention, it is possible to reproduce a pixel signal with a small error, which can be subjected to advanced interpolation processing, and to achieve a high chroma It is possible to record and transmit pixel data having a high compression ratio compared to the data amount of the pixel signal obtained from the solid-state imaging device, not only for a high object but also for an achromatic object.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration of a first embodiment according to a pixel signal processing method of a single-chip color camera of the present invention.

FIG. 2 is a diagram showing an example of an arrangement of color filters used in a single-chip color camera according to the present invention.

FIG. 3 is a diagram illustrating a configuration of a conventional single-chip color camera according to a pixel signal processing method.

FIG. 4 is a diagram illustrating a configuration of a second embodiment of the single-chip color camera according to the present invention, which is based on the pixel signal processing method;

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Solid-state image sensor 2 A / D converter 3 Processor 4 Memory 5 Data compressor 6 Storage device 7 Transmission device 8 Processor 9 Subtractor 10 Data compressor 11 Divider

Claims (2)

[Claims] 1. A signal of a plurality of pixels from a solid-state image sensor in which a plurality of types of microfilters having different spectral sensitivity characteristics correspond to each of a plurality of pixels arranged two-dimensionally on an imaging surface at a constant repetition cycle. In the signal processing method for a single-chip color camera that generates a data file by extracting the first signal group extracted at every predetermined repetition period from the signals of the plurality of pixels, At least a second signal group extracted for each of the predetermined repetition cycles is separated from the signals except for the first signal group, and the first signal present in the cycle for each of the predetermined repetition cycles is separated. A signal corresponding to the difference between the signal belonging to the group and the signal belonging to the second signal group is obtained to generate at least a third signal group, and the first signal group and at least the third signal group are generated. A pixel signal processing method for a single-chip color camera, characterized in that a data file is generated using the method. 2. A signal of a plurality of pixels from a solid-state image sensor in which a plurality of types of microfilters having different spectral sensitivity characteristics correspond to each of a plurality of pixels arranged two-dimensionally on an imaging surface at a constant repetition cycle. In a single-chip color camera pixel signal processing method for extracting a signal from the plurality of pixels and extracting a first signal group extracted at a predetermined repetition period from the plurality of pixel signals, Out of the signals except for the first signal group, at least a second signal group extracted at each of the predetermined repetition cycles is separated, and the first signal group existing within the cycle at each of the predetermined repetition cycles is separated. A signal corresponding to the ratio of the signal belonging to the signal group to the signal belonging to the second signal group is obtained to generate at least a fourth signal group, and the first signal group and at least the fourth signal A pixel signal processing method for a single-chip color camera, wherein a data file is generated using a group.