JPH10155157A - Image pickup device and processing method for color image signal - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image pickup device and a processing method for a color image signal in which high resolution is obtained at a low cost without sacrificing various functions such as an electronic shutter. SOLUTION: Color signals red(R), green(G) and blue(B) are fed to an adaptive interpolation processing circuit 15. Picture element signals R, G, B are stored in an arrangement corresponding to picture element arrangement of a charge coupled device(CCD) image sensor 11. That is, the adaptive interpolation processing circuit 15 receives and stores the picture element signal G arranged in tessellated pattern and the picture element signals R, B arranged in zigzag. The adaptive interpolation processing circuit 15 applies at first adaptive interpolation processing to the picture element signals R, B to arrange the signals in tessellated pattern state and applies interpolation processing in longitudinal and lateral directions to the picture element signals R, G, B arranged in tessellated pattern to improve the color resolution.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to an image pickup apparatus for obtaining a high resolution by performing an interpolation process between pixels and a method of processing a color image signal.

[0002]

2. Description of the Related Art In recent years, high-resolution color images are suitable as input images for personal computers, and the penetration of multimedia has led to a demand for higher-resolution color cameras. A three-panel camera has been provided as a camera for improving such a full-color resolution.

[0003]

However, a three-panel camera is very expensive compared to a single-panel camera because it has three image pickup devices. In particular, when a CCD image sensor having 1.3 million pixels or more is used, it is very expensive. It will be expensive. 3
The plate-type camera requires correction of color misregistration by the prism of the optical unit, and it is difficult to reduce the size because the fixing of the three imaging elements requires a high-precision fixing technology of about several μm. .

On the other hand, a single-panel camera is less expensive than a three-panel camera, does not require color misregistration correction, does not require a fixing technique, has a high imaging speed, and can be used with an electronic shutter. However, the single-plate camera has low color resolution, and as shown in FIG. 23, it has been difficult to increase the resolution even by using an optical low-pass filter or the like in order to reduce the influence of aliasing.

As a method of obtaining a high-resolution image, there is a method of obtaining full-color high-resolution from a plurality of frames by a wobbling method of optical path shift using a bimorph-driven CCD image sensor or the like. This method can be used for a single-plate type and a two-plate type, but has disadvantages such as a reduction in imaging speed and the inability to use an electronic shutter, and its use has been limited.

SUMMARY OF THE INVENTION The present invention has been made in view of such a problem, and an image pickup apparatus and a color image signal which can obtain high resolution at low cost without sacrificing various functions such as an electronic shutter. The purpose of the present invention is to provide a processing method.

[0007]

In order to solve the above-mentioned problems, an image pickup apparatus according to the present invention is arranged such that pixels of three primary colors are arranged in a matrix, and green pixels arranged in a checkered pattern in the matrix. An image sensor configured by continuously arranging three red and blue pixels each having the same number of pixels in a zigzag manner with the green pixel interposed therebetween in three spaces, and a color output by each pixel of the image sensor. A storage unit that fetches and stores signals, and a pixel interpolation point near a pixel corresponding to each color signal stored in the storage unit has a correlation degree between a plurality of pixels near the pixel interpolation point. Interpolating means for performing an interpolation process for each color signal in a larger direction.

Further, in the method of processing a color image signal according to the present invention, pixels of three primary colors are arranged in a matrix, and the number of pixels is the same in a gap between green pixels arranged in a checkered pattern in the matrix. The color signal output from each pixel of the image sensor constituted by the zigzag arrangement of three red pixels and three blue pixels in succession with the green pixel interposed therebetween is fetched and stored. Interpolation processing is performed for each color signal with respect to a pixel interpolation point near a corresponding pixel in a direction in which the degree of correlation between a plurality of pixels near the pixel interpolation point is large.

[0009]

Embodiments of the present invention will be described below with reference to the drawings.

The image pickup apparatus 1 according to the first embodiment is of a single-plate type, and performs adaptive interpolation between pixels for each color signal obtained from a CCD (Charge Coupled Device) image sensor. To achieve higher resolution.

As shown in FIG. 1, an imaging apparatus 1 according to a first embodiment includes a CCD image sensor 11 that outputs color signals of three primary colors, a correlated double sampling / automatic gain control (CDS / AGC: Correlated Double sampling / Aut
omatic Gain Control) circuit 12, an A / D converter 13 for converting to a digital signal, a white balance,
Camera signal correction processing circuit 14 that performs processing such as gamma correction
, An adaptive interpolation processing circuit 15, a video encoder 16 for converting into a predetermined television signal, and a system controller 17 for controlling each circuit.

In the CCD image sensor 11, as shown in FIG. 2, red (R) pixels of three primary colors, green (G), and blue (B).
Pixels are arranged in a matrix. In this matrix, G pixels are provided in a checkered pattern (twin-canx). R and B pixels having the same number of pixels are G
It is a gap between pixels, and is arranged in a zigzag manner by three pixels continuously with the G pixel interposed therebetween.

The CCD image sensor 11 is of an all-pixel readout type. When receiving incident light from a subject, the CCD image sensor 11 generates color signals R, G, and B in accordance with the incident light to generate a C signal.
It is supplied to the DS / AGC circuit 12.

The CDS / AGC circuit 12 samples and holds a precharge level and a data level of each color signal, detects a difference between the precharge level and the data level, detects an accurate signal level, and removes random noise. The gain of the CDS / AGC circuit itself is automatically controlled according to the strength, and a color signal having a stable signal level is always output.

The A / D converter 13 is driven based on a sampling pulse, and has a CDS / A
Each color signal from the GC circuit 12 is converted into a digital signal and supplied to the camera signal correction processing circuit 14.

The camera signal correction processing circuit 14 applies so-called digital signal processing such as gamma correction and knee processing to each color signal, and supplies the obtained three primary color signals to the adaptive interpolation processing circuit 15.

Each of the color signals R,
G and B are supplied, and pixel signals R, G and B are stored in an array corresponding to the pixel array of the CCD image sensor 11. That is, as shown in FIG. 3, the adaptive interpolation processing circuit 15 converts the pixel signals G arranged in a twin-canx manner and the pixel signals B and R arranged in a zigzag manner as shown in FIGS. Captured and stored. The adaptive interpolation processing circuit 15 first performs adaptive interpolation processing on the pixel signal R and the pixel signal B and arranges them in the twin-canx shape shown in FIG. 3, and then, the pixel signals R and pixels arranged in the twin-canx shape The interpolation processing in the vertical direction or the horizontal direction is performed on the signal G and the pixel signal B, respectively, to improve the color resolution.

More specifically, when the color signals R, G, and B are supplied from the camera signal correction processing circuit 14, the adaptive interpolation processing circuit 15 performs, for example, the step S1 shown in FIGS. Steps S13 to S13 are performed.

In step S1 shown in FIG. 6, when the color signal R is supplied, the adaptive interpolation processing circuit 15 fetches the pixel signal R (indicated by a triangle) in a twin-canx shape as shown in FIG. Proceed to S2.

In step S2, the adaptive interpolation processing circuit 15 sets the pointer to the first pixel interpolation point, and proceeds to step S3. In step S2 and step S4 described later, as shown in FIG. 8, the pointer is in a diagonal direction (diagonal direction) of the taken pixel signal R (hereinafter, referred to as an effective pixel R) and sandwiches the pixel interpolation point. Thus, six effective pixels R are set at the pixel interpolation points (marked by x) that are continuous.

In step S3, the adaptive interpolation processing circuit 15 performs an interpolation process of the pixel signal R in an oblique direction so as to make the pixel signal R into a twin-canx arrangement.

Here, since the pixel signal R is not taken in the vertical direction of the pixel interpolation point, the adaptive interpolation processing circuit 15 cannot perform the vertical interpolation process. Therefore,
By performing the interpolation processing in the oblique direction including the information in the vertical direction, it is possible to prevent the resolution in the horizontal direction from deteriorating.

More specifically, the adaptive interpolation processing circuit 15 performs the following based on six pixel signals R that are continuous with the pixel interpolation point interposed therebetween in either the upper left diagonal direction or the lower left diagonal direction. The formula is calculated, the interpolation process is performed, and the process proceeds to step S4.

R4 = aR1 + bR2 + cR3 + dR4 +
eR5 + fR6 Here, the values from a to f are weighting coefficients corresponding to the levels of the color signals R1 to R6.

In step S4, the adaptive interpolation processing circuit 15 sets the pointer to the next pixel interpolation point, and proceeds to step S5.

In step S5, the adaptive interpolation processing circuit 15 determines whether or not the pixel interpolation point indicated by the mark "x" has disappeared. If the pixel interpolation point has disappeared, the interpolation processing of the pixel interpolation point is terminated, and the process proceeds to step S6. If not, the process returns to step S3.

In step S6, the adaptive interpolation processing circuit 15 receives pixel signals R (marked by triangles) arranged in a mesh pattern as shown in FIG. The adaptive interpolation processing circuit 15 sets the pointer to the pixel interpolation point indicated by the triangle, and proceeds to step S7. Here, in step S6 and step S8 described later, as shown in FIG. 9, the pointer is set to a pixel interpolation point (marked by △) located at the center of the mesh.

In step S7, the adaptive interpolation processing circuit 15 determines, for a certain pixel interpolation point, a later-described step S7.
According to the subroutine of 21 to step S37, the correlation adaptive interpolation processing in the vertical direction, the horizontal direction or the diagonal direction is performed,
The process proceeds to step S8 shown in FIG.

In step S8, the adaptive interpolation processing circuit 15 sets a pointer to another pixel interpolation point indicated by a triangle, and
Proceed to step S9.

In step S9, the adaptive interpolation processing circuit 15 determines whether or not there is the pixel interpolation point. If there is such a pixel interpolation point, the process returns to step S7. The interpolation processing of the interpolation point ends, and the process proceeds to step S10.

In step S10, the adaptive interpolation processing circuit 15 receives pixel signals R (marked by triangles) arranged in a twin-canx pattern as shown in FIG. Therefore, the adaptive interpolation processing circuit 15 sets the pointer to a predetermined pixel interpolation point (square), and proceeds to step S11.

In step S11, the adaptive interpolation processing circuit 15 executes a vertical or horizontal interpolation process in accordance with a subroutine of steps S41 to S45 described later, and proceeds to step S12.

In step S12, the adaptive interpolation processing circuit 15 sets the pointer to another pixel interpolation point (marked by □), and proceeds to step S13.

In step S13, the adaptive interpolation processing circuit 15 determines whether or not there is a pixel interpolation point indicated by a square. If there is a pixel interpolation point, the process returns to step S11. finish. The color signal R composed of the pixel signal R thus interpolated is converted into a predetermined television signal via the video encoder 16 and output.

Here, in step S7, the adaptive interpolation processing circuit 15 performs the processing from step S21 to step S37 shown in FIGS. 11 and 12 so as to perform the interpolation processing at the pixel interpolation point where the pointer is set. It has become. That is, the adaptive interpolation processing circuit 15 calculates the vertical, horizontal, and vertical positions in the vicinity of the eight effective pixels R surrounded by a chain line shown in FIG.
Then, in order to calculate the degree of correlation S of the pixel signal R in the oblique direction and perform the interpolation in the direction in which the degree of correlation S becomes the maximum, the processing from step S21 shown in FIG. 11 is performed.

It should be noted that the degree of correlation S is represented by a pixel row Rn in a certain direction.
Is defined as follows.

S = min (Rn) / max (Rn) (the correlation is maximum when S = 1 because S ≦ 1) In step S21, the adaptive interpolation processing circuit 15 calculates the vertical correlation a. I do. Here, the vertical correlation degree a
Is specifically calculated by the following equation. Note that R1 to R9 shown in the following expressions indicate the level of the pixel signal R.

A = min (Rn) / max (Rn) min (Rn) = min (R1, R7) · min (R
2, R8) · min (R3, R9) max (Rn) = max (R1, R7) · max (R
2, R8) · max (R3, R9) The adaptive interpolation processing circuit 15 calculates the calculated correlation degree a as a variable X
And sets the flag A, and then proceeds to step S22.

In step S22, the adaptive interpolation processing circuit 15 calculates the horizontal correlation degree b based on the following equation, and proceeds to step S23.

B = min (Rn) / max (Rn) min (Rn) = min (R1, R3) · min (R
4, R6) · min (R7, R9) max (Rn) = max (R1, R3) · max (R
4, R6) .max (R7, R9) In step S23, the adaptive interpolation processing circuit 19 compares the value of X with the correlation b in order to compare the correlation in the vertical and horizontal directions.
It is determined whether X is greater than X. If X is greater, step S2
The process proceeds to step S5, and if X is not large, the process proceeds to step S24.

In step S24, the adaptive interpolation processing circuit 19 sets the flag B by substituting the degree of correlation b for the variable X, and proceeds to step S25.

In step S25, the adaptive interpolation processing circuit 15 calculates the degree of correlation c in the upper left and lower right diagonal directions by the following equation, and proceeds to step S26.

C = min (Rn) / max (Rn) min (Rn) = min (R2, R3, R6) · min
(R1, R9) · min (R4, R7, R8) max (Rn) = max (R2, R3, R6) · max
(R1, R9) · max (R4, R7, R8) In step S26, the adaptive interpolation processing circuit 19 determines whether the variable X is larger than c, and when it is larger than c, proceeds to step S28 shown in FIG. If it is not larger than c, the process proceeds to step S27.

In step S27, the adaptive interpolation processing circuit 19 substitutes the value of the degree of correlation c for
, And the process proceeds to step S28.

In step S28, the adaptive interpolation processing circuit 19 calculates the degree of correlation d in the lower left and upper right diagonal directions by the following equation, and proceeds to step S29.

D = min (Rn) / max (Rn) min (Rn) = min (R2, R1, R4) · min
(R3, R7) · min (R6, R9, R8) max (Rn) = max (R2, R1, R4) · max
(R3, R7) · max (R6, R9, R8) In step S29, the adaptive interpolation processing circuit 19 determines whether or not the correlation d is larger than the variable X. If X is larger, the process proceeds to step S31. If not, the process proceeds to step S30.

In step S30, the adaptive interpolation processing circuit 19 substitutes the degree of correlation d for the variable X, sets a flag D, and proceeds to step S31.

In step S31, the adaptive interpolation processing circuit 19 determines whether or not the flag set for the variable X is A. If the flag is A, the process proceeds to step S32.
If the flag is not A, the process proceeds to step S33.

In step S32, the adaptive interpolation processing circuit 15 calculates the average of the pixel signals R in the vertical direction near the pixel interpolation point. For example, assuming that the level of the pixel signal R to be interpolated is R5, by calculating R5 = (R2 + R8) / 2, the vertical interpolation processing is performed, and the subroutine processing ends.

In step S33, the adaptive interpolation processing circuit 15 determines whether or not the flag is B. When the flag is B, the process proceeds to step S34, and when the flag is not B, the process proceeds to step S35.

In step S34, the adaptive interpolation processing circuit 15 performs a horizontal interpolation process by calculating R5 = (R4 + R6) / 2, and terminates the subroutine process.

In step S35, the adaptive interpolation processing circuit 15 determines whether the flag is C. If the flag is C, the process proceeds to step S36. If the flag is not C, the process proceeds to step S37.

In step S36, the adaptive interpolation processing circuit 15 performs an oblique interpolation process by calculating R5 = (R1 + R9) / 2.
The subroutine processing ends.

In step S37, the adaptive interpolation processing circuit 15 performs an oblique interpolation process by calculating R5 = (R3 + R7) / 2.
The subroutine processing ends.

According to the above-described correlation adaptive interpolation processing, the adaptive interpolation processing circuit 19 performs the adaptive interpolation processing as shown in FIG. 13 by R5 = sR1 + tR2 + uR3 + vR4 + wR6 + xR
7 + yR8 + zR9. The optimum adaptive interpolation processing can be performed by setting a to h after obtaining the degree of correlation between the pixels surrounding the pixel interpolation point. That is, the adaptive interpolation processing circuit 15 calculates the degree of correlation S of the pixel signals R in the vertical, horizontal, and oblique directions, and performs interpolation in a direction in which the degree of correlation S is maximum, thereby increasing the color resolution of R. Can be improved.

Instead of calculating the degree of correlation, the level difference of the pixel signal R in each of the vertical and horizontal directions may be calculated. That is, when the level difference of the pixel signal R in each direction is small, the degree of correlation in that direction increases, and when the level difference of the color signal is large, the degree of correlation in that direction decreases. The adaptive interpolation processing may be performed in the direction in which the difference is the smallest.

In the above step S11, the adaptive interpolation processing circuit 15 performs the processing from step S41 shown in FIG. 14 onward in order to further execute the interpolation processing on the pixel signals R arranged in the twin-canc. Has become.

In step S41, the adaptive interpolation processing circuit 15 calculates the vertical correlation p. Here, the correlation p in the vertical direction is calculated by the following equation, assuming that the levels of the pixel signals R are R1 to R4, as shown in FIG.

P = min (R1, R2) / max (R1, R2) In step S42, the adaptive interpolation processing circuit 15 calculates the degree of correlation q in the horizontal direction. Here, the horizontal correlation degree q
Is specifically calculated by the following equation.

Q = min (R3, R4) / max (R3, R4) In step S43, the adaptive interpolation processing circuit 15 determines whether the correlation p is larger than the correlation q, and When it is larger, the process proceeds to step S44, and when the correlation p is not larger than the correlation q, the process proceeds to step S45.

In step S44, the adaptive interpolation processing circuit 15 performs a vertical interpolation process by calculating R0 = (R1 + R2) / 2, for example, when the level of the pixel interpolation point is R0.

In step S45, the adaptive interpolation processing circuit 15 performs a horizontal interpolation process by calculating R0 = (R3 + R4) / 2.

That is, the adaptive interpolation processing circuit 15 calculates the vertical and horizontal correlations with respect to the pixel interpolation points,
By performing interpolation processing adaptively in the direction of the higher correlation, the color resolution can be improved.

As described above, steps S1 to S13
(Steps S21 to S37 and steps S41 to S45
), The adaptive interpolation processing circuit 15 can output a high-resolution color signal R including the interpolated pixel signal R. Note that the pixel signal B captured by the adaptive interpolation processing circuit 15 can be subjected to interpolation processing in the same manner as the pixel signal R, and a high-resolution color signal B composed of the interpolated pixel signal B can be output.

The adaptive interpolation processing circuit 15 performs the vertical or horizontal interpolation processing on the pixel signals G arranged in the twin-canx and taken in according to the processing from step S41 to step S45 described above. To improve the color resolution.

That is, the imaging apparatus 1 according to the present embodiment
Then, by performing adaptive interpolation processing on each color signal output from a predetermined color filter array, it is possible to improve resolution not only in the vertical and horizontal directions but also in all directions including oblique directions. In addition, despite being inexpensive and compact because of the single-plate type, high resolution equivalent to the three-plate type can be obtained without using a pixel shifting mechanism or the like. In addition, since the imaging device 1 can obtain high resolution without using a pixel shifting mechanism or the like, an electronic shutter function can be used. Since the image pickup apparatus 1 uses the CCD image sensor 11 of the all-pixel readout type, it is possible to obtain a non-interlace image optimal for a monitor device of a personal computer or the like.

In the above-described embodiment, as shown in FIG. 8, the adaptive interpolation processing circuit 15 performs a diagonal interpolation process on a pixel interpolation point with a cross after taking in the pixel signal R, As shown in FIG. 9, the adaptive interpolation processing is performed on pixel interpolation points marked with a triangle, but this order may be reversed.

At this time, the adaptive interpolation processing circuit 15
As shown in 6, based on R2 to R4, R6, R8, and R9, the pixel interpolation point R5 is adaptively subjected to interpolation processing. Specifically, the adaptive interpolation processing circuit 15 performs the same processing as in steps S21 to S35 described above.

Here, the adaptive interpolation processing circuit 15 cannot calculate the average value of R1 and R9 as in step S36 when performing the interpolation process in the upper left and lower right directions, so that the level value of R9 is not changed. Hold, R5 = R
9 is assumed.

Similarly, when performing the interpolation process in the lower left and upper right diagonal directions, the adaptive interpolation processing circuit 15 cannot calculate the average value of R3 and R7 as in step S37, so the level value of R9 is held as it is. And R5 = R
3 is assumed. Therefore, when the degree of correlation in the oblique direction is high, the adaptive interpolation processing circuit 15 can improve the color resolution by taking in information of the neighboring pixels in the oblique direction into the pixel interpolation points.

Next, a second embodiment of the present invention will be described. Note that the same circuits and the like as those in the first embodiment are denoted by the same reference numerals, and detailed description thereof will be omitted.

An imaging apparatus 1A according to the second embodiment is
For example, as shown in FIG. 17, the camera head 10 and the personal computer 20
0 to the personal computer 20
This is suitable for performing image processing by importing the image data into a computer.

More specifically, the camera head 10
As shown in FIG. 7, each color signal R,
A CCD image sensor 11 for outputting G and B, and a CCD
A CDS / AGC circuit 12 for performing correlated double sampling processing and gain control on each color signal from the image sensor 11,
An A / D converter 13 for converting each color signal from the CDS / AGC circuit 12 into a digital signal; a camera signal correction processing circuit 14 for performing signal processing such as gamma correction on each color signal from the A / D converter 13; Correction circuit 1
A video encoder 16 for converting each color signal from the camera 4 into a predetermined method and outputting the converted signal, a system controller 17 for controlling the entire apparatus, and a simple interpolation circuit for performing a simple interpolation process described later on each color signal from the camera signal correction circuit 14 18 and
A personal computer interface (hereinafter, referred to as a personal computer I / F) 19 for converting each color signal from the camera signal correction processing circuit 14 into a predetermined signal and transmitting the signal to the personal computer 20 via the bidirectional bus 21 is provided.

As shown in FIG. 17, a personal computer 20 includes a personal computer I / F 22 to which each color signal is transmitted via a bidirectional bus 21 and a hard disk drive (HDD) for storing a predetermined program and the like. : Hard
Disk Drive) memory 23 and an adaptive interpolation processing circuit 15 for performing adaptive interpolation processing on each color signal from the personal computer I / F 34
And an accelerator 24 for outputting each color signal from the adaptive interpolation processing circuit 15 to the monitor 30.

Here, the simple interpolation circuit 18 is provided, for example, in FIG.
As shown in FIG. 8, when a color signal is supplied, a line memory 18a for storing one line of pixel signals, a line memory 18b for storing one line of pixel signals from the line memory 18a, and a line memory 18b. And a pixel calculator 18d that performs a simple interpolation process for each pixel based on the pixel signals from each of the line memories 18a, 18b, and 18c.

The pixel calculator 18d is provided with a line memory 18
When the pixel signals G are supplied from a, 18b, and 18c, as shown in FIG. 3 described above, the pixel signals G are arranged and taken in a twin-canx shape. As shown in FIG. 19, when the levels of the pixel signals G are G0 to G4 and the weighting coefficients a to d are used, the level G0 of the interpolation point can be obtained by the following equation.

G0 = aG1 + bG2 + cG3 + dG4 The pixel calculator 18d performs a simple interpolation process on the pixel signals R and B based on the image signals for three lines as shown in FIGS. 20 and 21, for example. It is supposed to do.

More specifically, the adaptive interpolation processing circuit 15 obtains the signal level of the pixel interpolation point from the interpolation points α to ζ using the weighting coefficients a to m and the following equation.

In the case of FIG. 20, α = aR1 + bR2 + cR3 β = dR1 + eR3 + fR5 γ = gR5 In the case of FIG. 21, δ = hR7 + iR8 ε = jR7 + kR12 ζ = lR11 + mR12 The sensor 11 generates color signals R, G, and B, and outputs a CDS / AGC circuit 12, an A / D
The signal is supplied to the camera signal correction processing circuit 14 via the converter 13. The camera signal correction processing circuit 14 converts the signal into a predetermined video signal via the video encoder 23 and outputs a luminance signal Y / chroma signal C.
, And outputs the respective color signals by performing simple interpolation, or transmits the respective color signals to the personal computer 20 via the personal computer I / F 19.

In the personal computer 20, the personal computer I / F 22 supplies each color signal to the adaptive interpolation processing circuit 15. The adaptive interpolation processing circuit 15 performs interpolation processing adaptively by performing the processing of the above-described steps S1 to S13 (including steps S21 to S37 and steps S41 to S45), and converts each color signal R, G, B into an accelerator. The signal is supplied to the monitor 50 via 36. Here, the personal computer 20 executes the high-resolution adaptive interpolation processing performed by the adaptive interpolation processing circuit 15 by software processing.

As described above, the imaging apparatus 1A according to the second embodiment performs the adaptive interpolation processing by the software processing in the personal computer, so that the size and weight of the camera head 10 can be reduced. Even if the software is upgraded, for example, the interpolation processing can be flexibly performed without changing the entire imaging apparatus 1A. In addition, since the imaging apparatus 1A is a single-panel type, it is inexpensive and compact, but can obtain high resolution without using a pixel shifting mechanism or the like, and can also use an electronic shutter function. In addition, the imaging device 1A
Uses the CCD image sensor 11 of the all-pixels readout type, so that a non-interlace image optimal for a monitor device of a personal computer can be obtained.

The present invention is not limited to the above-described embodiment. For example, an image sensor such as a MOS may be used instead of a CCD image sensor. It goes without saying that various changes can be made in.

Further, the present invention, as shown in FIG.
The color filters of the D image sensor 11 are not limited to those in which the pixel signals R and G are arranged in a zigzag manner in the vertical direction. For example, as shown in FIG. Needless to say, they may be arranged in a zigzag direction.

[0084]

As described above in detail, according to the image pickup apparatus of the present invention, the pixel interpolation point is located near the pixel corresponding to each color signal stored in the storage means. By performing an interpolation process for each color signal in a direction in which the degree of correlation between a plurality of pixels in the vicinity of is large, a high-resolution full-color image can be obtained even with a single-plate system.

The imaging device calculates the degree of vertical, horizontal, or oblique correlation of pixels near the pixel interpolation point,
The color resolution can be improved by performing the interpolation processing in the direction in which the calculated correlation degree is large.

When the image pickup apparatus includes a direction having no pixel information on a straight line passing through the pixel interpolation point, the pixel interpolation point is interpolated in a direction in which a plurality of pixels are continuous with the pixel interpolation point interposed therebetween. Accordingly, the interpolation processing can be performed without deteriorating the resolution in the predetermined direction, and the color resolution can be improved.

According to the color image signal processing method of the present invention, the correlation of a plurality of pixels near the pixel interpolation point with the pixel interpolation point near the pixel corresponding to each stored color signal is determined. By performing the interpolation process for each color signal in the direction in which the degree is large, a high-resolution full-color image can be obtained.

In the method of processing a color image signal, the degree of correlation in the vertical, horizontal, or oblique direction of a pixel near a pixel interpolation point is calculated, and interpolation processing is performed in a direction in which the calculated degree of correlation is large. , The color resolution can be improved.

In the method of processing a color image signal, when a straight line passing through a pixel interpolation point includes a direction having no pixel information,
By performing interpolation processing of pixel interpolation points in a direction in which a plurality of pixels are continuous with respect to the pixel interpolation point, interpolation processing can be performed without deteriorating resolution in a predetermined direction, and color resolution can be improved. Can be.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of an imaging device according to a first embodiment of the present invention.

FIG. 2 is a diagram illustrating a pixel array of a CCD image sensor.

FIG. 3 is a diagram illustrating a state when a color signal G is taken into an adaptive interpolation processing circuit.

FIG. 4 is a diagram illustrating a state when a color signal B is taken into an adaptive interpolation processing circuit.

FIG. 5 is a diagram illustrating a state when a color signal R is taken into an adaptive interpolation processing circuit.

FIG. 6 is a flowchart illustrating an interpolation process performed in the adaptive interpolation processing circuit.

FIG. 7 is a flowchart illustrating an interpolation process performed in the adaptive interpolation processing circuit.

FIG. 8 is a diagram for explaining pixel interpolation points for performing an interpolation process.

FIG. 9 is a diagram for explaining pixel interpolation points for executing an interpolation process.

FIG. 10 is a diagram illustrating the state of pixels arranged in twin-canks by interpolation processing.

FIG. 11 is a flowchart illustrating the content of an adaptive interpolation process.

FIG. 12 is a flowchart illustrating the contents of an adaptive interpolation process.

FIG. 13 is a diagram illustrating a state in which adaptive interpolation processing is performed based on pixels near a pixel interpolation point.

FIG. 14 is a flowchart illustrating the contents of an adaptive interpolation process.

FIG. 15 is a diagram illustrating a state when adaptive interpolation processing is performed based on pixels near the pixel interpolation point.

FIG. 16 is a diagram illustrating a state when adaptive interpolation processing is performed based on pixels near a pixel interpolation point.

FIG. 17 is a block diagram illustrating a configuration of an imaging device according to a second embodiment of the present invention.

FIG. 18 is a block diagram illustrating a configuration of a simple interpolator.

FIG. 19 is a diagram illustrating a state when the interpolation process of the pixel signal G is performed by the simple interpolator.

FIG. 20 is a diagram illustrating a state when the interpolation process of the pixel signal R and the pixel signal B is performed by the simple interpolator.

FIG. 21 is a diagram illustrating a state when the interpolation process of the pixel signal R and the pixel signal B is performed by the simple interpolator.

FIG. 22 is a diagram showing another arrangement of the color filters of the CCD image sensor.

FIG. 23 is a diagram illustrating the effect of aliasing of a color signal.

[Explanation of symbols]

 11 CCD image sensor, 15 Adaptive interpolation processing circuit

Claims (6)

[Claims] 1. A pixel of three primary colors is arranged in a matrix, and red and blue pixels having the same number of pixels are interposed between green pixels arranged in a checkered pattern in the matrix with the green pixel interposed therebetween. An image sensor configured to be continuously arranged in a zigzag manner by three pixels, a storage unit that captures and stores a color signal output from each pixel of the image sensor, and a color signal stored in the storage unit. Interpolation processing means for performing an interpolation process for each color signal in a direction in which a degree of correlation between a plurality of pixels near the pixel interpolation point is large for a pixel interpolation point near the corresponding pixel. Imaging device. 2. The interpolation processing means calculates a vertical, horizontal, or diagonal correlation degree of a pixel in the vicinity of the pixel interpolation point, and performs an interpolation process in a direction in which the calculated correlation degree is large. The imaging device according to claim 1, wherein: 3. The method according to claim 1, wherein when the straight line passing through the pixel interpolation point includes a direction having no pixel information, the interpolation processing means determines the pixel interpolation point in a direction in which a plurality of pixels are continuous with the pixel interpolation point interposed therebetween. The image pickup apparatus according to claim 1, wherein the interpolation processing is performed. 4. Pixels of three primary colors are arranged in a matrix, and red and blue pixels having the same number of pixels are interposed between green pixels arranged in a checkered pattern in the matrix, respectively. The color signals output from each pixel of the image sensor that is arranged in a zigzag pattern for each three pixels are fetched and stored, and a pixel interpolation point in the vicinity of the pixel corresponding to the stored color signal is obtained. And performing an interpolation process for each color signal in a direction in which the degree of correlation between a plurality of pixels near the pixel interpolation point is large. 5. The method according to claim 1, wherein a degree of correlation in a vertical direction, a horizontal direction, or an oblique direction of a pixel near the pixel interpolation point is calculated, and an interpolation process is performed in a direction in which the calculated degree of correlation is large. 5. The method for processing a color image signal according to item 4. 6. When a direction including no pixel information is included on a straight line passing through the pixel interpolation point, the pixel interpolation point is interpolated in a direction in which a plurality of pixels are continuous across the pixel interpolation point. 5. The method of processing a color image signal according to claim 4, wherein: