JPH06133320A - Image pickup signal processor - Google Patents

Abstract

PURPOSE:To provide the image pickup signal processor in which a memory capacity of a frame memory is not increased even when picture elements are deviated spatially. CONSTITUTION:Primary color signals R2, G2, B2 before interpolation are stored in a frame memory 16. A compression processing circuit 61 implements the compression processing. An expansion processing circuit 63 forms low frequency primary color signals R2L, G2L, B2L and high frequency primary color signals R2H, G2H, B2H corresponding to the primary color signals R2, G, B2. An interpolation circuit 64 regards the high frequency primary color signals R2H, G2H, B2H as a high frequency signal Y2 of a luminance signal to increase number of picture elements of the high frequency signal Y2H of the luminance signal and to increase number of picture elements of the low frequency primary color signals R2L, G2L, B2L. The low frequency primary color signals R2L, G2L, B2L whose picture element number is increased and the high frequency signal Y2H of the luminance signal whose picture element number is increased are added for each relevant picture element and primary color signals R3, G3, B3 are formed after interpolation.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup processing apparatus suitable for application to, for example, a high resolution electronic still camera.

[0002]

2. Description of the Related Art Conventionally, a spatial pixel shift method has been adopted in order to increase the resolution of an image with a relatively small number of image pickup devices.

This spatial pixel shifting method will be described by taking a so-called two-plate monochrome electronic still camera having two image pickup devices such as CCDs in which pixels are arranged vertically and horizontally.

In this example, the pixels forming the two image pickup devices are spatially displaced by 1/2 pixel.

In FIG. 7A, a digital signal of an output signal of a pixel forming one of the two image pickup elements arranged to be shifted by 1/2 pixel is a luminance signal YA and the other image pickup element is formed. When the digital signal of the output signal of the pixel to be processed is the luminance signal YB, the arrangement of the luminance signals YA and YB on the virtual screen 1 is shown.

Then, in the layout shown in FIG. 7A,
The luminance signal of the pixel in the blank portion 2 surrounded by the luminance signal YA and the luminance signal YB is created by the interpolation processing. When the luminance signal of the pixel of the blank portion 2 is the luminance signal Y, the luminance signal Y is, for example, the luminance signal YA of the pixels on the top, bottom, left and right,
For example, it is created from YB as shown in Expression 1.

[0007]

## EQU1 ## Y = {YA (upper) + YA (lower) + YB (left) +
YB (right) / 4

FIG. 7B shows the arrangement of the brightness signals of the entire virtual screen 1 formed by thus creating the brightness signals Y of the pixels in the other blank areas.

In this way, the resolution of the two-chip CCD camera is improved by the spatial pixel shift method. That is, in the example shown in FIGS. 7A and 7B, the number of pixels of the image pickup element for one plate is increased four times.

In the conventional electronic still camera, the interpolation processing circuit performs the above-described interpolation processing,
Pixel data with an increased number of data is stored in a built-in frame memory.

In order to save the pixel data stored in the frame memory, when the data is transferred to a so-called RAM card or the like as an external storage device, a discrete cosine transform (hereinafter referred to as necessary) is performed. (Hereinafter referred to as DCT transformation), quantization is performed, and then Huffman coding is performed to compress data. And R
The compressed image data is stored in the AM card.

[0012]

However, in the conventional image pickup processing apparatus such as the electronic still camera described above, the memory capacity of the frame memory for storing pixel data is four times as large as that in the case where the spatial pixel shift is not performed. There was a problem that it would increase. Further, since the image data is compressed after the interpolation processing, there is a problem that the time required for the image data compression processing is four times as long as that required when the interpolation processing is not performed. .

The present invention has been made in view of the above problems, and provides an image pickup signal processing apparatus in which the memory capacity of the frame memory does not increase and the time required for compression does not increase even if spatial pixel shifting is performed. The purpose is to do.

[0014]

According to the first aspect of the present invention, for example, as shown in FIG. 1, a green (G) image pickup device 12 in which pixels are arranged vertically and horizontally, and the G image pickup device 12 are configured. Red (R) having pixels spatially displaced by 1/2 pixel in the vertical and horizontal directions from the target pixel
And blue (B) image pickup devices 11 and 13 and output signals R1 and G of the R, G, and B image pickup devices 11 to 13
1 and B1 are digitized to obtain primary color signals R2 and G
2, the signal processing circuit 15 for converting into B2, and the primary color signal R
2, G2, B2, and the primary color signals R2, G2, B2 are read from the frame memory 16 and pixel interpolation processing is performed to perform post-interpolation primary color signals R3, G3, B.
3 and an interpolation circuit 18 that forms
The supplied primary color signals R2, G2 and B2 are converted into the low frequency primary color signal R
2L, G2L, B2L and high frequency primary color signals R2H, G2H,
B2H, of which high-frequency primary color signals R2H, G2
H and B2H are regarded as the high frequency signal of the luminance signal, pixel interpolation processing of the high frequency signal is performed, and the high frequency signal of the luminance signal Y2H
The number of pixels of the low-frequency primary color signals R2L, G
After the pixel interpolation processing is performed between 2L and B2L to increase the number of pixels of the low-frequency primary color signal, the low-frequency primary color signal with the increased number of pixels and the high-frequency signal of the luminance signal with the increased number of pixels are generated. A predetermined calculation is performed for each corresponding pixel and the interpolated primary color signal R3
G3 and B3 are formed.

The second aspect of the present invention is, for example, as shown in FIG. 5, a green (G) image pickup device 12 in which pixels are arranged vertically and horizontally.
And an image for red (R) and blue (B) having pixels that are spatially displaced by 1/2 pixel in the vertical and horizontal directions with respect to the pixels forming the G image pickup device 12 Elements 11, 13 and a signal processing circuit 15 for digitizing the output signals of the R, G, B image pickup elements 11 to 13 to convert them into primary color signals, and primary color signals R2, G.
2 and B2, and a compression processing circuit for compressing the primary color signals R2, G2 and B2 stored in the frame memory 16 into at least a discrete cosine conversion process to convert the compressed primary color signal Da. 61, a storage medium 62 for storing the compressed primary color signal Da, a compressed primary color signal Da read from the storage medium 62, and an expansion process including at least an inverse discrete cosine transform process to perform the primary color signals R2, G2. Low-pass primary color signals R2L, G corresponding to B2
2L, B2L and high-pass primary color signals R2H, G2H, B2H, expansion processing circuit 63, and low-pass primary color signals R2L,
G2L, B2L and high frequency primary color signals R2H, G2H, B2H
And an interpolating circuit 64 for forming post-interpolation primary color signals R3, G3, B3, the interpolating circuit 64 including the low-pass primary color signals R2L, G2L, B2L and the high-pass primary color signal R2H,
Of G2H and B2H, high frequency primary color signals R2H, G2H,
B2H is regarded as a high-frequency signal of the luminance signal, pixel interpolation processing of the high-frequency signal is performed to increase the number of pixels of the high-frequency signal Y2H of the luminance signal, and low-frequency primary color signals R2L, G2
Pixel interpolation processing is performed between L and B2L to obtain the low-frequency primary color signal R2.
After increasing the number of pixels of L, G2L, and B2L, the low-frequency primary color signals R2L, G2L, and B2L with the increased number of pixels and the high-frequency signal Y2H of the luminance signal with the increased number of pixels are provided for each corresponding pixel. Preliminarily calculated and interpolated primary color signals R3, G3,
B3 is formed.

In a third aspect of the present invention, the high frequency signal Y2H of the luminance signal having the increased number of pixels is converted into the high frequency primary color signals R2H and G2.
Of the H and B2H, the high-frequency primary color signal G2H and the high-frequency primary color signal R2H are formed.

In the fourth aspect of the present invention, a card-shaped storage medium 62 is used as the storage medium.

[0018]

According to the first aspect of the present invention, the output signals R of the R, G, and B image pickup devices 11 to 13 in which the spatial pixel shift is adopted.
1, G1, B1 digitized primary color signals R2, G
2, B2 are stored in the frame memory 16 before interpolation. Primary color signal R read from the frame memory 16
2, G2, B2 are low-pass primary color signals R2L, G2L, B2L and high-pass primary color signals R2H, G2 in the interpolation circuit 18.
H and B2H, of which the high-frequency primary color signal R2
H, G2H, and B2H are regarded as high-frequency signals of the luminance signal, pixel interpolation processing of the high-frequency signals is performed to increase the number of pixels of the high-frequency signal Y2H of the luminance signals, and low-frequency primary color signal R2L. , G2L, B2L are subjected to pixel interpolation processing to increase the number of pixels of the low-frequency primary color signals R2L, G2L, B2L, and then the low-frequency primary color signals R2L, G2L, B2L and the number of pixels of which the number of pixels are increased. Is calculated for each pixel corresponding to the high-frequency signal Y2H of the increased luminance signal to form post-interpolation primary color signals R3, G3, B3.

Therefore, the memory capacity of the frame memory becomes smaller than that of the conventional technique in which the interpolation is performed and then the data is stored in the frame memory. Specifically, the memory capacity when the pixel shift is not performed is sufficient. Further, the resolution of the image represented by the primary color signal after interpolation can be increased.

According to the second aspect of the present invention, the digitized primary color signals R2, G of the output signals R1, G1, B1 of the R, G, B image pickup devices 11-13 for which the spatial pixel shift is adopted.
2, B2 are stored in the frame memory 16 before interpolation. Primary color signal R read from the frame memory 16
2, G2, B2, in the compression processing circuit 61,
A compression process including at least a discrete cosine transform process is performed, and the compressed primary color signal Da is stored in the storage medium 62. Then, the decompression circuit 63 performs decompression processing including at least inverse discrete cosine transform processing on the compressed primary color signal Da read from the storage medium 62, and the low-pass primary colors corresponding to the primary color signals R2, G2, B2. Signal R2
L, G2L, B2L and high frequency primary color signals R2H, G2H, B
2H is formed. In the interpolation circuit 64, the low-pass primary color signals R2L, G2L, B2L and the high-pass primary color signals R2H, B
Of the 2H and G2H, the high frequency primary color signals R2H, G2H and B
2H is regarded as the high-frequency signal Y2H of the luminance signal, pixel interpolation processing of the high-frequency signal Y2H is performed to increase the number of pixels of the high-frequency signal Y2H of the luminance signal, and the low-frequency primary color signals R2L and G2L. , B2L is subjected to pixel interpolation processing to increase the number of pixels of the low-frequency primary color signals R2L, G2L, and B2L, and then the low-frequency primary color signal R2 with the increased number of these pixels.
L, G2L, B2L and the high-frequency signal Y2H of the luminance signal with the increased number of pixels are subjected to a predetermined calculation for each corresponding pixel to form interpolated primary color signals R3, G3, B3.

Therefore, the memory capacity of the frame memory becomes smaller than that of the conventional technique in which the frame memory is interpolated and then stored in the frame memory. Specifically, the memory capacity when the pixel shift is not performed is sufficient. Moreover, since the compression processing is performed before the interpolation processing, the compression processing time does not increase as compared with the conventional technique of performing the compression processing after the interpolation processing. Further, since the compressed primary color signals are stored in the storage medium, the memory capacity of the storage medium can be small. Furthermore, when decompressing the compressed data, the high-pass primary color signal and the low-pass primary color signal are separated, so that separate low-pass filters and high-pass filters are not required for separation. Furthermore, the resolution of the image represented by the primary color signal after interpolation can be increased.

According to a third aspect of the present invention, in the first or second aspect of the present invention, the high-frequency signal of the luminance signal with the increased number of pixels is the high-frequency primary color of G among the high-frequency primary color signals. The signal and the R high-frequency primary color signal are formed. Therefore, the interpolation process becomes relatively simple.

According to a fourth aspect of the present invention, in the second or third aspect of the present invention, the storage medium is a card-shaped storage medium. Therefore, handling becomes easy.

[0024]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the image pickup processing apparatus of the present invention will be described below with reference to the drawings. In the drawings referred to below, the same reference numerals are given to those corresponding to those shown in FIG.

FIG. 1 shows a schematic configuration of an image pickup processing apparatus according to an embodiment of the present invention.

In FIG. 1, light having image information incident on the lens 10 is decomposed by a prism (not shown), and a red channel image pickup device (hereinafter referred to as R image pickup device) 11 such as a CCD image pickup device. , Green channel image sensor (hereinafter referred to as G image sensor)
12 and a blue channel image pickup device (hereinafter, referred to as B image pickup device) 13.

Each of the R, G, and B image pickup devices 11 to 13 has a structure in which, for example, pixels are arranged vertically and horizontally, and each pixel forming the G image pickup device 12 is used for R. And the pixels forming the B image pickup elements 11 and 13 are spatially shifted by 1/2 pixel in the vertical direction and the horizontal direction.

The image pickup signals R1, G1 and B1 which are output signals of the R, G and B image pickup devices 11 to 13 are supplied to a known image pickup signal processing circuit 15. This imaging signal processing circuit 1
In FIG. 5, well-known signal processing such as γ correction processing is performed, and the digital signals are converted into digital signals by the A / D conversion circuit (not shown). 16 are stored. The frame memory 16 receives the primary color signals R2, G2, B2 in real time.

The primary color signals R2, G2, B2 output from the frame memory 16 are supplied to a low-frequency high-frequency separation processing circuit 20 which constitutes an interpolation circuit 18.

The interpolation circuit 18 includes the frame memories 23 to 28 and the synthesizing circuit 3 in addition to the low and high frequency separation processing circuit 20.
0, addition circuits 31 to 33, and coefficient circuits 35 to 37.

FIG. 2 shows the operation of the low-frequency / high-frequency separation processing circuit 20.

First, as shown in FIG. 2A, the primary color signal G2
Pixel data block 5 of 8 × 8 pixel data among the pixels
The DCT conversion process is performed every 0 to obtain a coefficient data block 52 from direct current to high frequency as shown in FIG. 2B. In FIG. 2A, all pixel data are represented by the level “G”, but the levels are not necessarily the same (usually, all are different depending on the subject). The same characters are used to avoid complexity. Similarly, in FIG. 2B, each frequency is represented by a level “F”, but this is not usually the same value.

As is well known, in FIG. 2B, the data in the upper left corner is DC coefficient data, and the data in the lower right corner is the highest frequency coefficient data. The direction of arrow P is from DC dc to the highest frequency f _max .

Here, in order to convert the primary color signal G2 into a primary color signal of a low frequency component of the spatial frequency (hereinafter, referred to as a low frequency primary color signal) G2L (see FIG. 2E), as shown in FIG. 2C. , The coefficient data block 53 is created by replacing the upper left half data including the DC coefficient data with the level “0 (zero)”. Then, the inverse discrete cosine transform (I
By performing the DCT), as shown in FIG. 2E, the pixel data block 54 of the low-pass primary color signal G2L corresponding to the pixel data block 50 for 8 × 8 pixel data (see FIG. 2A).
Is obtained.

On the other hand, in order to convert the primary color signal G2 into a primary color signal of a high frequency component of the spatial frequency (hereinafter referred to as a high frequency primary color signal) G2H (see FIG. 2F), as shown in FIG. 2D, Of the pixel data block 52 (see FIG. 2B), the lower right half data including the highest frequency coefficient data is replaced with the level “0 (zero)” to create the coefficient data block 55. Then, the inverse discrete cosine transform (I
By performing the DCT), as shown in FIG. 2F, the pixel data block 56 of the high frequency primary color signal G2H corresponding to the pixel data block 50 (see FIG. 2A) for 8 × 8 pixel data.
Is obtained.

In this way, the low-frequency / high-frequency separation processing is performed on the remaining pixel data other than the 8 × 8 pixel data.

Similarly, the primary color signals R2 and B2 other than the primary color signal G2 are also separated into low frequency primary color signals R2L and B2L and high frequency primary color signals R2H and B2H.

The low-frequency primary color signal R separated in this way
2L, G2L, B2L and high frequency primary color signals R2H, G2
H and B2H are stored in the frame memories 23 to 28, respectively.

Low-pass primary color signals R2L, G2L, B2L (pixel data levels are represented by "RL", "GL", "BL", respectively) and high-pass signals stored in the frame memories 23-28. Primary color signals R2H, G2H, B2H
(The pixel data levels R are "RH" and "G," respectively.
H "," BH ") virtual screen 1 (1a-1)
f) The arrangement above is shown corresponding to FIGS. 3A-3E. In addition,
In order to avoid complication, only 4 × 4 pixel data is drawn.

Low-pass primary color signal GL2 and high-pass primary color signal G
Low-pass primary color signals R2L,
It can be seen that the pixel data positions of B2L and the high-frequency primary color signals R2H and B2H are vertically and horizontally displaced by 1/2 pixel (pitch).

Therefore, the operation of creating the pixel data of all the pixel portions in the virtual screens 1a to 1f by an example of the interpolation process will be described below.

In this case, first, the high-frequency primary color signals R2H, G
2H and B2H are regarded as high frequency components (high frequency signals) of the luminance signal. The reason for this is that the characteristics of the human eye are such that it is difficult to distinguish colors for components with high spatial frequencies.

High frequency primary color signal R2 shown in FIGS. 3D to 3F.
When H, G2H, and B2H are drawn on one virtual screen 1g in an overlapping manner, they are represented as shown in FIG. 4A.

The pixel data of the high-frequency primary color signals R2H, G2H, and B2H read from the frame memories 26 to 28 are vertically and horizontally interpolated by the synthesizing circuit 30, and the pixel data shown in FIG.
As shown in FIG. 5, a luminance signal (hereinafter, referred to as a high frequency signal of the luminance signal, if necessary) Y2H in which the level of the high frequency signal (high frequency component) having the number of pixels is “YH” is obtained.

The level “YH” of the high frequency signal is formed based on, for example, the equation 1 or the equation 2.

[0046]

## EQU00002 ## YH1 = 0.5 GH + 0.35 RH + 0.15 BH

[0047]

[Formula 3] YH2 = 0.5GH + 0.5RH

In the case of Equation 1, the level YH of each pixel data of the luminance signal Y2H from the three primary color pixel data of R, G, B
1 is formed, and in the case of Expression 2, the level YH2 of each pixel data of the luminance signal Y2H is formed from the primary color data of the two colors R and G which are relatively sensitive to human eyes. Is. In practical use, using Equation 2 is often sufficient. In that case, in FIG. 1, the frame memory 2
8 is unnecessary, the calculation time in the synthesis circuit 30 is shortened, and the hardware and / or software of the synthesis circuit 30 is correspondingly simplified. When the formula 1 is used, the color can be reproduced more faithfully.

The pixel data of the luminance signal Y2H obtained by the interpolation in this way is transferred from the synthesis circuit 30 to the coefficient circuit 3
It is sequentially supplied to the other input terminals of the adder circuits 31 to 33 via 5 to 37.

On the other hand, one of the input terminals of the adder circuits 31 to 33 is supplied from each of the frame memories 23 to 25 in order to match the number of pixels of the luminance signal Y2H (see FIG. 4B).
With respect to each pixel data of the low-pass primary color signals R2L, G2L, and B2L, the same pixel data is read and supplied twice each in the vertical and horizontal directions.

Then, the levels of the pixel data at the same pixel positions on the virtual screen are added by the adder circuits 31 to 33, respectively, so that the primary color signals after interpolation processing (post-interpolation primary color signals) R3, G3, B3. Is obtained at the output side of the adder circuits 31 to 33.

The level of each pixel data of the post-interpolation primary color signals R3, G3, B3 is expressed as shown in Equation 3.

[0053]

## EQU00004 ## R3 = RL + 0.30YH G3 = GL + 0.59YH B3 = BL + 0.11YH

These interpolated primary color signals R3, G3, B3 which are digital signals are converted into interpolated primary color signals R4, G4, B4 which are analog signals by D / A conversion circuits 41-43, and output terminals 44-46. To a monitor (not shown) through which the primary color signals R2, G2, B2 recorded in the frame memory 16 are interpolated to obtain high-resolution interpolated primary color signals R3, G3, B3 (R4, G4, B4). Based on this, a still image is displayed on the monitor.

According to the embodiment shown in FIG. 1, the R, G, and B image pickup devices 11 to 13 adopting the spatial pixel shift.
The digitalized primary color signals R2, G2, B2 of the image pickup signals R1, GG1, B1 are stored in the frame memory 16 before interpolation. The primary color signals R2, G2, B2 read out from the frame memory 16 are supplied to the low-frequency primary color signal R2 in the low-frequency high-frequency separation processing circuit 20 constituting the interpolation circuit 18.
L, G2L, B2L and high frequency primary color signals R2H, G2H, B
It is divided into 2H. Among them, the high frequency primary color signals R2H,
G2H and B2H are regarded as the high frequency signal Y2H of the luminance signal, pixel interpolation processing of the high frequency signal Y2H is performed to increase the number of pixels of the high frequency signal Y2H of the luminance signal, and the low frequency primary color signal R2L. , G2L, B2L are subjected to pixel interpolation processing to increase the number of pixels of the low-frequency primary color signals R2L, G2L, B2L, and then the low-frequency primary color signals R2L, G2L, B2L and the number of pixels of which the number of pixels are increased. And the high frequency signal Y2H of the increased luminance signal are added for each corresponding pixel to form post-interpolation primary color signals R3, G3, B3.

Therefore, after the interpolation, the frame memory 1
The memory capacity of the frame memory is reduced as compared with the conventional technique of storing the data in FIG. Specifically, the memory capacity when the pixel shift is not performed is sufficient. Further, there is an effect that the resolution of the image represented by the primary color signal after the interpolation can be increased.

FIG. 5 shows the configuration of another embodiment.
In the example of FIG. 5, the components corresponding to those of the example of FIG. 1 are designated by the same reference numerals, and detailed description thereof will be omitted. The example shown in FIG. 5 is an example applied to an electronic still camera having a removable memory card built therein, and has a recording system shown in FIG. 5A and a reproducing system shown in FIG. 5B. The playback system may be separate from the electronic still camera.

In the example of FIG. 5, compared with the example of FIG. 1, a compression processing circuit 61, a memory card 62 and an expansion circuit 63 are inserted between the frame memory 16 and the frame memories 23 to 28. A part of the low frequency band high frequency separation circuit 20 in the example of FIG. 1 is included in the expansion circuit 63. Further, in the example of FIG. 5, the interpolation circuit 64 includes the frame memories 23 to 28, the synthesizing circuit 30, and the adding circuits 31 to 31.
33 and coefficient circuits 35 to 37.

In FIG. 5, the primary color signals R2, G2, B2 output from the frame memory 16 are compressed by the compression processing circuit 6.
1 is supplied.

In the compression processing circuit 61, known discrete cosine transform (DCT) processing, quantization processing, and Huffman coding processing are performed to obtain primary color signals R2, G which are image data.
2, B2 is compressed, and the digital signal Da after the compression is processed.
Is supplied to a memory card 62 having an SRAM integrated circuit which is a card-shaped storage medium. This compression processing is a so-called JPEG compression method. The memory card 6
Instead of 2, EAROM, EEPROM, floppy disk, magneto-optical disk or hard disk may be used. When the memory card 62 is adopted, it is easy to handle.

The digital signal Da stored in the memory card 62 is expanded by the expansion processing circuit 63 and output as the digital signal Db. This digital signal Db
Is composed of the low-frequency primary color signals R2L, G2L, B2L and the high-frequency primary color signals R2H, G2H, B2H. That is, in the decompression processing circuit 63, the well-known Huffman decoding process and the inverse quantization process are performed, and then the low-frequency high-frequency separation processing process shown in FIGS. 2A to 2F is performed.

The subsequent data processing by the interpolation circuit 64 is the same as that in the example of FIG. Note that the frame memory 28 can be omitted by performing the processing shown in the above-mentioned Equation 2 in the synthesizing circuit 30.

As described above, according to the example of FIG. 2, the digitized primary color signals R of the image pickup signals R1, G1, B1 of the R, G, B image pickup elements 11 to 13 for which the spatial pixel shift is adopted.
2, G2, B2 are stored in the frame memory 16 before the interpolation processing. The primary color signals R2, G2, B2 read from the frame memory 16 are subjected to a compression process including at least a discrete cosine transform process in the compression processing circuit 61, and a digital signal Da which is the primary color signal after the compression is performed. Is stored in the memory card 62. Then, the decompression circuit 63 performs decompression processing including at least inverse discrete cosine transform processing on the digital signal Da which is the compressed primary color signal read from the memory card 62 to obtain a primary color signal corresponding to the digital signal Da. A certain digital signal Db is formed.

This digital signal Db is the low-frequency primary color signal R
2L, G2L, B2L and high frequency primary color signals R2H, G2H,
B2H. Then, in the interpolation circuit 64, it is divided into low-pass primary color signals R2L, G2L, B2L and high-pass primary color signals R2H, G2H, B2H, of which the high-pass primary color signals R2H, G2H, B2H are the high-pass signals of the luminance signal. Assuming that the pixel interpolation processing of the high frequency signal is performed to increase the number of pixels of the high frequency signal of the luminance signal, the pixel interpolation processing is performed between the low frequency primary color signals and the low frequency primary color signal After the number of pixels is increased, the low-frequency primary color signals R2L, G2L, and B2L with the increased number of pixels and the high-frequency signal Y2H of the luminance signal with the increased number of pixels are added for each corresponding pixel and interpolated. The rear primary color signals R3, G3 and B3 are formed.

For this reason, the memory capacity of the frame memory 16 becomes smaller than that of the conventional technique in which interpolation is performed and then stored in the frame memory. Specifically, the memory capacity when the pixel shift is not performed is sufficient. Moreover, since the compression processing is performed before the interpolation processing, the compression processing time does not increase as compared with the conventional technique of performing the compression processing after the interpolation processing (if the interpolation processing is performed before the compression processing is performed. Then, four times as much compression processing time is required as compared with the case where interpolation processing is not performed). Further, since the compressed primary color signals are stored in the memory card 3, the memory capacity of the memory card 3 can be small. Furthermore, when the compressed data is expanded, it is separated into the high-frequency primary color signals R2H, G2H, B2H and the low-frequency primary color signals R2L, G2L, B2L.
Separate low pass and high pass filters are not required for separation. Furthermore, the resolution of the image represented by the primary color signal after interpolation can be increased.

FIG. 6 shows the configuration of another embodiment (PAL system reproduction system) of the reproduction system shown in FIG. 5B. In FIG. 6, parts corresponding to those shown in FIG. 5B are designated by the same reference numerals, and detailed description thereof will be omitted.

In the reproducing system shown in FIG. 6, the synthesizing circuit 65 forming the interpolating circuit 67 uses the low-pass primary color signals R2L, G2L,
From B2L to the PAL color difference signal V1,
U1 and the low frequency signal Y1L of the luminance signal are formed. The addition circuit 66 adds the low frequency signal Y1L of the luminance signal and the high frequency signal Y2H of the luminance signal to form the luminance signal Y1.

Color difference signal V which is an analog signal of these
1, U1 and the luminance signal Y1 are D / A conversion circuits 41 to
At 43, color difference signals V2, U2 and a luminance signal Y2 which are digital signals are formed. The PAL-type interpolating circuit 67 (frame memories 23 to 28, synthesizing circuits 30 and 65, and adding circuit 66) shown in FIG. Can be replaced by

Further, the present invention is not limited to the above-mentioned embodiments, and it goes without saying that various configurations can be adopted without departing from the gist of the present invention.

[0070]

As described above, according to the first aspect of the present invention, the digitized primary color signals of the output signals of the R, G and B image pickup elements in which the spatial pixel shift is adopted are stored in the frame memory. It is stored before interpolation. The primary color signal read from the frame memory is divided into a low-pass primary color signal and a high-pass primary color signal by the interpolation circuit, and the high-pass primary color signal is regarded as the high-pass signal of the luminance signal, After the pixel interpolation processing of the signal is performed to increase the number of pixels of the high frequency signal of the luminance signal, and the pixel interpolation processing between the low frequency primary color signals is performed to increase the number of pixels of the low frequency primary color signal. The low-pass primary color signal having the increased number of pixels and the high-pass signal of the luminance signal having the increased number of pixels are subjected to a predetermined calculation for each corresponding pixel to form an interpolated primary color signal.

Therefore, it is possible to obtain the effect that the memory capacity of the frame memory is reduced as compared with the conventional technique in which the interpolation is performed and the result is stored in the frame memory. In particular,
It is possible to obtain the effect of improving the memory capacity when the pixel shift is not performed. Further, there is an effect that the resolution of the image represented by the primary color signal after the interpolation can be increased.

According to the second aspect of the present invention, the digitized primary color signals of the output signals of the R, G, and B image pickup elements in which the spatial pixel shift is adopted are stored in the frame memory before interpolation. In the compression processing circuit, compression processing including at least discrete cosine transform processing is performed on the primary color signal read from the frame memory, and the compressed primary color signal is stored in the storage medium. And by the expansion circuit,
A decompression process including at least an inverse discrete cosine transform process is performed on the compressed primary color signal read from the storage medium to form a low frequency primary color signal and a high frequency primary color signal corresponding to the primary color signal. In the interpolation circuit, the high-frequency primary color signal of the low-frequency primary color signal and the high-frequency primary color signal is regarded as the high-frequency signal of the luminance signal, and pixel interpolation processing is performed on the high-frequency signal to perform the high-frequency signal of the luminance signal. As the number of pixels of the signal is increased, pixel interpolation processing is performed between the low-pass primary color signals to increase the number of pixels of the low-pass primary color signal, and The high frequency signal of the luminance signal with the increased number of pixels is added for each corresponding pixel to form the interpolated primary color signal.

Therefore, it is possible to obtain the effect that the memory capacity of the frame memory is reduced as compared with the conventional technique of storing in the frame memory after the interpolation. In particular,
The memory capacity can be improved when pixel shifting is not performed. In addition, since the compression processing is performed before the interpolation processing, there is an advantage that the compression processing time does not increase as compared with the conventional technique of performing the compression processing after the interpolation processing. Further, since the compressed primary color signals are stored in the storage medium, the effect that the memory capacity of the storage medium is small can be obtained. Furthermore, when decompressing the compressed data, it is separated into a high-pass primary color signal and a low-pass primary color signal, so there is an effect that a separate low-pass filter or high-pass filter is not required for separation. To be Furthermore, there is an effect that the resolution of the image represented by the primary color signal after interpolation can be increased.

According to a third aspect of the present invention, in the first or second aspect of the present invention, the high-frequency signal of the luminance signal with the increased number of pixels is the high-frequency primary color of G among the high-frequency primary color signals. The signal and the R high-frequency primary color signal are formed. Therefore, it is possible to obtain an effect that the interpolation process is relatively simple.

According to a fourth aspect of the present invention, in the second or third aspect of the present invention, the storage medium is a card-shaped storage medium. Therefore, it is possible to obtain an effect of easy handling.

[Brief description of drawings]

FIG. 1 is a diagram showing a configuration of an embodiment of an image pickup processing device of the present invention.

FIG. 2 is a diagram used for explaining an operation of a low frequency band high frequency separation processing circuit in the example of FIG.

FIG. 3 is a diagram used for explaining the operation of the interpolation circuit in the example of FIG.

FIG. 4 is a diagram provided for explaining the operation of the interpolation circuit in the example of FIG.

FIG. 5 is a diagram showing the configuration of another embodiment of the image pickup device of the present invention.

FIG. 6 is a diagram showing the configuration of another example of the reproduction system in FIG.

FIG. 7 is a diagram provided for explaining an operation of a conventional technique.

[Explanation of symbols]

 11-13 R, G, B image sensors, R2, G2, B2 primary color signals 16, 23-28 frame memory 18, 64 interpolation circuit 20 low-frequency high-frequency separation processing circuit R2L, G2L, B2L low-frequency primary-color signal Y2H High frequency signal of luminance signal R3, G3, B3 Primary color signal after interpolation

Claims (4)

[Claims] 1. A green (G) image pickup device in which pixels are arranged vertically and horizontally, and is spatially displaced by 1/2 pixel in the vertical and horizontal directions with respect to the pixels forming the G image pickup device. A red (R) and blue (B) image pickup device having arranged pixels; a signal processing circuit for digitizing output signals of the R, G, B image pickup devices to convert them into primary color signals; A frame memory that stores a primary color signal, and an interpolation circuit that reads the primary color signal from the frame memory and performs pixel interpolation processing to form a post-interpolation primary color signal are provided. The number of pixels of the high frequency signal of the luminance signal is divided into the low frequency primary color signal and the high frequency primary color signal, of which the high frequency primary color signal is regarded as the high frequency signal of the luminance signal and the pixel interpolation processing of the high frequency signal is performed. With increasing After the pixel interpolation processing is performed between the low-frequency primary color signals to increase the number of pixels of the low-frequency primary color signal, the low-frequency primary color signal of which the number of pixels is increased and the high-frequency signal of the luminance signal of which the number of pixels is increased. An image pickup signal processing apparatus, characterized in that a predetermined calculation is performed for each corresponding pixel to form the post-interpolation primary color signal. 2. A green (G) image pickup device in which pixels are arranged vertically and horizontally, and is spatially displaced by 1/2 pixel in the vertical and horizontal directions with respect to the pixels forming the G image pickup device. A red (R) and blue (B) image pickup device having arranged pixels; a signal processing circuit for digitizing output signals of the R, G, B image pickup devices to convert them into primary color signals; A frame memory that stores a primary color signal, a compression processing circuit that converts the primary color signal stored in the frame memory into a compressed primary color signal by performing a compression process including at least a discrete cosine conversion process, and the compressed primary color signal A storage medium for storing, and a low-pass primary color signal and a high-pass primary color signal corresponding to the primary color signal by performing a decompression process including at least an inverse discrete cosine transform process by reading the compressed primary color signal from the storage medium. And a low-pass primary color signal and a high-pass primary color signal are supplied, and an interpolation circuit that forms a post-interpolation primary color signal is provided. Of the high-frequency primary color signals, the high-frequency primary color signal is regarded as the high-frequency signal of the luminance signal, pixel interpolation processing is performed on the high-frequency signal to increase the number of pixels of the high-frequency signal of the luminance signal, and After performing pixel interpolation processing between primary color signals to increase the number of pixels of the low frequency primary color signal, the low frequency primary color signal with an increased number of these pixels and the high frequency signal of the luminance signal with an increased number of pixels are corresponded An image pickup signal processing device, characterized in that a predetermined calculation is performed for each pixel to form the interpolated primary color signal. 3. A high-frequency signal of a luminance signal having an increased number of pixels is formed from a G high-frequency primary color signal and an R high-frequency primary color signal of the high-frequency primary color signals. The imaging signal processing device according to claim 1 or 2, characterized in that 4. The image pickup signal processing device according to claim 1, wherein the storage medium is a card-shaped storage medium.