JPH07131800A - Video signal processing unit - Google Patents

Abstract

PURPOSE:To improve the resolution by establishing shift of picture elements for each color signal so as to eliminate a reflected component included in each color signal outputted from a solid-state image pickup element. CONSTITUTION:Color signals R,G,B outputted from a three-board type solid-state image pickup elements for each color are fed respectively to terminals a,b,a of switch circuits 32R, 32G, 32B of a picture element shift correction section 31. Furthermore, a value ''0'' is set in common to terminals b, a, b of the circuits 32R, 32G, 32B. Through the changeover of a clock CK with a frequency equal to an operating frequency of the solid-state image pickup element, '0' is inserted between picture elements of each color signal. Thus, color signals R', G', B' are obtained by correcting picture element shift among the color signals R, G, B. Then the signals R', G', B' are fed to a reflected component detection section 34, in which a reflected component is eliminated by using a reflected component signal H. Thus, a color signal without picture element shift is obtained.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention comprises at least two solid-state image sensors each having a large number of light-receiving elements arranged in a matrix. A color television camera using a plurality of color signals output from a solid-state image pickup device in which spatial pixel shifts are performed by arranging them so that they are displaced from each other by approximately half in the main scanning direction, and in particular, a three-plate solid-state image pickup element employing the spatial pixel shift method The present invention relates to a video signal processing device that processes a plurality of color signals output from the.

[0002]

2. Description of the Related Art For example, in a three-panel color television camera, in order to obtain a high-resolution video signal with a solid-state image sensor having a small number of pixels, a red color is obtained with reference to the spatial position of the green (G) solid-state image sensor. A spatial pixel shift method is used in which the positions of the solid-state image pickup devices for (R) and blue (B) are shifted in the main scanning direction (horizontal direction) by half the pixel arrangement pitch.

FIG. 1 shows an example of the structure of an image pickup section of a three-panel color television camera which adopts the above-described spatial pixel shift method. Incident light from a subject is split into three primary color lights of R, G, and B by a color separation prism 1, and optical images of the respective colors are received by three solid-state image pickup devices 2R, 2G, and 2B, and red signals are obtained. It is configured to output R, green signal G and blue signal B. Figure 2 shows these three
The solid-state image pickup elements 2R, 2G, and 2B each show a spatial arrangement relationship of pixels, and the pixels of the solid-state image pickup element 2R for red and the solid-state image pickup element 2B for blue are the same as those of the solid-state image pickup element 2G for green. The pixels are arranged so as to be shifted in the horizontal direction by half the pixel pitch P. Regarding the reading of this solid-state image pickup device, as disclosed in Japanese Patent Laid-Open No. 52-129321, a method of shifting the read clock pulse corresponding to the dislocation of the solid-state image pickup devices for red, green and blue, There is a method of reading with the same clock pulse. The luminance signal Y is created by mixing the color signals output from these solid-state image pickup devices 2R, 2G, and 2B at a prescribed ratio in the encoder matrix processing unit, and the pixels corresponding to the respective colors are as shown in FIG. Therefore, the number of pixels is apparently increased, and the luminance signal Y with improved resolution can be obtained. The mixing ratio when these color signals R, G, B are mixed to create the luminance signal Y is defined as the following equation in the case of the NTSC system, for example.

[Equation 1] Y = 0.3R + 0.59G + 0.11B (1)

[0004]

In the three-panel color television camera adopting the pixel shifting method as described above, the luminance signal Y to be output has a wavy response as shown in FIG. There is a drawback that a small amount of aliasing component is generated, so that the resolution is not so high and the frequency characteristic is deteriorated. In addition, each color signal R,
In G and B, since the signal output from the solid-state image sensor is output as it is, many aliasing components are generated, and the resolution is only the number of pixels of the solid-state image sensor. Further, for each color signal, a pixel shift causes a registration error, which cannot be used as it is.

[0005] Here, the aliasing component, if the operating frequency of the solid-state image pickup device and f _s, the frequency 0.5 f _s or more high frequency components folded around the 0.5 f _s to the following lower frequency band 0.5 f _s It is returned. When the subject having the frequency characteristic as shown in FIG.
Signals having frequency characteristics as shown in FIGS. 6 and 7 are output from the G, red, and blue solid-state imaging devices 2R and 3B, respectively. Although spectrum in these figures are wavy large, which represents aliasing components, frequency 0.5 f _s or more high frequency components aliasing low-pass, is because it is added to the low frequency component . As described above, each color signal output from the three-panel color television camera adopting the pixel shift method includes many folding components.

The phase of the folding component is the green signal G
And the red and blue signals R and B have a phase of FIG.
And as shown in FIG. 7, they are shifted by 180 degrees. This is because in the spatial pixel shifting method, the pixels of the red and blue solid-state image pickup devices 2R and 2B are located at intermediate positions between the pixels of the green solid-state image pickup device 2G as shown in FIG. Is. For example, in the case of capturing an image of a subject that gives a high-frequency signal as shown in FIG. 8, the green solid-state image sensor 2G images the position of the mark ●, and the red and blue solid-state image sensors 2R and 2B. Since the image is taken at the position of the X mark, the green signal, the red signal, and the blue signal have low-frequency waveforms as shown in FIGS. 9 and 10, respectively, and the green signal waveform and the red and blue signal waveforms are different from each other. Will be 180 degrees out of phase.

When the luminance signals Y are created by mixing the color signals R, G and B as described above at the ratios described above, most of the aliasing components are inverted in phase between the green signal and the red and blue signals. However, since the mixing ratio of the color signals when creating the luminance signal is not 1: 1 for the green signal and the red and blue signals, it is folded back as shown in the following equation (2). Some components will remain (see Figure 4). For example, in the case of the NTSC system, 18% of the folding component remains as shown in Expression (3).

[Formula 2] Remaining amount of folding component = Green mixing ratio-Red mixing ratio-Blue mixing ratio (2) Remaining amount of folding component = 0.59-0.3 -0.11 = 0.18 (3)

Further, in the case of a high quality television system, the luminance signal Y is mixed according to the following equation (4), so that as many folding components as 40% remain as shown in equation (5). It will be.

[Formula 3] Y = 0.701G + 0.087B + 0.212R (4) Remaining amount of folding component = 0.701-0.087-0.212 = 0.402 (5)

Color signals G, R, and B of FIGS. 9 and 10.
When the luminance signal Y is created by mixing with the mixing ratio shown in Expression (1), a signal as shown in FIG. 11 is obtained. When the waveform of this luminance signal is decomposed into low-frequency components and high-frequency components,
As shown in FIG. 12, it is divided into a low frequency band Y _L whose phase and frequency are close to the waveform of the green signal, and a high frequency band Y _H whose frequency is the same as in FIG. That is, the waveform obtained by combining these two waveforms becomes the luminance signal Y shown in FIG. In the case of such an image, the low-frequency part is more noticeable than the high-frequency part, so that there is a drawback that the apparent resolution is lowered. As described above, in the video signal processing device of the three-panel color television camera adopting the conventional spatial pixel shift method, the aliasing component is not completely removed, and the resolution is improved by the amount of the aliasing component remaining. There are no drawbacks.

Further, the conventional video signal processing device has a problem that the frequency characteristic of the luminance signal Y is deteriorated with respect to the frequency characteristic of the subject as shown in FIG.
This will be described below. The spatial arrangement of the pixels of the solid-state image sensor of each color channel by the pixel shift method and the luminance signal Y obtained by mixing these color signals according to the above equation (1) is shown in FIG. As shown. As shown in FIG. 2, a pixel for green color,
Since the pixel for red and the pixel for blue are deviated by a half of the pixel pitch, it is necessary to mix them so as not to shift the positional relationship when the luminance signal Y is created. As described above, since the luminance signal Y shifts the positions of the red and blue solid-state image pickup devices 2R and 2B with respect to the position of the green solid-state image pickup device 2G by half the pixel pitch, and mixes them according to the equation (1). Since the number is twice that of the solid-state image sensor alone, the resolution is improved. Here, the number of pixels of each color signal is substantially doubled, but the interpolation pixel (pixels shown in parentheses in FIG. 13) at that time is the same as the previous pixel as seen in the scanning direction. It becomes the pixel level. Such an interpolation method is known as a zero-order hold method. The frequency characteristic is deteriorated by the zero-order hold method. Since the luminance signal Y is created by mixing the color signals whose frequency characteristics have deteriorated by the zero-order hold method, the frequency characteristics of the luminance signal Y also deteriorate as shown in FIG. In the conventional video signal processing device that processes the color signals output from the three-panel color television camera that employs the spatial pixel shift method as described above, the number of pixels of each color signal is substantially equal when the luminance signal Y is created. To 2
However, since the interpolation is performed by the zero-order hold method, there is a disadvantage that the frequency characteristic is inevitably deteriorated.

Next, the resolution of the color signals R, G, B output from the solid-state image pickup device for each color and the problem of registration error will be described. FIG. 14 shows an example of the configuration of a conventional video signal processing device. The green signal G output from the solid-state image pickup device for each color is delayed by the delay 8
The signals delayed by 1/2 pixel pitch and the red and blue signals R and B are supplied to the Y, I and Q matrixes 3 to generate the luminance signal Y and the color difference signals I and Q. The luminance signal Y is supplied to the encoder 6 through the filters 4 and 5 and the encoded color television signal ENC is generated, and the encoded color television signal ENC is supplied to the output terminal 7. When the solid-state image pickup devices for red, green, and blue are driven by clock pulses having the same phase, video signals can be obtained with the phase shown in FIG. 17A. This means that the phase of the green signal G is shifted from the phase of the red and blue signals R and B by an amount corresponding to 1/2 pixel pitch with respect to the arrangement of the solid-state image sensor as shown in FIG. . This is a registration error. In the pixel shift method, the red, green, and blue signals R, G, and B are mixed based on, for example, the equation (1) to create a luminance signal Y, thereby improving the resolution of the luminance signal. . That is, since the pixel shift is established in the matrix 3, the video signal processing apparatus shown in FIG. 14 can obtain only the resolution of the solid-state image sensor for each color signal output, and is a signal including many foldings. . Here, in order to extract each color signal without causing a registration error, the output of the delay 8 may be extracted for the green signal. This is a case of taking out as an analog signal. However, there is a problem in extracting it as a digital signal. If the operating frequency of the solid-state image sensor is f _s , the sampling frequency for digital signals is also f _s, but only the green signal is delayed 8.
If you sample the signal after that, the clock signal for the green signal will be the same as the clock signal for the red and blue signals.
The phase must be shifted by an amount equivalent to 1/2 pixel pitch. If the green signal is taken out from before the delay 8 and sampled, this problem does not occur, but a registration error occurs between the color signals. In this case, it suffices to have a digital filter for shifting the phase for the green signal output, but there is a disadvantage that the number of circuits is increased because of that.

In order to eliminate such a defect, FIG.
The video signal processing device shown in FIG. In this video signal processing device, output from luminance signals Y, I and Q filters 4 and 5 of Y, I and Q matrix 3 for producing luminance signal Y and color difference signals I and Q from respective color signals R, G and B. The color difference signals I and Q are supplied to the dematrix 10 so as to obtain the respective color signals R ', G', B '. However, in such a conventional video signal processing device, the color signals R ', B', G'output to the color signal output terminals 9R, 9G, 9B have some folding components. That is, since the luminance signal Y having the deteriorated frequency characteristic is decoded to create the color signal, it is considered that the frequency characteristic is deteriorated because there is some folding component remaining, though not more than the method of FIG. There is a drawback.

An object of the present invention is to eliminate the above-mentioned drawbacks of the conventional video signal processing device, remove the aliasing component contained in each color signal R, G, B output from the solid-state image sensor,
It is possible to improve the resolution, output the color signals R, G, B without pixel shift, so-called registration error, and prevent the deterioration of the frequency characteristic and the residual of the aliasing component that occur when the luminance signal Y is created. It is intended to provide a video signal processing device capable of performing the above.

[0014]

A video signal processing apparatus according to the present invention comprises at least two solid-state image pickup elements each having a large number of light-receiving elements arranged in a matrix. The elements are arranged so as to be displaced from each other in the main scanning direction by about half the arrangement pitch of the light receiving elements, and the spatial pixel shift is performed. While correcting the pixel shift correction means, the aliasing component detecting means for detecting the aliasing component included in each color signal, the high-frequency detecting means for detecting the high-frequency component above the frequency that can be output by the solid-state image sensor, and the aliasing The aliasing component removing means for removing the aliasing component in each color signal by the aliasing component detected by the component detecting means, and the detection by the high frequency detecting means. It is characterized in that it comprises a high-frequency adding means for the high-frequency component is added to each color signal.

[0015]

In the video signal processing device according to the present invention as described above, the drawback of the conventional video signal processing device occurs because the pixel shift is established when the luminance signal is created. In order to solve the conventional drawbacks, the pixel shift is not established when the luminance signal is created to improve the resolution, but each color signal R, G,
It was made based on the recognition that it was possible to achieve the above-mentioned object by establishing the pixel shift in B.

[0016]

FIG. 16 shows the overall structure of a three-panel color television camera equipped with a video signal processing device according to the present invention. The optical image of the subject is separated into red, green, and blue optical images by the color separation prism 21, and the respective optical images are received by the solid-state image pickup devices 22R, 22G, and 22B, and the respective color signals R, G, B are output. Let These color signals R, G, B are supplied to the video signal processing device 23 according to the present invention to generate color signals R ₀ , G ₀ , B ₀ . These color signals R ₀ , G ₀ and B ₀ are supplied to the matrix 24 to create the luminance signal Y and the color difference signals I and Q, and the luminance signal Y is supplied to the encoder 25 and the color difference signals I and Q are filtered respectively. Encoder 25 via 26 and 27
To produce an encoded color television signal ENC and supply it to the output terminal 28. Also,
The red, green and blue signals R ₀ , G ₀ and B ₀ created by the video signal processing device 23 are output to output terminals 29R, 29G and 29B, respectively.

The video signal processing device 23 of the present invention is provided with the pixel shift correcting means, the aliasing component detecting means, the high frequency detecting means, the aliasing component removing means, and the high frequency adding means as described above. The means will be described. Pixel shift correction means for reading three solid-state image pickup devices with clock pulses of the same phase is a color signal R, G, B output from the solid-state image pickup device for each color of a color television camera adopting the spatial pixel shift method. Since there is a gap between them as shown in FIG. 17A, this is corrected. As a simple example of such pixel shift correction means, when the solid-state image sensor is arranged by the spatial pixel shift method as shown in FIGS. If R and B are delayed by a scanning time corresponding to half of the pixel arrangement pitch, each color signal becomes
Therefore, the pixel shift is corrected. However, in such a configuration, since the interpolation is performed by the zero-order hold method as described in the conventional video signal processing device, the frequency characteristic is deteriorated. Therefore, in the present invention, interpolation is performed by an ideal interpolation method that does not cause such deterioration of frequency characteristics.

First, between each pixel of each color signal R, G, B
"0" is inserted as shown in FIG. 17B. In this example,
As shown in FIG. 3, only half of the pixel pitch is converted into the green signal G.
On the other hand, since the red and blue signals R and B are displaced,
Set the position where "0" is inserted into the green signal G to the red and blue signals.
Figure 17B for the position where "0" is inserted in R and B.
It is necessary to shift by 1/2 pixel pitch as shown. now,
Figure 6 shows the frequency characteristics of the color signals output by the solid-state image sensor.
If it is something like "0" between pixels
As shown in Fig. 18, the frequency of 0.5f
_sA mirror image spectrum is generated around
Doubled. Therefore, the frequency 0.5f_sIs a stop zone
With a pass filter, the frequency is 0.5f _sThe following ingredients remain
The frequency is 0.5f_sThe following is shown in Figure 5
Therefore, the frequency characteristic is maintained as it is. This
In this way, the pixel part where "0" is inserted is interpolated.
become.

As described above, the color signals R, G and B have "0".
Is inserted and is passed through a low-pass filter having a frequency of 0.5 _fs or more as a stop band, so that the pixel portion in which "0" is inserted is ideally interpolated, and the deterioration of the frequency characteristic unlike the zero-order hold method is eliminated. Also, the position where "0" is inserted is 1/2 between the green signal G and the red and blue signals R and B.
These color signals R, G, B are shifted because they are shifted by the pixel pitch.
There is no pixel shift between them. It should be noted that the color signals having no pixel shift obtained in this way are represented by R ', G', and B '.

Next, the folded component detection based on the first method is performed.
The output means will be described. This means
The color signal R ′ whose pixel shift has been corrected by the correction means,
The folded component is detected from G'and B '. Occasion
The term “return” means, for example, the operating frequency of the solid-state image sensor is f_sWhen
Then, a black and white object having a frequency characteristic as shown in FIG.
0.5 f when capturing an image_sHigh frequency component above 0.5 f
_sCentered around 0.5 f _sWhich is folded back below
It As described above, this folding component is shown in FIGS.
As shown in, the green signal G and the red and blue signals R and
The phase is 180 ° different from that of B. this
Is a solid pixel for green as shown in Fig. 3 by the spatial pixel shift method.
Solid-state imaging for red and blue between pixels of body image sensor
Shifted by half the pixel pitch so that the pixel of the element is positioned
This is because

[0021] Therefore, a frequency 0.5 f _s following signal components of the subject is P, the loop signal if A, the color signals R, G, that folded signals as shown below in B is added Become.

## EQU00004 ## R = P-A G = P + A B = P-A (6) Therefore, the folding signal A can be detected from the color signals R, G, and B according to the following equation (7). In addition,
In the equation (7), the coefficients r and b may be any values as long as their sum is 0.5.

## EQU5 ## A = 0.5 G-rR-bB (7) However, r + b = 0.5

The frequency characteristic of the object is as shown in FIG. 5, and the green signal G output from the solid-state image pickup device for green color is output.
When the frequency characteristics of the red and blue signals R and B output from the solid-state image pickup device for red and blue are as shown in FIG. 7, the frequency characteristics of the folding signal A are shown in FIG. It is above the frequency 0.5 f _s of FIG. 5 is 0.
The characteristic is such that it folds back to the low frequency side around 5 f _s, and is as shown in FIG. However, this is the case where the subject is black and white as described above. When the subject has saturation, a level difference occurs between the color signals R, G, and B, so that the aliasing signal A has some level even if there is no aliasing component. As it is, it is impossible to know whether the folding signal A is a signal obtained by detecting a true folding component or the level difference between the color signals R, G and B due to the saturation of the subject.

Although the signal for transmitting color information is a color difference signal,
This color difference signal also has a fairly narrow band, and in the case of the NTSC system
The I signal is 1.5MHz. Frequency band of this color difference signal
F _cThen, if the subject is saturated,
Frequency f of the return signal A_cIngredients within
(Indicated by diagonal lines) is not the folding component, but the color component
Is. Therefore, the components generated in the shaded area in FIG.
Is not due to wrapping, so it is necessary to remove this.
There is a point. Therefore, the return signal A is transmitted at the frequency f_cSince
The frequency is passed through a high-pass filter that can fully attenuate the inside
f_cOnly the above high frequency components are extracted. This is the case in the present invention
In this way, the aliasing component signal H is detected.
The detection process of the component is expressed as follows.
It

H = A × HPF _fc where HPF _fc is the frequency characteristic of the high pass filter. (8)

Next, the high frequency detecting means according to the first method will be described. In the present invention, the high frequency component is detected from the aliasing component signal H detected as described above.
Aliasing component signal H is more than originally frequency 0.5 f _s
Around the 0.5 f _s is obtained back folded to the low frequency side.
Therefore, the high frequency component can be restored by returning the folding component signal H to a frequency of 0.5 _fs or higher. To do so, refer to FIG.
As shown in, the high frequency component can be reproduced by modulating the folding component signal H with the modulation signal C having the spectrum only at the frequency f _s . In the case of a digital signal, the high frequency component T can be obtained by performing the processing according to the following formula.

[Equation 7] T [n] = H [n] × (−1) ⁿ (9) where n = 0, 1, 2, 3 --- (pixel number)

Next, an example of the high frequency detecting means according to the second method of the present invention will be described. In this example, the high-frequency components are detected from the color signals R, G, and B before the pixel shift correction means corrects the pixel shift. As described above, the pixels of the solid-state image pickup device for green and the pixels of the solid-state image pickup device for red and blue are arranged with a shift of half the pixel pitch by the spatial pixel shift method. Therefore, if a pixel in which the red and blue signals R and B are weighted and averaged with an arbitrary mixing ratio is interpolated between the pixels of the green signal G, if the subject is black and white, the color signals R, G and B are Since the signals have the same level, the signal of the subject is reproduced, and a broadband signal without aliasing can be created. For example, the weighted average of the green signal G and the blue signal B at a ratio of 1: 1 is as shown in FIG. The red signal R
And the blue signal B are arbitrary.

Further, when the subject has the frequency characteristic as shown in FIG. 5, the frequency characteristic of each color signal R, G, B output from each solid-state image pickup device is as shown in FIGS. However, by interpolating a pixel obtained by weighted averaging the red signal R and the blue signal B at an arbitrary ratio between the pixel of the green signal G and the pixel, the same as that shown in FIG. Frequency characteristics can be obtained. The signal created in this way is called a wideband signal W. When the subject is black and white and has the frequency characteristic as shown in FIG. 5, the wideband signal W also has the same frequency characteristic as in FIG. The high-frequency detecting means according to the present invention is provided for each color signal R, G, B
Since the purpose is to detect high frequency components that should be added to
Extracting a frequency 0.5 f _s or more components of FIG. Here, when the subject has saturation, a level difference occurs in each color channel, and the wideband signal W alternates between the green signal G and the red and blue signals every 1/2 pixel pitch as shown in FIG. R
Since B and B are interpolated, there is a peak near the frequency f _s even if the subject has no high-frequency component.

Here, it is the color difference signal that transmits the color information, but the band of each color signal R, G, B is quite narrow as described above, and the I signal is 1.5 MHz. Assuming that the frequency band of this color difference signal is f _c , when the subject has saturation, the frequencies f _s −f _c to f _s of the wide band signal W are obtained.
A spectrum is generated up to (up to (A in FIG. 25)). That is, even if the subject originally has only low-frequency components, if the subject is colored, the components from the frequencies f _s −f _c to f _s of the wideband signal W must be removed. That is,
In the high frequency detecting means of the present invention, the frequency of 0.
Component of between 5 f _s to f _s -f _c (the portion B in FIG. 25) may be a band pass filter for extracting. The high frequency signal detected in this way is represented by T.

In the aliasing component detecting means according to the second method of the present invention, the aliasing component signal is detected from the high frequency component T detected by the above high frequency detecting means. Aliasing components of the respective color signals R, G, B in, since the high-frequency component is that back folding lowpass, if Utsuse below 0.5 f _s about the frequency f _s of the higher-band signal T, it is folded It becomes an ingredient.
To this end, as shown in FIG. 23, if the high frequency signal T is modulated with a modulation signal having a spectrum only at the frequency f _s , the high frequency signal will shift to a frequency of 0.5 f _s or less.
In the case of a digital signal, processing may be performed according to the following equation. The folding component signal obtained in this way is set to H.

[Equation 8] H [n] = T [n] × (-1) ⁿ n = 0, 1, 2, 3 --- (pixel number) (10)

Next, the folding back removing means will be described. This aliasing removing means uses the aliasing component signal H detected by the aliasing component detecting means for each color signal R, G,
The folding component is removed from B. The folding components included in the respective color signals R, G, B have a phase difference of 180 ° between the green signal G and the red and blue signals R and B as described above. Therefore, if the aliasing component signal H is added to the color signals R, G, and B as shown in the following expression (11), the aliasing component included in each color signal is canceled and removed. Thus the color signal aliasing components are removed by a _{R a, G a, B a} .

## EQU9 ## R _a = R + H G _a = G−H B _a = B + H (11)

As described above, the aliasing component signal H obtained by extracting the frequency f _c or higher in FIG. 21 from the signal having the frequency characteristic shown in FIG. 19 is converted into each color signal R, G, B by the equation (11). the addition can be removed aliasing components, only the frequency 0.5 f _s following signal components as shown in FIG. 22 is reproduced.

Next, the high frequency adding means will be described. Starting
In light, each color signal R from which the aliasing component is removed
A high frequency component is added to a, Ga and Ba. This is mentioned above
Color from which aliasing components have been removed by aliasing removing means
Signal R_a, G_a, B_aDetected by the high-frequency detection means described above
It can be realized by adding the high frequency signals T. Di
In case of digital signal, processing according to the following equation (12) can be performed.
Good. As described above, each color signal to which the high frequency component is added is R
_b, G _b, B_bIt is represented by.

## EQU10 ## R _b = R _a + T G _b = G _a + T B _b = B _a + T (12) By this processing, the color signals R _a , G _a , and B _a having the frequency characteristics shown in FIG. The high frequency signal T of the portion indicated by B (T) or the portion B of FIG. 25 can be added to create the color signals R _b , G _b , B _b having the frequency characteristics as shown in FIG.

Some embodiments of the video signal processing device of the present invention constituted by the above means will be described below. FIG. 27
Shows a first embodiment of the video signal processing apparatus according to the present invention, and the folding component detecting means and the high frequency detecting means adopt the above-mentioned first system. The color signals R, G, B output from the solid-state image sensor for each color are first supplied to the pixel shift correction unit 31, and the color signals R, B, G are supplied.
Correct the pixel shift between them. FIG. 28 shows the pixel shift correction unit 31.
2 shows a detailed configuration of the above. The red and blue signals R and B are supplied to the first input terminals a of the switch circuits 32R and 32B, respectively, and the green signal G is supplied to the switch circuit 3
It is supplied to the second input terminal b of 2G. In addition, a “0” value is commonly supplied to the second input terminal b of the switch circuits 32R and 32B and the first input terminal a of the switch circuit 32G. Further, these switch circuits 32R, 32G,
32B has a frequency equal to the operating frequency f _s of the solid-state image sensor, and is switched by the clock CK having a duty of 50%. That is, when the clock is logic "L", the color signal supplied to the first input terminal a is output, and when the clock is logic "1", the value "0" supplied to the second input terminal b is output. "0" is inserted between the pixels of each color signal so as to be output. A "0" value is supplied to the first input terminal a only to the switch circuit 32G, and a "0" value is supplied to the second input terminal b to the other switch circuits 32R and 32B.
Since the value is supplied, the position where the "0" value is inserted is shifted by 1/2 pixel pitch between the green channel and the red and blue channels as shown in FIG. These switch circuits 32R, 32G, the output signal of 32B, respectively low-pass filter 33R, 33G, and supplied to 33B, and passes only signals below component frequency 0.5 f _s.
In this way, the pixel at the position where "0" is inserted can be interpolated. In this case, the interpolation positions are different between the green channel and the red and blue channels, so
It is possible to obtain the color signals R ', G', B'in which the pixel shift between the color signals R, G, B is corrected.

Colors corrected for pixel shift as shown in FIG.
Supply signals R ', G', B'to folding component detector 34
To do. The detailed configuration of the aliasing component detector 34 will be described with reference to FIG.
Shown in. Each color signal corrected for pixel shift as described above
R ', G', B'is set to one of multipliers 35R, 35G, 35B
Supply to one input terminal. The other input end of the multiplier 35R
The coefficient "r" is supplied to the child, and the other input of the multiplier 35G
The coefficient "0.5" is supplied to the terminal and the other side of the multiplier 35B is supplied.
The coefficient "b" is supplied to the input terminal. Where the coefficient r
If b and b are such that the sum is 0.5, how
Any value may be set. Outputs of multipliers 35R and 35B
The force signals are added by the adder 36, and this is supplied to the subtractor 37
To do. The subtractor 37 also receives the output signal of the multiplier 35G.
Supply and output from the adder 36 output from the multiplier 35G
The signal is subtracted and the return signal A is extracted. This turn
Then, the signal A is further supplied to the high pass filter 38. This
The high-pass filter 38 of, for example, the frequency band of the color difference signal
Area f_cThen, the frequency f _cThe following signal components are sufficient
Attenuated, frequency f_cPass only the high frequency components above
It is a thing. In this way, from the high pass filter 38
The folded component signal H can be obtained.

As shown in FIG. 27, the aliasing component signal H detected by the aliasing component detector 34 is supplied to the high frequency detector 39. FIG. 30 shows a detailed configuration of the high frequency detection unit 39. The folded component signal H is supplied to one input terminal of the multiplier 40. On the other hand, "1" and "-1" are supplied to the first and second input terminals a and b of the switch circuit 41, respectively, and the switch circuit is driven by the clock signal CK having the frequency f _s . If the clock signal CK has a low logical value "L", the switch circuit 41 outputs the value "1" supplied to the first input terminal a, and if it has a high logical value "H",
It outputs "-1" supplied to the second input terminal b. That is, this switch circuit 41 has a frequency f _s.
A modulated signal having a spectrum only will be generated. This modulated signal is supplied to the second input terminal of the multiplier 40, and the above-mentioned folded component signal H is modulated to produce the high frequency signal T.
To create.

As shown in FIG. 27, the pixel shift correction unit 31
From each pixel signal R ′, from which the pixel shift has been corrected,
G'and B'and the aliasing component signal H detected by the aliasing component detecting unit 34 are supplied to the aliasing removing unit 42 to remove the aliasing component contained in each color signal. Figure 31
Shows a detailed configuration of the turnback removing unit 42.
The red and blue signals R ′ and B ′ whose pixel shifts between the respective colors have been corrected by the pixel shift correction unit 31 are supplied to the adders 43R and 43B together with the folding component signal H detected by the folding component detecting unit 34. The green signal G ′ in which the pixel shift is corrected is supplied to the subtractor 43G together with the folding component signal H, and the calculation according to the above equation (11) is performed to remove the folding signal R _a ,
Create G _a and B _a .

As shown in FIG. 27, the color signals R _a , G _a and B _a from which the aliasing component has been removed and the high frequency signal T detected by the high frequency detecting section 39 are supplied to the high frequency adding section 44. . Figure 3
2 shows the detailed structure of the high frequency adding unit 44, the color signals R _a to the above-mentioned aliasing components are removed, G _a, B
_a together with the high frequency signal T, adders 44R, 44
The color signals R _b , G _b , and B _b to which the high-frequency signal T is added are created as the final color signal output by supplying the signals to the G and 44 B and performing the calculation according to the above-described formula (12).

FIG. 33 shows the overall construction of the second embodiment of the video signal processing device according to the present invention. In the above-described first embodiment, the aliasing component signal H is detected from the color signals R ', G', B'in which the pixel shift between the respective colors is corrected, and the high frequency signal T is detected from this aliasing component. However, in this example, the high frequency signal T is extracted from the color signals R, G and B before the pixel shift correction, and the folding component signal H is detected from this high frequency signal. That is, the folding component detecting means and the high frequency detecting means are configured by the above-mentioned second method.

FIG. 34 shows a detailed structure of the high frequency detecting section 39 of this example. The multiplier 51R is supplied with the red signal Ro output from the red solid-state image sensor and the coefficient "rr" to create a product of these, and the multiplier 51B outputs the blue signal output from the blue solid-state image sensor. Signal B and coefficient "b
b ”to produce the product of these. The output signals of these multipliers are added by an adder 52 to obtain a weighted average. Here, the sum of the coefficients "rr" and "bb" is 1.
If it is 0, it may be set to any value. On the other hand, the green signal G output from the solid-state image pickup device for green is supplied to the first input terminal a of the switch circuit 53, and the output signal of the adder 52 described above is supplied to the second input terminal b. The switch circuit 53 is driven by the clock signal CK described above to generate the wide band signal W as shown in FIG. The wideband signal W is passed through the bandpass filter 54. The band-pass filter 54, and the frequency band of the chrominance signal and f _c, is configured to pass below the frequency 0.5 f _s or f _s -f _c. In this way, the bandpass filter 5
4 can output the high frequency signal T.

As shown in FIG. 33, the above-mentioned high frequency detecting section 3
The high-frequency signal T output from the aliasing component detector 34
Supply to. FIG. 35 shows the detailed configuration of the aliasing component detector 34. A switch circuit 55 is provided, and its first and second input terminals a and b have a value of "1" and "-".
1 "value to feed this switch circuit to the clock signal CK
Drive with. The output signal of the switch circuit 55 is a modulation signal (similar to the modulation signal C shown in FIG. 23) having a spectrum only at the frequency f _s . The modulated signal C can be supplied to the multiplier 56 together with the high frequency signal T, the high frequency signal T can be modulated by the modulated signal, and the processing of formula (10) can be performed to detect the aliasing component signal H. The other configuration of the video signal processing apparatus according to the present invention shown in FIG. 33 is the same as the configuration of the first embodiment described above, and therefore its description is omitted.

FIG. 36 shows the overall construction of the third embodiment of the video signal processing apparatus according to the present invention. In this example, the high frequency signal T is detected from the color signals R, G and B before the pixel shift correction as in the second embodiment, and the folding component signal H is the same as in the first embodiment. The detection is performed from the color signals R ', G', and B 'whose deviations have been corrected, and the connection positions of the aliasing removing section 42 and the high frequency band adding section 44 are reversed. That is, since the removal of the aliasing component and the addition of the high frequency signal are performed by the subtraction and addition processing,
The order may be arbitrary.

The present invention is not limited to the above-mentioned embodiments, but various modifications and variations are possible. For example, the aliasing component signal H at the colored part of the subject
It is also possible to eliminate the aliasing component and add the high frequency signal only to the achromatic portion of the subject or a portion close to the achromatic color by setting the level of 0 to zero.
That is, in the third method of the aliasing component detecting means, the aliasing component signal included in each color signal is detected in phase difference among the respective color signals, the aliasing component signal is detected, and each color signal is examined to determine the color. If it is an achromatic part without a mark, the aliasing component signal is output as it is, and if it is a colored part with a color, the aliasing component signal is set to zero and the aliasing component signal is detected.

In the above-described first-type folding component detecting means, if the object is colored, the color signals R,
A level difference occurs between G and B, and when the folding signal A is created according to the equation (7), the folding signal A has any level even if there is no folding. Therefore, the aliasing component signal H is obtained by extracting the components in the color difference signal band f _c or more from the aliasing component signal so as not to affect the color difference signal transmitting the color information. This is based on the premise that the color difference signal that conveys color information should not be affected. However, in reality, the subject is colored, and the color difference signal band f
If there is a frequency component equal to or higher than _c , the aliasing component signal H has some level although it is not aliasing, and as a result, erroneous processing is performed by the aliasing removing means and the high frequency detecting means.

In order to eliminate the above-mentioned drawback, the folding component detecting means of the third method detects whether or not the subject is colored, and at that portion, the level of the folding component signal H is forced to zero. By doing so, aliasing removal and high frequency signal addition are performed only in the achromatic portion of the subject or in a portion close to the achromatic color.

First, a method for detecting whether or not a subject is colored will be described. In order to detect whether or not the subject is colored, there is a method of detecting whether or not the ratio of the color signals R, G, B is 1: 1: 1. Here, the first method described above is used. A method that can be easily realized by changing the configuration of the aliasing component detecting means will be described. If the subject is colored, there will be a level difference between the color signals R, G and B, and the folding signal A will have some level in the equation (7).
If there is no color, there is no level difference between the color signals R, G and B, so the folding signal A becomes zero. However, this is the case where the color signals R ′, G ′, and B ′ whose pixel shifts have been corrected have no aliasing, and when the color signals whose pixel shifts have been corrected have aliasing, they have a relatively high frequency range. It should be included a lot. Therefore, if the low frequency part of the folding signal A is taken out, the folding component can be ignored. The low frequency component of the folding signal A is A _L. This low frequency component A
_{If L} is zero, the subject is not colored in that portion, and if this low-frequency component A _L is not zero, it can be determined that the subject is colored.

Based on the above-described principle, it is determined by the low frequency component A _L of the aliasing component signal A whether the subject is colored or not, and the determination signal A _C is created. This judgment signal A _C
If the subject is determined to be colored, that portion is set to 0.0, and the portion not determined to be colored is set to 1.0.
Signal. Whether there is a color or not
The value should be between 0.0 and 1.0. Finally, when the determination signal A _C indicates that the subject is colored, the level of the aliasing component signal H is set to 0 at that portion, and aliasing is performed at the portion where it is determined that the subject is not colored. The component signal H is output as it is. The folding component signal generated in this way is designated as H '. The above-described processing can be actually performed by the following calculation.

(11) H ′ = H × A _C (13)

According to the folding component detecting means described above,
Whether it is colored or not, it is relatively colored.
The judgment signal A_CIf is close to 0.0, then
The applied folding component signal 'approaches zero and vice versa.
Judgment signal A with little sticking _CWhere the value of is close to 1.0
Is a signal having a value close to the level of the folding component signal H
Will be obtained. In this way, the achromatic color of the subject
Extracted the aliasing component of a part or a part close to achromatic color
A folded component signal H'can be obtained. like this
The aliasing component signal H ′ is converted into the aliasing removing means or the high
If it is given to the area detection means, the achromatic part of the subject or that
Elimination of aliasing components and high range only for the part close to
Addition, no processing is performed on the colored part.
Therefore, the color difference signal band f_cThe above frequency components
Effectively prevent wrong processing even if there is
You can

FIG. 37 shows a concrete structure of the above-mentioned folding component detecting means, and a multiplier 35 shown on the left side.
The R, 35G, 35B, the adder 36, the subtractor 37, and the high-pass filter 38 correspond to the configuration of the folding component detection unit shown in FIG. 29, and the portion surrounded by the broken line on the left side is added. In this example, the output H of the high-pass filter 38 is subtracted from the aliasing signal A output from the subtractor 37 by the subtractor 61 to detect the low-frequency component A _L of the aliasing signal A. This low frequency component A _L can also be obtained by passing the folding signal A through a low pass filter.

Next, the low frequency signal A _L output from the subtractor 61 is supplied to the absolute value circuit 62 to obtain the absolute value.
Since the folding signal A is processed as shown in the equation (7), it takes a positive or negative value, and therefore the low-frequency component A _L also takes a positive or negative value. Therefore, it is necessary to obtain its absolute value. The absolute value of the low frequency component A _L obtained in this way is then supplied to the multiplier 63 and amplified with an arbitrary gain amount. This gain amount is processed as follows in the subsequent processing. If this gain amount is reduced, the achromatic portion and the colored portion of the subject become less distinct, and the relatively colored portion is processed so as not to be colored. Further, if the gain amount is made excessively large, even a portion that is not so much colored will be treated as if it is colored.

Further, the output signal of the multiplier 63 is supplied to the clipping circuit 64, and when the output signal of the multiplier exceeds 1.0, it is clipped to 1.0. The output signal of the clipping circuit 64 is further supplied to the subtractor 65, and this output signal is subtracted from the "1.0" value. The output signal of the subtractor 65 is the judgment signal A _C , and if the subject is colored, the judgment signal A _C is 0.0 in that part and 1.0 in the uncolored part, and is it colored? In the subtle part, it takes a value between 0.0 and 1.0.

The aliasing component signal H output from the high-pass filter 38 and the decision signal A _C output from the subtractor 65 described above are supplied to the multiplier 66, and the product of these is obtained to obtain a color. The level of the aliasing component signal H is set to zero in the portion, and the aliasing component signal H ′ is obtained by outputting the aliasing component signal H as it is in the portion where the subject is not colored.

[0051]

As described above, according to the video signal processing device of the present invention, aliasing included in the color signal output from the solid-state image pickup device in which the spatial pixel shift is performed is completely removed, and the frequency characteristic is excellent. High resolution color signals can be obtained. That is, in the conventional video signal processing device, since the pixel shift is established in the matrix for creating the luminance signal, the high frequency region is not sufficiently extended, the frequency characteristic is deteriorated, and the folding component is slightly left. However, according to the present invention, since the aliasing component is removed in each color signal and the high frequency part is added, there is no residue of the aliasing component,
The limit resolution can be improved, and the deterioration of frequency characteristics can be almost eliminated. Further, since the pixel shift is established for each color signal, when the color signal is output from the three-panel color television camera, it is possible to obtain a color signal having no registration error as in the conventional video signal processing device. Therefore, the aliasing component is completely removed from the luminance signal formed by matrixing the color signals, and the high-quality video signal having high horizontal resolution can be obtained without deterioration of the frequency characteristic. Note that the pixel shift correction is effective only when each solid-state image pickup element is read with a clock pulse of the same phase, but other effects can be obtained regardless of the reading of the solid-state image pickup element. .

[Brief description of drawings]

FIG. 1 is a diagram showing a configuration of an image pickup unit of a color television camera adopting a spatial pixel shift method.

FIG. 2 is a diagram similarly showing the arrangement of pixel positions for each color.

FIG. 3 is a diagram showing a relationship of spatial arrangement positions of pixels for each color.

FIG. 4 is a diagram showing folding components included in a luminance signal.

FIG. 5 is a diagram showing frequency characteristics of a subject.

FIG. 6 is a diagram showing frequency characteristics of a green signal.

FIG. 7 is a diagram showing frequency characteristics of red and blue signals.

FIG. 8 is a diagram showing a scanning position by the image sensor for each color.

FIG. 9 is a diagram showing scanning positions by a green image pickup element.

FIG. 10 is a diagram showing scanning positions by red and blue image pickup devices.

FIG. 11 is a diagram showing a luminance signal created from the color signals shown in FIGS. 9 and 11;

FIG. 12 is a diagram showing a high band part and a low band part of a luminance signal.

FIG. 13 is a diagram showing interpolation processing by a conventional zero-order hold method.

FIG. 14 is a block diagram showing an example of a conventional video signal processing device.

FIG. 15 is a block diagram showing another example of a conventional video signal processing device.

FIG. 16 is a diagram showing an overall configuration of a color television camera including a video signal processing device according to the present invention.

FIG. 17A is a diagram showing a shift between color signals output from a solid-state image sensor of each color signal of a color television camera adopting a spatial pixel shift method, and FIG. 17B is a pixel shift according to the present invention. It is a diagram which shows a correction process.

FIG. 18 is a diagram showing a mirror image spectrum in the pixel shift correction processing according to the present invention.

FIG. 19 is a diagram showing a color signal obtained through a low-pass filter in the pixel shift correction process.

FIG. 20 is a diagram showing folding components.

FIG. 21 is a diagram showing a folding component and a color difference signal component.

FIG. 22 is a diagram showing a color signal from which aliasing components have been removed.

FIG. 23 is a diagram showing a process of reproducing a high frequency component from a folding component.

FIG. 24 is a diagram showing a process of detecting a high frequency component by weighted averaging red and blue signals.

FIG. 25 is a diagram illustrating a process of detecting a folding component from a high frequency component.

FIG. 26 is a diagram illustrating a process of adding a high frequency component after removing a folding component.

FIG. 27 is a block diagram showing a configuration of a first embodiment of a video signal processing device according to the present invention.

FIG. 28 is a diagram showing details of the pixel shift correction unit.

FIG. 29 is a diagram showing details of the aliasing component detection unit.

FIG. 30 is a diagram showing details of the high-frequency detector.

FIG. 31 is a diagram showing details of the turn-back removing unit.

FIG. 32 is a diagram showing details of the high-frequency adding unit.

FIG. 33 is a block diagram showing a second embodiment of the video signal processing device according to the present invention.

FIG. 34 is a diagram showing details of a high frequency detection unit of the second embodiment.

FIG. 35 is a diagram showing details of a folding component detection unit of the second embodiment.

FIG. 36 is a block diagram showing a third embodiment of the video signal processing device according to the present invention.

FIG. 37 is a diagram showing details of the aliasing component detection unit of the third embodiment.

[Explanation of symbols]

 21 Color Separation Prism 22R, 22G, 22B Solid-state Image Sensor 23 Video Signal Processing Device 24 Matrix 25 Encoder 26 I Filter 27 Q Filter 31 Pixel Displacement Correction Unit 34 Folding Component Detection Unit 39 High Frequency Detection Unit 42 Folding Removal Unit 44 High Band Addition Department

Claims (6)

[Claims] 1. A solid-state image pickup device comprising at least two solid-state image pickup devices, each of which has a large number of light-receiver devices arranged in a matrix. A pixel shift correction unit that corrects a pixel shift between a plurality of color signals output from a solid-state imaging device that is arranged so as to be shifted from each other in a spatial pixel shift manner, while suppressing deterioration of frequency characteristics; The aliasing component detecting means for detecting the aliasing component included in, the high frequency detecting means for detecting the high frequency component higher than the frequency that the solid-state image sensor can output, And a high frequency component for adding the high frequency component detected by the high frequency detection means to each color signal. A video signal processing apparatus characterized by comprising a means. 2. The aliasing component detecting means is configured to detect the aliasing component by utilizing that the aliasing component included in each color signal has a different phase in each color signal. The described video signal processing device. 3. The video signal processing apparatus according to claim 1, wherein the high-frequency detecting means is configured to detect a high-frequency component by utilizing a spatial arrangement of solid-state image pickup elements of each color signal. . 4. The folding component detecting means is configured to detect the folding component by frequency-modulating the high-frequency component detected by the high-frequency detecting means with an operating frequency of a solid-state image sensor. The video signal processing device according to claim 1. 5. The high-frequency detecting unit is configured to detect the high-frequency component by frequency-modulating the folding component detected by the folding-component detecting unit with the operating frequency of the solid-state image sensor. The video signal processing device according to claim 1. 6. The video signal processing apparatus according to claim 1, wherein the aliasing component detecting means is configured to set an aliasing component of a colored portion of the subject to zero or substantially zero.