JPH07154805A - Video signal processor - Google Patents

Abstract

PURPOSE:To eliminate folded components and to output a color signal free from a picture element deviation by forming the picture element deviation by each color signal R, G and B without picture element deviation when a luminance signal is prepared. CONSTITUTION:The color signals R, G and B outputted from a solid-state image pickup element for each color are supplied to a picture element deviation correction part 41, and color signals R', G' and B' where the picture element deviation between each color signal R, B and G is corrected are prepared. These color signals R', G' and B' are supplied to a folded component detection part 42. A folded component signal H is detected and the color signals are also supplied to a folded component elimination part 43. The folded component signal H detected in the folded component detection part 42 is supplied to this folded component elimination part and folded components are eliminated. Thus, the picture element deviation is corrected and color signals R0, G0 and B0 where folded components are eliminated are obtained.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention comprises at least two solid-state image sensors each having a large number of light-receiving elements arranged in a matrix. A color television camera using a plurality of color signals output from a solid-state image pickup device in which spatial pixel shifts are performed by arranging them so that they are displaced from each other by approximately half in the main scanning direction, and in particular, a three-plate solid-state image pickup element employing the spatial pixel shift method The present invention relates to a video signal processing device that processes a plurality of color signals output from the.

[0002]

2. Description of the Related Art For example, in a three-panel color television camera, in order to obtain a high-resolution video signal with a solid-state image sensor having a small number of pixels, a red color is obtained with reference to the spatial position of the green (G) solid-state image sensor. A spatial pixel shift method is used in which the positions of the solid-state image pickup devices for (R) and blue (B) are shifted in the main scanning direction (horizontal direction) by half the pixel arrangement pitch.

FIG. 1 shows an example of the structure of an image pickup section of a three-panel color television camera which adopts the above-described spatial pixel shift method. Incident light from a subject is split into three primary color lights of R, G, and B by a color separation prism 1, and optical images of the respective colors are received by three solid-state image pickup devices 2R, 2G, and 2B, and red signals are obtained. It is configured to output R, green signal G and blue signal B. Figure 2 shows these three
The solid-state image pickup elements 2R, 2G, and 2B each show a spatial arrangement relationship of pixels, and the pixels of the solid-state image pickup element 2R for red and the solid-state image pickup element 2B for blue are the same as those of the solid-state image pickup element 2G for green. The pixels are arranged so as to be shifted in the horizontal direction by half the pixel pitch P. The luminance signal Y is created by mixing the color signals output from these solid-state image pickup devices 2R, 2G, and 2B at a prescribed ratio in the encoder matrix processing unit, and the pixels corresponding to the respective colors are as shown in FIG. Therefore, the number of pixels is apparently increased, and the luminance signal Y with improved resolution can be obtained. When the luminance signals Y are created by mixing these color signals R, G, and B, the mixing ratio is, for example, NTSC.
In the case of the method, it is specified as the following equation.

[Equation 1] Y = 0.3R + 0.59G + 0.11B (1)

[0004]

In the three-panel color television camera adopting the pixel shifting method as described above, the number of pixels apparently increases, and the luminance signal Y with improved resolution is obtained, but is output. As shown in FIG. 4, the luminance signal Y having such a response has a wavy response, and some aliasing components are generated. Therefore, the resolution is not so high and the frequency characteristic is deteriorated. Although a high resolution is not required in an actual broadcasting system, many folding components will be generated in the luminance signal Y unless the spatial pixel shift method is adopted. Also,
For each of the color signals R, G, and B, the signal output from the solid-state image sensor is output as it is, so many aliasing components are generated, and the pixel shift causes a registration error, which cannot be used as it is. It has the drawback of disappearing.

[0005] Here, the aliasing component, if the operating frequency of the solid-state image pickup device and f _s, the frequency 0.5 f _s or more high frequency components folded around the 0.5 f _s to the following lower frequency band 0.5 f _s It is returned. When the subject having the frequency characteristic as shown in FIG.
Signals having frequency characteristics as shown in FIGS. 6 and 7 are output from the G, red, and blue solid-state imaging devices 2R and 3B, respectively. Although spectrum in these figures are wavy large, which represents aliasing components, frequency 0.5 f _s or more high frequency components aliasing low-pass, is because it is added to the low frequency component . As described above, each color signal output from the three-panel color television camera adopting the pixel shift method includes many folding components.

The phase of the folding component is the green signal G
And the red and blue signals R and B have a phase of 180
There is a deviation, and the wave pattern of the spectrum is opposite between FIG. 6 and FIG. 7. This is because in the spatial pixel shifting method, the pixels of the red and blue solid-state image pickup devices 2R and 2B are located at intermediate positions between the pixels of the green solid-state image pickup device 2G as shown in FIG. Is. For example, in the case of imaging a subject that gives a high-frequency signal as shown in FIG. 8, the solid-state image sensor 2G for green images the position of the mark ●, and the solid-state image sensors 2R and 2B for red and blue are imaged. Since the image is taken at the position of the X mark, the green signal, the red signal, and the blue signal have low-frequency waveforms as shown in FIGS. 9 and 10, respectively, and the green signal waveform and the red and blue signal waveforms are different from each other. Is 180 degrees out of phase.

When the luminance signals Y are created by mixing the color signals R, G and B as described above at the ratios described above, most of the aliasing components are inverted in phase between the green signal and the red and blue signals. However, since the mixing ratio of the color signals when creating the luminance signal is not 1: 1 for the green signal and the red and blue signals, it is folded back as shown in the following equation (2). Some components will remain (see Figure 4). For example, in the case of the NTSC system, 18% of the folding component remains as shown in Expression (3).

[Formula 2] Remaining amount of folding component = Green mixing ratio-Red mixing ratio-Blue mixing ratio (2) Remaining amount of folding component = 0.59-0.3 -0.11 = 0.18 (3)

Further, in the case of a high quality television system, the luminance signal Y is mixed according to the following equation (4), so that as many folding components as 40% remain as shown in equation (5). It will be.

[Formula 3] Y = 0.701G + 0.087B + 0.212R (4) Remaining amount of folding component = 0.701-0.087-0.212 = 0.402 (5)

Further, the conventional video signal processing device has a problem that the frequency characteristic of the luminance signal Y is deteriorated with respect to the frequency characteristic of the subject as shown in FIG.
This will be described below. The solid-state image sensor of each color channel by the pixel shift method and the spatial arrangement of the pixels of the luminance signal Y obtained by mixing these color signals according to the above equation (1) are shown in FIG. As shown. As shown in FIG. 2, a pixel for green color,
Since the pixel for red and the pixel for blue are deviated by a half of the pixel pitch, it is necessary to mix them so as not to shift the positional relationship when the luminance signal Y is created. As described above, since the luminance signal Y shifts the positions of the red and blue solid-state image pickup devices 2R and 2B with respect to the position of the green solid-state image pickup device 2G by half the pixel pitch, and mixes them according to the equation (1). Since the number is twice that of the solid-state image sensor alone, the resolution is improved.

Here, although the number of pixels of each color signal is substantially doubled, the interpolated pixel (pixel shown in parentheses in FIG. 11) at that time is the same as the previous pixel as seen in the scanning direction. Therefore, the level of the previous pixel is set. Such an interpolation method is known as a zero-order hold method. The frequency characteristic is deteriorated by the zero-order hold method. Since the luminance signal Y is created by mixing the color signals whose frequency characteristics have deteriorated by the zero-order hold method, the frequency characteristics of the luminance signal Y also deteriorate as shown in FIG. In the conventional video signal processing device that processes the color signals output from the three-panel color television camera that employs the spatial pixel shift method as described above, the number of pixels of each color signal is substantially equal when the luminance signal Y is created. However, since the interpolation is performed by the zero-order hold method, there is a disadvantage that the frequency characteristic is inevitably deteriorated.

Next, the resolution of the color signals R, G, B output from the solid-state image pickup device for each color and the problem of registration error will be described. FIG. 12 shows a configuration of an example of a conventional video signal processing device, which delays a green signal G output from a solid-state image pickup device for each color by a delay 8
The signals delayed by 1/2 pixel pitch and the red and blue signals R and B are supplied to the Y, I and Q matrixes 3 to generate the luminance signal Y and the color difference signals I and Q. The luminance signal Y is supplied to the encoder 6 through the filters 4 and 5 and the encoded color television signal ENC is generated, and the encoded color television signal ENC is supplied to the output terminal 7. In the pixel shift method, the red, green, and blue signals R, G, and B are mixed based on, for example, the equation (1) to create a luminance signal Y, thereby improving the resolution of the luminance signal. . That is, since the pixel shift is established in the matrix 3, the video signal processing device shown in FIG. 12 can obtain only the resolution of the solid-state image sensor for each color signal output, and is a signal including many foldings. . Here, in order to extract each color signal without causing a registration error, the output of the delay 8 may be extracted for the green signal. This is a case of taking out as an analog signal. However, there is a problem in extracting it as a digital signal. If the operating frequency of the solid-state image sensor is f _s , the sampling frequency for the digital signal is also f _s, but if only the green signal is sampled after the delay 8, The clock signal in 1 must be out of phase with the red and blue clock signals by an amount equivalent to 1/2 pixel pitch. In this case, the clock signals of the color signals R, G and B cannot be the same, which is inconvenient. If the green signal is taken out from before the delay 8 and sampled, this problem does not occur, but a registration error occurs between the color signals. In this case, it suffices to have a digital filter for shifting the phase for the green signal output, but there is a disadvantage that the number of circuits is increased because of that.

In order to eliminate such a defect, FIG.
The video signal processing device shown in FIG. In this video signal processing device, output from luminance signals Y, I and Q filters 4 and 5 of Y, I and Q matrix 3 for producing luminance signal Y and color difference signals I and Q from respective color signals R, G and B. The color difference signals I and Q are supplied to the dematrix 10 so as to obtain the respective color signals R ', G', B '. However, in such a conventional video signal processing device, the color signals R ', B', G'output to the color signal output terminals 9R, 9G, 9B have some folding components. That is, since the luminance signal Y having the deteriorated frequency characteristic is decoded to create the color signal, it is considered that the frequency characteristic is deteriorated because there is some folding component remaining, though not more than the method of FIG. There is a drawback.

As described above, there are several spatial pixel shift methods.
However, the solid-state image sensor with a small number of pixels
This is one effective method for obtaining high resolution on the child.
However, in reality, the video signals of the broadcast standard and VTR standard are not
It does not have such high resolution. For example,
The signal output from the solid-state image sensor of Lath is the operating frequency f _s
Is 14.32MHz, and the frequency band of the output signal is 7.16MHz.
It becomes 0.5 f as shown in Fig. 6 and Fig. 7._sUp to the band
However, when the luminance signal Y is created, the number of apparent pixels is doubled.
Therefore, the frequency band is f as shown in FIG._s= 14.32MH
Extends close to z. However, when recording video signals
In the case of the digital VTR D2 system, for example,
The signal bandwidth that can be recorded is only about 7MHz, so
0 to 0.5 f_sCan only store signals in the frequency band between
Absent. Therefore, when using the spatial pixel shift method,
Is a high-resolution luminance signal created by pixel shifting.
Bandwidth was purposefully limited to 0.5 f in FIG._sTo f_sUp to
It must be recorded after cutting the wave number component. An example
For example, as shown in FIG. 14, after Y, I, Q matrix 3
In addition to the usual I, Q filters 4 and 5, the luminance signal Y
Luminance filter 11 that limits the bandwidth to 7MHz or less is required.
It In addition, in the case of high definition type,
2 million pixels, its operating frequency is 74.25MHz, output
The frequency band of the signal is about 37MHz. But spatial pixels
When the shift method is adopted, the frequency band of the luminance signal Y is 74MH
It becomes z. Therefore, conform to the standard of high quality method
In order to do this, it is necessary to limit the bandwidth and restore it.

In any case, unless the spatial pixel shift method is adopted, there is only the frequency band of the output signal of the solid-state image pickup device, so that band limitation is unnecessary, but in that case, many folding components are included in the output signal. Will occur. The reason is that when the spatial pixel shift method is not adopted, the phases of the folding components are the green signal G and the red and blue signals R.
This is because B and B have the same phase, and therefore the aliasing component is not canceled even by the mixing process when creating the luminance signal. As described above, although some folding components remain, the spatial pixel shift method is an effective method for reducing the folding components. However, it is necessary to limit the band in order to adapt to the broadcasting system and the recording system, and there is a disadvantage that an extra circuit is required accordingly.

An object of the present invention is to solve the above-mentioned drawbacks of the conventional video signal processing device, remove aliasing components contained in each color signal R, G, B output from the solid-state image pickup device.
It is possible to output the color signals R, G, and B without pixel deviation, so-called registration error, prevent the deterioration of the frequency characteristic and the residual of the aliasing component that occur when the luminance signal Y is created, and broadcast. An object of the present invention is to provide a video signal processing device that does not need to limit the frequency band in order to adapt to the recording system and the recording system.

[0016]

A video signal processing apparatus according to the present invention comprises at least two solid-state image pickup elements each having a large number of light-receiving elements arranged in a matrix. The elements are arranged so as to be displaced from each other in the main scanning direction by about half the arrangement pitch of the light receiving elements, and the spatial pixel shift is performed. While correcting the pixel shift correction means, the aliasing component detecting means for detecting the aliasing component included in each color signal, and the aliasing component removal for removing the aliasing component in each color signal by the aliasing component signal detected by this aliasing component detecting means. It is characterized by comprising means and. In the present invention, each of the pixel shift correction means, the aliasing component detecting means, and the aliasing component removing means,
The solid-state image pickup device is configured so as not to handle signals in a band that is at least half the operating frequency.

[0017]

In the video signal processing device according to the present invention as described above, the drawback of the conventional video signal processing device occurs because the pixel shift is established when the luminance signal is created. In order to solve the conventional drawbacks, it is possible to achieve the above-mentioned object by establishing pixel shift for each color signal R, G, B instead of establishing pixel shift when creating a luminance signal. It was confirmed and made based on that recognition. In addition, the pixel shift correcting means, the aliasing component detecting means, and the aliasing component removing means are configured so as not to handle signals in a frequency band of 1/2 or more of the operating frequency of the solid-state image sensor, and thereby the video signal of the present invention. It is not necessary to bother to limit the frequency band of the video signal output from the processing device to conform to the standards of the broadcasting system and the recording system.

[0018]

FIG. 15 shows the overall construction of a three-panel color television camera equipped with a video signal processing device according to the present invention. The optical image of the subject is separated into red, green, and blue optical images by the color separation prism 21, and the respective optical images are received by the solid-state image pickup devices 22R, 22G, and 22B, and the respective color signals R, G, B are output. Let These color signals R, G, B are supplied to the video signal processing device 23 according to the present invention to generate color signals R ₀ , G ₀ , B ₀ . These color signals R ₀ , G ₀ and B ₀ are supplied to the matrix 24 to create the luminance signal Y and the color difference signals I and Q, and the luminance signal Y is supplied to the encoder 25 and the color difference signals I and Q are filtered respectively. Encoder 25 via 26 and 27
To produce an encoded color television signal ENC and supply it to the output terminal 28. Also,
The red, green and blue signals R ₀ , G ₀ and B ₀ created by the video signal processing device 23 are output to output terminals 29R, 29G and 29B, respectively.

In the video signal processing device 23 of the present invention, the pixel shift correcting means, the aliasing component detecting means and the aliasing component removing means are provided as described above, and these means will be described next. The pixel shift correction means corrects the color signals R, G, B output from the solid-state image pickup device for each color of the color television camera adopting the spatial pixel shift method, and corrects them. As a simple example of such pixel shift correction means, when the solid-state image sensor is arranged by the spatial pixel shift method as shown in FIGS. If R and B are delayed by a scanning time corresponding to half the pixel arrangement pitch, the relationship between the respective color signals becomes as shown in FIG. 11 and the pixel shift is corrected, but the apparent number of pixels is doubled. Therefore, the frequency band is doubled. This may be sufficient if the pixel shift is simply corrected, but a digital filter is used in the present invention to correct the pixel shift without widening the band. In this case, if the number of stages of the digital filter is small, and if the shift amount is large, the frequency characteristic will be deteriorated.
To correct the pixel shift with the digital filter, the green signal G is shifted by 0.5 pixel and the red and blue signals R and B are shifted.
There is a way to match. The transfer function and frequency characteristics of this 0.5 pixel shift filter are shown by the curve A in FIG. In this case, the frequency characteristics of the green signal G is degraded extremely near 0.5 f _s. Since the red and blue signals R and B are not passed through the filter, the frequency characteristic is not deteriorated. Thus, if only the green signal G is passed through the digital filter, the green signal G
And the red and blue signals R and B have different frequency characteristics, which causes inconvenience in the subsequent processing. Therefore, in the present invention, in order to prevent such deterioration of the frequency characteristic, as shown in FIG. 16A, the green signal G is shifted to the right by 0.25P (P is a pixel pitch), and the red and blue signals R and B are shifted. Try to shift it to the left by 0.25P. By performing such pixel shift correction, it is possible to reduce the shift amount, and therefore it is possible to reduce the deterioration of the frequency characteristic even if a digital filter having a small number of stages is used. Further, even if the frequency characteristic is deteriorated, the deteriorations in the green, red and blue signals are the same, which is advantageous. Figure 1 shows the frequency characteristics of a conventional digital filter that shifts pixels by 0.5P.
The curve A of Fig. 7 shows the frequency characteristic of the digital filter which shifts the pixel by 0.25P like the present invention, and the curve B shows that the deterioration of the frequency characteristic is suppressed according to the present invention.

The above-described pixel shift correcting means of the present invention can be realized by using a digital filter whose left and right coefficients are asymmetric. An example of such a digital filter is shown in FIG.
Shown in. If the transfer function shown in FIG. 17 is converted into a circuit,
Although 16 multipliers are required, for simplicity of explanation, a case of configuring with 4 multipliers will be described below as an example. First, the case where the green signal G is shifted to the right by 0.25P will be described. The video signals applied to the input terminal 31a are sequentially supplied to the delay devices 32b to 32d which delay the scanning time corresponding to one pixel pitch. Input terminal 31
When the pixel G4 in FIG. 16A is input to a, the pixel G3 is output to the output of the delay device 32b, the pixel G2 is output to the output of the delay device 32c, and the pixel G1 is output to the output of the delay device 32d. Further, the output terminals of these delay devices 32b to 32d are supplied to one input terminals of the multipliers 33b to 33d, respectively. The input terminal 31a is directly connected to one input terminal of the multiplier 31a. These multipliers 33a to 33d
The other input terminal has a coefficient "C-2 = -0.05", "C-1 = 0.
Supply "3", "C1 = 0.9" and "C2 = -0.15" respectively. The output signals of the multipliers 33a to 33d are added by the adder 34, and the result is output from the output terminal 35 as a filter output. By selecting the coefficients as described above, the sum of C1 and C2 becomes 0.75, and the sum of C-1 and C-2 becomes 0.25. Therefore, as shown in FIG.
, And the pixel G3, the signal level at a position apart from the pixel G2 by 0.25P can be interpolated. next,
A case where the red and blue signals R and B are shifted to the left by 0.25P will be described. The video signal supplied to the input terminal 31b is sequentially supplied to the delay devices 32a to 32d, and the outputs of the delay devices 32a to 32d are supplied to one input terminals of the multipliers 32a to 32d. When the pixel R4 / B4 in FIG. 16A is input to the input terminal 31b, the pixel R3 / B3 is output to the delay device 32a, the pixel R2 / B2 is output to the multiplier 32b, and the pixel R2 / B2 is output to the multiplier 32c. R1 / B1
However, the pixel R0 / B0 is output to the output of the multiplier 32d. Coefficients are "C-2 = -0.15", "C-1 = 0.9", "C1 = 0.3"
, And “C2 = −0.05”, the left and right of the above-mentioned coefficients are reversed, and as shown in FIG. 19B, the pixel R1 / B1 and the pixel R2 /
It is possible to interpolate the signal level at a position between B2 and -0.25P away from the pixel R2 / B2. Where to the right
When shifting by 0.25 pixel and shifting by 0.25 pixel to the left,
Since the same coefficient value may be set to the left and right, the frequency characteristics of the filter are the same. According to the present invention, such a pixel shift correcting means is not limited to the one having 4 taps as shown in FIG. 18, but may have a larger number of taps, and by increasing the number of taps. The frequency characteristics can be ideally flat.
The red, green, and blue signals obtained in this way and having no pixel shift will be referred to as R ', G', and B '. Since these color signals are signals in which the pixel shift is simply corrected, if the filter characteristics are flat, the frequency characteristics are the same as those shown in FIGS. 6 and 7. As another method of the pixel shift correction means, there is also a method of shifting the red and blue signals to the right by 0.25 pixels and the green signal to the left by 0.75 pixels as shown in FIG. 16B. In this case, the green signal and the red and blue signals are given to the input terminal 31a in FIG. 18, and the coefficient values in the case of the green signal are "C-2 = -0.15" and "C-2.
If -1 = 0.9 "," C1 = 0.3 ", and" C2 = -0.05 ", it becomes a filter that shifts 0.75 pixels, and the coefficient values for red and blue signals are" C-2 = -0.05 "and" C-1. = 0.3 "," C1 = 0.9 ",
If "C2 = -0.15" is set, the filter shifts by 0.25 pixels.
Also in this case, since the same coefficient value may be set to the left and right, the frequency characteristics of the filter are the same.

Next, the folding component detecting means will be described. This means detects the aliasing component signal from the color signals R ', G', B 'whose pixel shifts have been corrected by the pixel shift correction means described above. Folding means, for example, when the operating frequency of the solid-state image sensor is f _s , when capturing a black and white subject having frequency characteristics as shown in FIG. 5, a high frequency component of 0.5 f _s or more is centered on 0.5 f _s.
In which it folded below 0.5 f _s. As described above, this folding component has a phase difference of 18 between the green signal G and the red and blue signals R and B as shown in FIGS.
0 ° different. This is because by the spatial pixel shift method, as shown in FIG. 3, the pixels of the solid-state imaging device for green are displaced from each other by half the pixel pitch so that the pixels of the solid-state imaging device for red and blue are located between the pixels. is there.

[0022] Therefore, a frequency 0.5 f _s following signal components of the subject is P, if the aliasing signal A, the color signals R ', G', the B 'are subject to aliasing signal as shown below Will be there.

## EQU4 ## R '= P-A G' = P + A B '= P-A (6) Therefore, the folding signal A is detected from the color signals R', G ', B'according to the following equation (7). You can In the equation (7), the sum of the coefficients r and b is 0.5.
Any value can be used as long as

## EQU5 ## A = 0.5 G'-rR'-bB '(7) However, r + b = 0.5

The frequency characteristic of the object is as shown in FIG. 5, and the green signal G output from the solid-state image pickup device for green color is output.
When the frequency characteristics of the red and blue signals R and B output from the solid-state image pickup device for red and blue are as shown in FIG. 7, the frequency characteristics of the folding signal A are shown in FIG. It is above the frequency 0.5 f _s of FIG. 5 is 0.
The characteristic is such that it folds back to the low frequency side around 5 f _s, and is as shown in FIG. However, this is the case where the subject is black and white as described above. When the subject has saturation, a level difference occurs between the color signals R, G, and B, so that the aliasing signal A has some level even if there is no aliasing component. As it is, it is impossible to know whether the folding signal A is a signal obtained by detecting a true folding component or the level difference between the color signals R, G and B due to the saturation of the subject.

Although the signal for transmitting color information is a color difference signal,
This color difference signal also has a fairly narrow band, and in the case of the NTSC system
The I signal is 1.5MHz. Frequency band of this color difference signal
F _cThen, if the subject is saturated,
Frequency f of the return signal A_cIngredients within
(Indicated by diagonal lines) is not the folding component, but the color component
Is. Therefore, the components generated in the shaded area in FIG.
Is not due to wrapping, so it is necessary to remove this.
There is a point. Therefore, the return signal A is transmitted at the frequency f_cSince
The frequency is passed through a high-pass filter that can fully attenuate the inside
f_cOnly the above high frequency components are extracted. This is the case in the present invention
In this way, the aliasing component signal H is detected.
The detection process of the component signal is expressed as follows.
is there.

H = A × HPF _fc where HPF _fc is the frequency characteristic of the high pass filter. (8) By detecting the aliasing component signal in this way, it is possible to prevent at least the frequency component within the transmission band of the color difference signal of the specified broadcasting system from being affected in the subsequent processing.

Next, the folding back removing means will be described. This aliasing removing means uses the aliasing component signal H detected by the aliasing component detecting means for each color signal R ',
The folded component is removed from G'and B '. The folding components included in the respective color signals R ', G', B'are 180 ° out of phase with each other between the green signal G'and the red and blue signals R'and B'as described above. Therefore, if the aliasing component signal H is added to the color signals R ', G', and B'as shown in the following equation (9), the aliasing components included in each color signal are canceled out. In this way, the color signals from which the aliasing components have been removed are set to Ra, Ga, and Ba.
And

## EQU7 ## Ra = R '+ H Ga = G'-H Ba = B' + H (9)

As described above, the aliasing component signal H obtained by extracting the frequency f _c or higher in FIG. 21 from the signal having the frequency characteristic shown in FIG. 22 is used as each color signal R ', G',
When added by the formula (9) to B 'can be removed aliasing components, only the signal component of the frequency 0.5 f _s or less as shown in FIG. 23 is reproduced.

Some embodiments of the video signal processing apparatus of the present invention constituted by the above means will be described below. Figure 24
Shows a first embodiment of a video signal processing device according to the present invention. The color signals R, G, B output from the solid-state image sensor for each color are first supplied to the pixel shift correction unit 41 to correct the pixel shift between the color signals R, B, G.
Create G'and B '. These color signals are supplied to the aliasing component detecting unit 42, which detects the aliasing component signal H and also supplies the aliasing component removing unit 43. The aliasing component signal H detected by the aliasing component detecting unit 42 is supplied to this aliasing component signal H. It is supplied to the removing unit to remove the aliasing component. In this way, the pixel shift is corrected, and the color signals R0, G0, B0 from which the aliasing component is removed are obtained.

FIG. 25 shows a detailed structure of the pixel shift correction section 41. Digital filters 51R, 51G and 51B for shifting pixels of red and blue signals R and B
Supply to each. The digital filter 51R shifts the red signal R to the left of the screen by 0.25P pixels, and the digital filter 51G shifts the green signal G to the right of the screen by 0.25P pixels. Is the blue signal B to the left of the screen
The pixel is shifted by 0.25P. These digital filters have the coefficient C-2, shown in FIG.
It can be realized by appropriately setting C-1, C1, and C2.

The color signals R ', G', and B'corrected for the pixel shift as described above are supplied to the folding component detector 42. FIG. 26 shows the detailed configuration of the aliasing component detection unit 42.
Shown in. Each color signal R ', G', B'corrected for pixel shift
Is supplied to one input terminal of each of the multipliers 55R, 55G, 55B. The coefficient "r" is applied to the other input terminal of the multiplier 55R.
To the other input terminal of the multiplier 55G.
5 "and the coefficient" b "is supplied to the other input terminal of the multiplier 55B. Where the sum of the coefficients r and b is
Any value may be set as long as it is 0.5. The output signals of the multipliers 55R and 55B are added to the adder 5
The addition is made in 6 and this is supplied to the subtractor 57. The output signal of the multiplier 55G is also supplied to the subtractor 57, and the adder 5G
The output signal of the multiplier 55G is subtracted from the output signal of No. 6 to obtain the folding signal A. The folding signal A is further supplied to the high pass filter 58. The high pass filter 58, for example, when the frequency band of the chrominance signal and f _c, the signal components below the frequency f _c is sufficiently attenuated, and passes only the high frequency components above the frequency f _c. In this way, the aliasing component signal H can be obtained from the high pass filter 58.

As shown in FIG. 24, the pixel shift correction section 41.
From each pixel signal R ′, from which the pixel shift has been corrected,
G'and B'and the aliasing component signal H detected by the aliasing component detecting unit 42 are supplied to the aliasing component removing unit 43 to remove the aliasing component contained in each color signal. FIG. 27 shows a detailed configuration of the aliasing component removing unit 43. The red and blue signals R ′ and B ′ whose pixel shifts between the respective colors are corrected by the pixel shift correction unit 41 are supplied to the adders 61R and 61B together with the folding component signal H detected by the folding component detecting unit 42. The green signal G ′ in which the pixel shift has been corrected is supplied to the subtractor 61G together with the folding component signal H, and the color signal R from which the folding component has been removed by performing the operation according to the above equation (9).
Create a, Ga, and Ba.

The present invention is not limited to the above-mentioned embodiments, but various modifications and variations are possible. For example, the aliasing component signal H at the colored part of the subject
By setting the level of 0 to 0, the aliasing component can be removed only in the achromatic portion or near the achromatic portion of the subject. That is, in the second embodiment of the aliasing component detecting means, the aliasing components included in each color signal have different phases in each color signal, so that the aliasing component signal is detected, and further each color signal is examined. If it is an achromatic part without color, the aliasing component signal is output as it is, and if it is a colored part with color, the aliasing component signal is set to zero and the aliasing component signal is detected.

In the above-described aliasing component detecting means, if the subject is colored, a level difference occurs between the color signals R, G and B, and the aliasing signal A is calculated according to the equation (7).
, The folding signal A has no level even if there is no folding. Therefore, the aliasing component signal H is obtained by extracting the components in the color difference signal band f _c or more from the aliasing component signal so as not to affect the color difference signal transmitting the color information. This is based on the premise that the color difference signal that conveys color information should not be affected. However, if the subject is actually colored and the subject has a frequency component equal to or higher than the color difference signal band f _c , the aliasing component signal H has some level although it is not aliasing, and as a result, aliasing removal is performed. Incorrect means will be performed by the means. In order to eliminate the above-mentioned drawback, in another embodiment of the aliasing component detecting means, it is detected whether or not the subject is colored, and the level of the aliasing component signal H is forced to be zero at that portion. Therefore, the aliasing removal is performed only on the achromatic portion of the subject or a portion close to the achromatic color.

First, a method of detecting whether or not a subject is colored will be described. In order to detect whether or not the subject is colored, there is a method of detecting whether or not the ratio of the respective color signals R, G, B is 1: 1: 1. Here, the first embodiment described above is used. A method that is easily realized by changing the configuration of the aliasing component detecting means will be described. If the subject is colored, there will be a level difference between the color signals R, G and B, and the folding signal A will have some level in the equation (7). If there is no color, there is no level difference between the color signals R, G and B, so the folding signal A becomes zero. However, this is the case where the color signals R ′, G ′, and B ′ whose pixel shifts have been corrected have no aliasing, and when the color signals whose pixel shifts have been corrected have aliasing, they have a relatively high frequency range. It should be included a lot. Therefore, if the low frequency part of the folding signal A is taken out, the folding component can be ignored. The low frequency component of the folding signal A is A _L. This low frequency component A _L
If is zero, it means that the subject is not colored in that portion, and if this low frequency component A _L is not zero, it can be determined that the subject is colored.

Based on the above-described principle, it is determined whether or not the subject is colored by the low frequency component A _L of the folding signal A, and the determination signal A _C is created. This determination signal A _C is
If it is determined that the subject is colored, that part is set to 0.
The signal is 0, and the part that is determined not to be colored is 1.0. 0.0 is the subtleties of color
The value should be between 1.0 and 1.0. Finally, when the determination signal A _C indicates that the subject is colored, the level of the aliasing component signal H is set to 0 at that portion, and aliasing is performed at the portion where it is determined that the subject is not colored. The component signal H is output as it is. The folding component signal generated in this way is designated as H '. The above-described processing can be actually performed by the following calculation.

[Equation 8] H ′ = H × A _C (10)

According to the folding component detecting means described above,
Whether it is colored or not, it is relatively colored.
The judgment signal A_CIf is close to 0.0, then
The applied folding component signal 'approaches zero and vice versa.
Judgment signal A with little sticking _CWhere the value of is close to 1.0
Is a signal having a value close to the level of the folding component signal H
Will be obtained. In this way, the achromatic color of the subject
Extracted the aliasing component of a part or a part close to achromatic color
A folded component signal H'can be obtained. like this
The aliasing component signal H'is applied to the aliasing removing means described above.
If so, make sure to match the achromatic part of the subject or a part close to it.
Then, the aliasing components are removed and the colored parts
Minute processing does not perform any processing, so
f_cEven if there are frequency components above, erroneous processing will not occur.
Can be effectively prevented.

FIG. 28 shows a specific structure of the above-mentioned folding component detecting means, and a multiplier 55 shown on the left side.
The R, 55G, 55B, the adder 56, the subtracter 57, and the high-pass filter 58 correspond to the configuration of the folding component detection unit shown in FIG. 26, and are the parts surrounded by the broken line on the left side. In this example, the output H of the high-pass filter 58 is subtracted from the folding signal A output from the subtractor 57 by the subtractor 65 to detect the low frequency component A _L of the folding signal A. This low frequency component A _L can also be obtained by passing the folding signal A through a low pass filter.

Next, the low frequency signal A _L output from the subtractor 65 is supplied to the absolute value circuit 66 to obtain the absolute value.
Since the folding signal A is processed as shown in the equation (7), it takes a positive or negative value, and therefore the low-frequency component A _L also takes a positive or negative value. Therefore, it is necessary to obtain its absolute value. The absolute value of the low frequency component A _L obtained in this way is then supplied to the multiplier 67 and amplified with an arbitrary gain amount. This gain amount is processed as follows in the subsequent processing. If this gain amount is reduced, the achromatic portion and the colored portion of the subject become less distinct, and the relatively colored portion is processed so as not to be colored. Further, if the gain amount is made excessively large, even a portion that is not so much colored will be treated as if it is colored.

Further, the output signal of the multiplier 67 is supplied to the clipping circuit 68, and when the output signal of the multiplier exceeds 1.0, it is clipped to 1.0. This clip circuit 68
Is further supplied to the subtractor 69, and this output signal is subtracted from the "1.0" value. The output signal of the subtractor 69 is the judgment signal A _C , and if the subject is colored, the judgment signal A _C is 0.0 in that part, and 1.0 in the uncolored part, and whether it is colored or not. In the subtle part, it takes a value between 0.0 and 1.0.

The aliasing component signal H output from the high-pass filter 58 and the determination signal A _C output from the subtractor 69 described above are supplied to the multiplier 70, and the product of these is obtained to obtain a color. The level of the aliasing component signal H is set to zero in the portion, and the aliasing component signal H ′ is obtained by outputting the aliasing component signal H as it is in the portion where the subject is not colored.

[0040]

As described above, according to the video signal processing device of the present invention, aliasing included in the color signal output from the solid-state image pickup device in which the spatial pixel shift is performed is completely removed, and the frequency characteristic is excellent. A color signal can be obtained. That is, in the conventional video signal processing device, since the pixel shift is established in the matrix for creating the luminance signal, the frequency characteristic is deteriorated and some folding components remain, but according to the present invention, the asymmetric digital Since the ideal pixel shift correction is performed by the filter, the deterioration of the frequency characteristic can be almost eliminated. In addition, since the frequency band is not twice the driving frequency of the solid-state image sensor as in the conventional method in any of the pixel shift correcting means, the aliasing component detecting means, and the aliasing component removing means, in order to meet the broadcast standard or the VTR standard. There is no need to bother limiting the frequency band. Further, since the pixel shift is established for each color signal, when the color signal is output from the three-panel color television camera, it is possible to obtain a color signal having no registration error as in the conventional video signal processing device. Therefore, the aliasing component is completely removed from the luminance signal formed by matrixing the color signals, and the high-quality video signal can be obtained without the D characteristic of the frequency characteristic.

[Brief description of drawings]

FIG. 1 is a diagram showing a configuration of an image pickup unit of a color television camera adopting a spatial pixel shift method.

FIG. 2 is a diagram similarly showing the arrangement of pixel positions for each color.

FIG. 3 is a diagram showing a relationship of spatial arrangement positions of pixels for each color.

FIG. 4 is a diagram showing folding components included in a luminance signal.

FIG. 5 is a diagram showing frequency characteristics of a subject.

FIG. 6 is a diagram showing frequency characteristics of a green signal.

FIG. 7 is a diagram showing frequency characteristics of red and blue signals.

FIG. 8 is a diagram showing a scanning position by the image sensor for each color.

FIG. 9 is a diagram showing scanning positions by a green image pickup element.

FIG. 10 is a diagram showing scanning positions by red and blue image pickup devices.

FIG. 11 is a diagram showing interpolation processing by a conventional zero-order hold method.

FIG. 12 is a block diagram showing an example of a conventional video signal processing device.

FIG. 13 is a block diagram showing another example of a conventional video signal processing device.

FIG. 14 is a block diagram showing still another example of a conventional video signal processing device.

FIG. 15 is a diagram showing an overall configuration of a color television camera including a video signal processing device according to the present invention.

16A and 16B are diagrams showing pixel shift correction processing according to the present invention.

17A and 17B are diagrams showing frequency characteristics in the pixel shift correction processing according to the present invention in comparison with a conventional example.

FIG. 18 is a circuit diagram showing a configuration of an example of a digital filter used in the pixel shift correction means.

FIG. 19 is a diagram showing an interpolation position in the pixel shift correction means.

FIG. 20 is a diagram showing folding components.

FIG. 21 is a diagram showing a folding component and a color difference signal component.

FIG. 22 is a diagram showing a signal processed by a pixel shift correction means.

FIG. 23 is a diagram showing a signal from which aliasing components have been removed.

FIG. 24 is a block diagram showing a configuration of a video signal processing device according to the present invention.

FIG. 25 is a diagram showing details of the pixel shift correction unit.

FIG. 26 is a diagram showing details of the aliasing component detection unit.

FIG. 27 is a diagram showing details of the aliasing component removal unit.

FIG. 28 is a diagram showing details of another embodiment of the aliasing component detection unit.

[Explanation of Codes] 21 Color Separation Prisms 22R, 22G, 22B Solid-state Image Sensor 23 Video Signal Processing Device 24 Matrix 25 Encoder 26 I Filter 27 Q Filter 41 Pixel Shift Correction Unit 42 Folding Component Detection Unit 43 Folding Component Removal Unit

Claims (6)

[Claims] 1. A solid-state image pickup device comprising at least two solid-state image pickup devices, each of which has a large number of light-receiver devices arranged in a matrix. Pixel deviation correction means for correcting pixel deviation between a plurality of color signals output from a solid-state imaging device that is arranged so as to be displaced from each other in the direction and shifted spatial pixels while suppressing deterioration of frequency characteristics, and for each color signal. Video signal processing comprising: a folding-back component detecting means for detecting a folding-back component included therein; and folding-back component removing means for removing a folding-back component in each color signal by the folding-back component signal detected by the folding-back component detecting means. apparatus. 2. The pixel shift correcting means, the aliasing component detecting means, and the aliasing component removing means are each configured not to handle a signal in a band equal to or more than 1/2 of the operating frequency of the solid-state imaging device. The video signal processing apparatus according to claim 1, wherein the video signal processing apparatus is a video signal processing apparatus. 3. The aliasing component detecting means is configured to detect the aliasing component by utilizing the fact that the aliasing component included in each color signal has a different phase in each color signal. The described video signal processing device. 4. The aliasing component detection means is configured to detect the aliasing component signal so as not to affect at least the transmission band of the color difference signal of the specified broadcasting system. Video signal processing device. 5. When the pixel pitch is P in the pixel shift correction means, one of the color signals is + 0.25P.
3. The video signal processing apparatus according to claim 2, wherein the pixel is shifted by -0.25P and the pixel of another color signal is shifted so as to match the position to correct the pixel shift. 6. The video signal processing apparatus according to claim 2, wherein the aliasing component detecting means is configured to set an aliasing component signal of a colored portion of the subject to zero or substantially zero.