JP3134513B2 - Imaging device with horizontal line interpolation function - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an electronic zoom function circuit and a motion compensating circuit for performing motion compensation in an image pickup apparatus such as a video camera, and more particularly to an electronic horizontal line by performing arithmetic processing on a video signal. The present invention relates to an image pickup apparatus having a horizontal line interpolation function for performing the interpolation processing.

[0002]

2. Description of the Related Art In recent years, with respect to imaging devices such as video cameras and the like, miniaturization, light weight, high magnification zoom, and further multi-functionalization have been advanced, and products having an optical zoom function and an electronic zoom function interlocked have been developed. In addition, as the user class expands from children to the elderly in addition to conventional mania, image shakers occur due to camera shake, and imaging devices equipped with a motion correction circuit using an electronic zoom function have been commercialized. .

[0003] As a conventional motion compensator using an electronic zoom function, for example, a technical report of the TV Society, Vol. 11,
NO3 (may. 1987).

Hereinafter, a conventional motion compensation circuit having an electronic zoom function will be described. FIG. 19 is a technical report VOL. 11, a block diagram of a camera shake correction device including a conventional motion detection circuit shown in NO3 (may. 1987), where 1901 is an A / D conversion circuit, and 1902 is a motion vector detection circuit , 1
903 is a memory control circuit, 1904 is a memory circuit, 19
05 is an interpolation control circuit, 1906 is an interpolation circuit, and 1907 is a D / A conversion circuit.

The operation of the conventional camera shake correction apparatus including the above-described motion detection circuit will be described below.

An input signal is converted into a digital signal by an A / D conversion circuit 1901, and is input to a motion vector detection circuit 1902 and a memory circuit 1904.
At 902, a motion vector is detected by comparing the video signals of the two fields, and at the memory control circuit 1903, a cutout signal corrected for camera shake is obtained from the memory circuit 1904 using the motion vector. The memory output signal is the interpolation control circuit 1
A normal video signal is obtained by an interpolation circuit 1906 controlled by 905, and an analog signal is obtained by a D / A conversion circuit 1907.

As described above, the motion vector is detected from the video signal of two fields, and the camera shake is corrected by the interpolation function.

As a conventional image pickup apparatus having an interpolation function, for example, Japanese Patent Application Laid-Open No. 1-261086 discloses an "image pickup apparatus".
Is shown in

Hereinafter, a conventional imaging apparatus having an interpolation function will be described. FIG. 20 is a block diagram of another conventional imaging device with an interpolation function. In FIG.
2001 is a solid-state image sensor, 2002 is a solid-state image sensor 20
The drive circuit 01 controls the transfer / stop of the vertical transfer of the solid-state imaging device 1 by the control signal C1. In addition, the unnecessary signals are swept out by the control signal C4. 200
Reference numeral 3 denotes a process circuit that generates a luminance signal, a color signal, a color difference signal, and the like from an output signal of the solid-state imaging device 2001. 2
Reference numeral 004 denotes an output signal of the process circuit 2003 as a control signal C
2, a selector for selecting and outputting two line memories in accordance with the control signal C3. 200
Reference numerals 9 and 2010 denote multipliers for multiplying the output signals of the selector 2008 by the weight signals W1 and W2, respectively. The signal of the upper scanning line on the screen is supplied to the multiplier 2009, and the signal of the lower scanning line is supplied to the multiplier 2010. . 2011 is a multiplier 200
9, an adder for adding the output signals of No. 9 and 2010 and outputting the result to the output terminal 2012. Reference numeral 2013 denotes a control signal generator that supplies a control signal and a weight signal to each unit and an address signal to the line memories 2005 to 2007.

The operation of the conventional image pickup apparatus with an interpolation function configured as described above will be described below.

Scanning line signals output from the solid-state image sensor 2001 are stored in line memories 2005 to 2007,
A selector 2008 selects upper and lower two scanning line signals necessary for generating an interpolation signal, and outputs the signals to multipliers 2009 and 2010. Further, the control signal generation circuit 2013 determines and outputs the weighting factors W1 and W2 of the multipliers from the value of the line below the vertical address of the interpolation signal, that is, the decimal part. After being multiplied by the weight coefficients by the multipliers 2009 and 2010 and the adder 2011, they are added and a linear primary interpolation signal is output.

[0012]

However, the conventional interpolation method has the following problems. That is,
The vertical frequency response characteristic of the video signal after the interpolation processing changes with the interpolation coefficient. For example, a line interpolated with interpolation coefficients of 1/2 and 1/2 is a perfect average of two input lines, so that the frequency response characteristic in the vertical direction is the lowest, and a line interpolated with interpolation coefficients of 1 and 0 is Since one line of the interpolation coefficient 1 is output as it is, the frequency response characteristic in the vertical direction is the highest. In other words, there is a problem that the frequency response characteristic of the output line interpolated near the center between the two input lines is reduced.

Therefore, the image pickup apparatus that performs the interpolation process using the above-described conventional interpolation circuit has a problem that the sharpness of the image in the vertical direction is deteriorated. The present invention has been made to solve the conventional problem, and has as a technical problem to provide an imaging apparatus with a horizontal line interpolation function that can reduce deterioration of sharpness in the vertical direction of an image when performing horizontal line interpolation processing. .

[0014]

[MEANS FOR SOLVING THE PROBLEMS] To achieve this object
The imaging device with horizontal line interpolation function of the present invention
Obtain two color signals C1, C2 and C3ImagingMeans and said
The vertical phase of the color signal C2 with respect to the color signal C1
First shift by a fixed position interval p1 (p1 ≠ 0)
Vertical phase shift ofmeansAnd the color signal C1
The phase of the color signal C3 in the vertical direction is set to a fixed position interval p2 (p
Second vertical phase shift shifted by 2 ≠ 0)means
And a color whose phase is vertically shifted from the color signal C1.
From signals C2 and C3, Interpolated horizontal line signals in phase with each other
Signal is created by interpolation processing, and the vertical phase
An arithmetic circuit for compensating for the displacement; and an output signal of the arithmetic circuit.
With the enlargement means to obtain the signalWith configuration
are doing.

[0015]

According to the present invention, the first and second vertical phase shifters change the vertical phase of three color signals obtained from a plurality of solid-state image pickup devices by the above-described configuration, and the pseudo phase generated by the frame operation circuit is generated. By performing the interpolation processing using the frame signal, the deterioration of the frequency characteristics in the vertical direction due to the interpolation is reduced.

[0016]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing an imaging apparatus having a horizontal line interpolation function according to a first embodiment of the present invention. 1, reference numeral 101 denotes an image sensor having a photoelectric conversion function; 102, an image sensor driver for the image sensor 101; 103, a drive control circuit for controlling the image sensor driver 102; 104, an output signal of the image sensor 101; An analog signal processing circuit for performing processing such as sampling and amplification; 105, an analog-to-digital conversion circuit (hereinafter referred to as an A / D conversion circuit) for an output signal of the analog signal processing circuit 102; A digital signal processing circuit that generates a luminance signal, a color signal, a color difference signal, or the like, or performs RGB signal processing; 107, a field memory circuit that stores an output signal of the digital signal processing circuit 106; 108, a field memory that controls the field memory circuit 107 A control circuit 109 is an output of the field memory circuit 107 Electronic zoom circuit for performing interpolation and expansion process using the No., 110 electronic zoom circuit 109
An electronic zoom control circuit 111 controls the drive control circuit 103, a field memory control circuit 108, and an electronic zoom control circuit 110.
An encoder circuit 112 obtains an NTSC signal from RGB.

The operation of the imaging apparatus having the horizontal line interpolation function of the present embodiment configured as described above will be described below. RGB output from the image sensor 101
The plurality of output signals B are converted into digital signals by analog signal processing and A / D conversion processing. This digital signal is subjected to RGB signal processing in the digital signal processing circuit 106 and input to the field memory circuit 107. The signal input to the field memory circuit 107 is interpolated by the field memory control circuit 108, the electronic zoom circuit 109, and the electronic zoom control circuit 110.

FIG. 2 shows a solid-state image pickup by the drive control circuit 103.
5 shows frame accumulation driving control of an element (hereinafter, also simply referred to as an imaging element) . FIG. 2A shows a normal interlace read drive control of the image sensor, and FIG. 2B shows R, G, and B signals for obtaining respective signals of R, G, and B, which are configuration examples of the image sensor 101 according to the present embodiment. The outline of the read drive control of the B image sensor is shown.
In the frame accumulation mode, as shown in FIG.
During the field shift period in the field, every other pixel in the photosensitive area is read out in the vertical direction, and the signal of the pixel in the odd-numbered line is read out. Has been realized. In the present embodiment, as shown in FIG. 2B, in the odd field, the R / B image sensor among the R / G / B image sensors captures the signal of the pixel of the odd-numbered line in the vertical direction and captures the G image. The element reads out the signal of the pixel of the even-numbered line, and then, in the even field, the signal of the pixel of the even-numbered line in the vertical direction is read out of the R, G, and B image pickup elements by G in the even field. Read out the signals of the pixels on the odd-numbered lines. In this manner, the readout of odd / even of the R, G, and B image sensors by the frame accumulation drive control is performed by using R
-The B image pickup device and the G image pickup device are reversed.

Next, FIG. 3 shows the field accumulation drive control of the imaging element by the drive control circuit 103 . FIG. 3A shows an ordinary interlaced read drive control of the image sensor, and FIG.
(B) schematically shows read drive control of an R, G, and B image sensor that obtains R, G, and B signals, which is another configuration example of the image sensor unit 101 according to the present embodiment. As shown in FIG. 3A, in the field accumulation mode, the signal of the odd-numbered line and the signal of the next even-numbered line are simultaneously added from the pixels of the line near the horizontal transfer CCD (not shown) in the odd field ( PDmix), and then, by changing the combination of addition in the even field, the signal of the even-numbered line from the bottom and the signal of the next odd-numbered line are simultaneously added and read, thereby implementing interline transfer. In this embodiment, FIG.
As shown in FIG. 3B, in the odd field, among the R, G, and B image pickup devices, the R, B image pickup device shown in FIG.
Field readout is performed with an even field readout for the G image sensor, and then, even field readout is performed for the R, G, and B imagers among the R, G, and B imagers, and odd field is read for the G imager. Reading is being performed. As described above, in the field accumulation driving control, R
-Reads the PD / odd / even PDmix of the G / B image sensor
The B image pickup device and the G image pickup device are reversed.

As shown in FIGS. 2 and 3, the spatial position of the obtained R / B signal and G signal is obtained by reversing the odd / even reading between the R / B image pickup device and the G image pickup device. (Phase) is 1/2 line (line interval in one field)
It will shift.

As described above, according to the present embodiment, the drive control circuit is provided to shift the phases of the three color signals R, G, and B, respectively, so that the solid-state imaging device can obtain three color signals. The need for an assembly process with strict accuracy of shifting the position of the three-color separation prism or the two-color separation prism in the vertical direction and bonding and fixing the same was reduced, and the total manufacturing cost including manufacturing equipment and manufacturing time was reduced. It is possible to obtain an imaging apparatus that reduces the deterioration of the frequency response characteristics in the vertical direction due to the interpolation processing and the deterioration of the sharpness in the vertical direction during horizontal line interpolation.

Next, a signal processing method for the obtained R, G, B color signals will be described below. FIG. 4 shows an outline of the signal processing. FIG. 4A shows a case where the interpolation process is not performed at an arbitrary spatial position, and FIG. 4B shows a case where the interpolation process is performed at an arbitrary spatial position . As shown in FIG. 4A, the RB signal and the G signal
Since the phase of the signal is shifted by 1/2 line (line interval in one field), it is necessary to match the phase in the vertical direction for signal processing. Therefore, two consecutive G signals
By performing the line averaging process (interpolation process of the interpolation coefficients 1/2 and 1/2), the phases of the RGB signals can be matched. This is shown in (Equation 1). FIG. 4 (b)
As shown in, when the interpolation process is performed, the interpolation coefficient is set to w,
Assuming that the phase difference between the R / B signal and the G signal is p (= 位相 line), it can be expressed by a function of w and p, and the obtained interpolation signal is shown in (Equation 2).

[0024]

(Equation 1)

[0025]

(Equation 2)

When the interpolation lines are combined in this way, an in-phase interpolation signal can be synthesized from two different types of color signals, and the deterioration of the frequency response characteristic due to the interpolation differs depending on the phase. When the three color signals are viewed as a whole (for example, when a luminance signal synthesized by matrix operation from G, R, and B signals is considered), deterioration of the frequency response characteristics can be reduced, and the sharpness of the horizontal line interpolation signal in the vertical direction can be reduced. It is possible to reduce the degree of deterioration. For example, if p = [1/2 line of a video signal to be interlaced scanned], even if interpolation processing is performed with w = 0.5 and the frequency response characteristic is most deteriorated, either the G signal or the R or B signal is used. Since the interpolation processing is not performed and a state in which the frequency response characteristic does not deteriorate can be maintained, the deterioration of the vertical sharpness of the horizontal line interpolation signal is reduced in the entire video signal.

In FIGS. 2 to 4, the control (C1) for spatially shifting the position of the G signal has been described.
Control (C2) for spatially shifting the position of the signal is also possible. Further, it is possible to spatially shift only (C3) the position of the R signal and (C4) only the position of the B signal. Hereinafter, the spatial position control will be described. The luminance (Y) signal is created by the RGB signals, which are represented by (Equation 3).

[0028]

(Equation 3)

Here, the spatial shift included in the luminance signal
The signal (Yb) deviating from the signal (Ya) not present is represented by the following equation (Equation 4) in the case of C1 to C4.

[0030]

(Equation 4)

The color difference signals (RY and BY) are R
It is created from the GB signal, which is shown in (Equation 5).

[0032]

(Equation 5)

[0033] Here, spatially not included in the color difference signals
The signal (Cb) deviating from the signal (Ca) that is not present is represented by the following equation (Equation 6) in the case of C1 to C4.

[0034]

(Equation 6)

In order to reduce the deterioration of the sharpness of the horizontal line interpolation signal in the vertical direction in the entire video signal, Ya and Yb are substantially equal and Ca and Cb are substantially equal in (Equation 4) and (Equation 6). There is a need. From this, the spatial position control is C1 or C2.
Is appropriate. Therefore, only the case where the spatial position control is C1 and C2 will be described below.

Next, the digital signal processing circuit 1 shown in FIG.
Circuit configuration 06 will be described. FIG. 5 shows a circuit configuration example. FIG. 5A shows the case where the spatial position control is C1, and FIG. 5B shows the case where the spatial position control is C2. In FIGS. 11A and 11B, the same reference numerals are given to the components having the same effect, and the description is omitted. FIG. 5A shows a 1H memory 501 for delaying one horizontal line period of the G signal, an adder 502, a 1/2 amplifier 503 for gain adjustment, and luminance for performing the arithmetic processing shown in (Equation 3). It has a signal matrix 504 and a color signal matrix 505 for performing the arithmetic processing shown in (Equation 5). Similarly, FIG. 5B shows the 1H memory 501 and the adder 502 for delaying the R signal and the B signal by one horizontal line period.
A 1/2 amplifier 503 for gain adjustment, a luminance signal matrix 504 for performing the arithmetic processing shown in (Equation 3),
Color signal matrix 50 for performing the arithmetic processing shown in (Equation 5)
Five. The digital signal processing circuit 106 having the above-described structure can match the phases of the color signals by providing the vertical interpolation function having the 1H memory, that is, the 2H line averaging circuit. Perform signal processing. Also, FIG.
It can be seen from FIG. 2B that FIG. 2A, that is, C1 (shifting the G signal) in the spatial position control is more suitable for reducing the circuit scale. On the other hand, in signal processing when interpolation processing is not performed on an arbitrary spatial position, the spatial position (phase) shown in FIG.
Since the phase matching is performed on the color signal having a shift in the vertical direction by performing the LPF processing in the vertical direction, the high-frequency characteristics deteriorate. The video signal is divided into three color signals (eg, G,
When viewed as a luminance signal synthesized by a matrix operation from the R and B signals), the frequency characteristic of the luminance signal matrix for performing the arithmetic processing shown in (Equation 3) is as shown in (Equation 4). The spatial position control C2 that shifts the phase is
It has better high frequency characteristics than C1 which shifts the G signal. Also, in the color signal matrix for performing the arithmetic processing shown in (Equation 5), as shown in (Equation 6), the spatial position control C2 has better high frequency characteristics than C1. As described above, it can be seen that the spatial position control C2 is more suitable for high frequency characteristics in signal processing when interpolation processing is not performed at an arbitrary spatial position .

In a color signal matrix for producing a color difference signal, a false signal is generated in a high frequency band because the high frequency of the color signal is different. This will be described below. For example, when photographing white, the color difference signals RY and BY need to be at the zero level. However, in the case of the spatial position control C2, in the high frequency band, a G signal exists (G ≠ 0), but no RB signal exists (R = B).
B = 0), the color difference signals RY and BY generate false color signals instead of zero level as shown in (Equation 7). FIG. 6 shows a circuit configuration example of the digital signal processing circuit 106 in FIG. 1 for removing the false color signal. In FIG. 6, 601 is a 1H memory for delaying one horizontal line period, 602 is an adder, and 603 is 1/2 for gain adjustment.
An amplifier, 604 is a luminance signal matrix for performing the arithmetic processing shown in (Equation 3), 605 is a chrominance signal matrix for performing the arithmetic processing shown in (Equation 5), and 606 is a phase adjustment circuit composed of the above 601 to 603 , 607 are VLPFs for attenuating high frequency components of the G signal, and 608 and 609 are VLPFs for attenuating high frequency components of the color difference signals RY and BY.

[0038]

(Equation 7)

The signal processing circuit configured as described above will be described below. 6, reference numerals 601 to 605 in FIG.
Same as 505, except for 607, 608, 60
9 VLPF. In FIG. 6A, VLPF6
Reference numeral 07 attenuates the high frequency components of the G signal to make them equal to the R and B signals, thereby reducing false signals generated in the high frequency band of the color difference signal. Similarly, in FIG. 6B, the VLPFs 608 and 609 reduce the false signal generated in the high frequency band by attenuating the high frequency component of the color difference signal. FIG. 6 (c) is a sum of the effects of FIGS. 6 (a) and 6 (b). The high frequency component of the G signal is attenuated, and the high frequency component of the color difference signal is further attenuated. A false signal generated in a frequency band is reduced. By providing the vertical LPF in this manner, it is possible to reduce a false signal generated in a high frequency band, and to obtain an imaging device with a horizontal line interpolation function without a false signal.

Next, an example of the configuration of the interpolation function part shown in the block diagram of the image pickup apparatus with horizontal line interpolation function shown in FIG. 1 is shown in FIG.
One signal will be described with reference to FIG. In the figure,
Reference numerals 701 to 704 denote vertical spatial position / phase compensating units for performing two-line averaging processing (interpolation processing of interpolation coefficients 1/2 and 1/2) shown in FIG.
Reference numeral 4 denotes a phase compensation circuit including a 1H memory 701, an adder 702, and a 1/2 amplifier 703. Also, 7
05 is a field memory, 706 is a switch, and switches the WRITE operation of the 1H line memories 707 to 709 described below. A selector 710 selects two of the line memories 707 to 709. 711,712
Is a multiplier, 713 is an adder, 714 is a vertical interpolation circuit composed of 710 to 713, 715 is a latch circuit for delaying in the horizontal direction, 716 and 717 are multipliers, 718 is an adder, and 719 is 715 to 718. 720 is a vertical interpolation circuit 714 and a horizontal interpolation circuit 719
An electronic zoom circuit composed of In the interpolation function part configured in this way , first, the phase compensation circuit
Phase-shifted color signal.
No interpolation is performed on the same spatial position as other color signals
A signal in which the phase shift is compensated is obtained. After that, the data is stored in the field memory circuit 705, and the control (not shown) of the field memory circuit 705 and the three line memories in the electronic zoom circuit 720 are performed.
For interpolation processing at any vertical spatial position within
Performs an interpolation operation using the calculated data w1 and w2 (w2 = 1−
When w1, primary interpolation is performed). Next, the horizontal interpolation circuit 719 performs interpolation processing at an arbitrary spatial position in the horizontal direction.
Is performed using the multiplied data h1 and h2 (when h2 = 1-h1, primary interpolation is performed). However, the vertical compensation circuit 704 is necessary for a signal whose phase is shifted in the vertical direction, and is not necessary for a signal whose phase is not shifted in the vertical direction.

As described above , the phase is shifted in the vertical direction.
Other color signals that are not vertically phase shifted
Interpolation processing at the same spatial position as the color signal and vertical phase
A phase compensation circuit for obtaining a compensation signal the deviation of any vertical
A vertical interpolation circuit that performs interpolation processing on a spatial position in the vertical direction ;
A horizontal interpolation circuit that performs interpolation processing at a desired horizontal spatial position, thereby shifting the vertical phase of the color signal . An interpolation function for a spatial position can be performed.

FIG. 8 is a block diagram of an image pickup apparatus having a horizontal line interpolation function according to a second embodiment of the present invention. In the figure, reference numerals 801 to 810 are the same as 101 to 110 in FIG. 1 except for a vertical interpolation SW. Circuit 811 and a system control circuit 812 for comprehensively controlling the above circuits. The imaging apparatus having the horizontal line interpolation function configured as described above will be described below focusing on different points.

In FIG. 8, when the interpolation processing is not performed, the vertical interpolation SW. Circuit 811 is turned off, and the image pickup device section 801 passes through the system control circuit 812, the drive control circuit 803, and the image pickup device drive circuit 802 to the R position. Perform a normal drive operation in which the vertical phases of the G and B signals match. On the other hand, when creating a horizontal line by the interpolation processing, vertical interpolation S
The W. circuit 811 is turned on, and the system control circuit 81
2, through the drive control circuit 803 and the image sensor drive circuit 802, the image sensor unit 801 performs a driving operation in which the vertical phases of the R / B signal and the G signal are different by means shown in FIGS. 2 and 3 of the first embodiment. Do.

In the image pickup apparatus having the horizontal line interpolation function according to the present embodiment having such a configuration, in the case of signal processing in which interpolation processing is not performed, spatial positions (phases) coincide with each other. And the LPF processing in the vertical direction does not need to be performed, so that the high frequency characteristics do not deteriorate. In the case of the signal processing for performing the interpolation processing, similarly to the first embodiment, since the deterioration of the frequency response characteristics due to the interpolation processing differs depending on the phase, when the video signal is viewed in all three color signals ( For example, G,
When a luminance signal synthesized by matrix operation from R and B signals is considered), deterioration of the frequency response characteristic can be reduced, and deterioration of the sharpness of the horizontal line interpolation signal in the vertical direction can be reduced.

As described above, according to the present embodiment, by providing the drive control circuit and the vertical interpolation SW. Circuit, when the interpolation processing is not performed, it is possible to obtain a high-resolution image without deterioration of the vertical resolution. When performing the processing, it is possible to reduce the deterioration of the sharpness of the horizontal line interpolation signal in the vertical direction and obtain an image with little image quality deterioration.

FIG. 9 is a block diagram of an image pickup apparatus having a horizontal line interpolation function according to a third embodiment of the present invention. In the figure, reference numerals 901 to 905 denote the same as 101 to 105 in FIG. 1 except for a field memory circuit 906 for storing an output signal of the A / D conversion circuit 905, and a field memory control circuit for controlling the field memory circuit 906. 90
7. A vertical interpolation circuit 908 that performs vertical interpolation processing using an output signal of the field memory circuit 906, a vertical interpolation control circuit 909 that controls the vertical interpolation circuit 908, and a luminance signal and a color signal from the output signal of the vertical interpolation circuit 908. A digital signal processing circuit 910 for generating a color difference signal or the like or an RGB signal processing, a horizontal interpolation circuit 911 for performing horizontal interpolation processing using an output signal of the digital signal processing circuit 910, and a horizontal interpolation for controlling the horizontal interpolation circuit 911 A control circuit 912 and a system control circuit 913 for comprehensively controlling the above circuits.

The operation of the imaging apparatus having the horizontal line interpolation function of the present embodiment having the above-described configuration will be described below. This embodiment is significantly different from the first embodiment in that the electronic zoom circuit is divided into a vertical interpolation circuit 908 and a horizontal interpolation circuit 911. Hereinafter, the electronic zoom function will be described with reference to FIGS.

FIG. 10 shows a configuration example of the vertical electronic zoom function. In the figure, 1001, 1002 and 1003 are:
A field memory circuit that stores R, B, and G signals, 1004 is a field memory control circuit that controls a field memory circuit 1003 (G signal), 1005 is a field memory circuit 1001 (R signal) and 1002
Field memory control circuit for controlling (B signal), 10
Reference numerals 06, 1007, and 1008 denote vertical interpolation circuits that perform vertical interpolation and enlargement processing on the R, B, and G signals from the filled memory circuits 1001, 1002, and 1003, respectively.
9, a vertical interpolation control circuit for controlling the vertical interpolation circuit 1008; 1010, a vertical electronic zoom control circuit for controlling the vertical interpolation circuits 1006 and 1007; 1011, a vertical control circuit included in the system control circuit 913 of FIG. It is a direction control circuit.

FIG. 11 shows a vertical interpolation circuit. As shown in the figure, reference numeral 1109 denotes the entire vertical interpolation circuit.
Reference numerals 101 to 1108 denote 706 to 71 in FIG. 7 of the first embodiment.
Since the configuration is the same as that of No. 3, the description is omitted. Also, R.
The G and B signals have the same configuration, except that the multiplication coefficient (interpolation coefficient) from the vertical interpolation control circuit to the multiplier is R · B
The signal is at v1, v2, and the G signal is at v3, v4.

A description will be given of the vertical electronic zoom function having such a configuration in the case where the vertical phases of the R and B signals are shifted. The R and B signals are applied to the field memory circuit 100
1, 1002 and the field memory control circuit 1005, and the vertical interpolation circuits 1006 and 1007 and the vertical interpolation control circuit 1010, and as shown by the G signal in FIG. / 2 lines) and vertical interpolation coefficients v1 and v2 determined from the vertical interpolation coefficient w to interpolate the horizontal lines. The G signal is controlled by a field memory circuit 1003 and a field memory control circuit 1004, and further by a vertical interpolation circuit 1008 and a vertical interpolation control circuit 1009.
As shown by the B signal, the horizontal line is interpolated by the calculation using the vertical interpolation coefficients v3 and v4 determined from the vertical interpolation coefficient w. Next, after vertical interpolation processing, the RGB signal or Y (luminance) and C (color) are processed by the digital signal processing circuit 910.
, And the horizontal interpolation circuit 911 performs horizontal interpolation processing. Since this horizontal interpolation circuit can be constituted by a circuit similar to the horizontal interpolation circuit 719 in FIG. 7, the description is omitted here.

As described above, the vertical interpolation circuits 1006 and 1006
07, the phase compensation function for the phase shift p of the color signal in the vertical direction and the interpolation processing to an arbitrary spatial position
Perform the vertical interpolation function for the interpolation coefficient w at the same time,
Circuits for vertical phase compensation can be reduced, and an imaging device with a horizontal line interpolation function can be obtained with a small circuit scale.

FIG. 12 is a block diagram showing a digital signal processor of an image pickup apparatus having a horizontal line interpolation function according to a fourth embodiment of the present invention. In the figure, reference numeral 1201 denotes R ·
A digital signal processing circuit for obtaining two types of luminance signals Y1 and Y2 and two types of color signals C1 and C2 from the G / B signals;
202 to 1205 are field memories for storing signals, 1206 is a field memory control circuit for controlling the field memory, 1207 is Y1, Y2, C1, C
Electronic zoom circuit 1208 for performing interpolation and enlargement using 2
Is a system control circuit for comprehensively controlling the digital signal processing circuit, the field memory control circuit, and the electronic zoom circuit. FIG. 13 is a block diagram showing a configuration of the digital signal processing circuit 1201 in FIG. Referring to FIG. 13, reference numeral 1301 denotes a line memory for storing a signal for a 1H period, 1302 an adder, and 1303 a gain adjustment.
A 1/2 amplifier, 1304 is a selector circuit for selecting two signals R1 and R2 from three signals based on information from a system control circuit;
1305, 1306, and 1307 are 1301 to 130, respectively.
4, a signal selection circuit 1308 includes a luminance signal Y1.
, And 1309 is a luminance signal Y
2 is a Y2 matrix circuit, and 1310 is a color signal C.
1 is a C1 matrix circuit, and 1311 is a color signal C.
2 is a C2 matrix circuit, and 1312 is 13
This is a matrix circuit composed of 08 to 1311.

The operation of the imaging apparatus having the horizontal line interpolation function configured as described above will now be described with reference to FIGS.
6 will be described. In FIG. 13, the signal selection circuit 1
In steps 305 to 1307, three signals are generated from two consecutive lines of signals. This is shown in FIG. FIG. 14 shows a signal generated from an RGB signal in which the phase of the RGB signal is shifted from the G signal by 1/2 line in the vertical direction. For example, in the RB signal, the (n-2) line and n
An (m-1) line interpolation signal is created from the line signal, and an (m + 1) line interpolation signal is created from the n line and (n + 2) line signals. Similarly, the (n-1) line and (n + 1) line are created in the G signal. ) An m-line interpolation signal is created from the line signal, and an (m + 2) -line interpolation line signal is created from the (n + 1) -line and (n + 3) -line signals. In this way, three consecutive signals of two lines including the interpolation signal are obtained.
Create a line signal. Next, the selector 1304 in FIG. 13 selects two signals from the three signals. This is shown in FIGS. In FIG. 15, similarly to FIG. 14, the interpolation line h1 created from the RGB signal in which the phase of the RGB signal is shifted by 1/2 line in the vertical direction with respect to the G signal.
FIG. 16 shows an interpolation line h2 similarly created. As shown in FIG. 15, signals R _m−1 , necessary to generate an interpolation line h1 between the (n−1) line and the n line,
G _m−1 , B _m−1 and R _m , G _m , B _m are given by (Equation 8), and luminance signals Y _m−1 , Y _m and color difference signals (RY) _m−1 , ( R-
Y) _m and _{(B-Y) m-1} , shown in (a B-Y) _m (Equation 9).

[0054]

(Equation 8)

[0055]

(Equation 9)

Further, as shown in FIG. 16, n lines and (n
+1) The signals R _m , G _m , B _m and R _{m + 1} , G _{m + 1} , B _{m + 1} required to create the interpolation line h2 between the lines are given by
0), the created luminance signals Y _m and Y _{m + 1} and the color difference signal (R
−Y) _m , (RY) _{m + 1} and (BY) _m , (BY) _{m + 1} are shown in (Equation 11).

[0057]

(Equation 10)

[0058]

[Equation 11]

As described above, by selecting two signals necessary to create an interpolation line from three signals created from two consecutive lines of signals, an interpolation signal can be created at an arbitrary position. . The selection of the two signals shown performs the control of the selector circuit system control circuit, at the time of creating an interpolation line h1 Y _m-1 and interpolation line h2
Matrix operation when creating Y _m Y1 matrix circuit 1
308, a matrix computation of Y _{m + 1} of Y _m and interpolation line h2 creation when creating interpolation line h1 is Y2 matrix circuit 1309 performs. Similarly, interpolation line h1 when creating (R-Y) _{m over 1,} (B-Y) _m-1 and interpolation line h2 when creating _{(R-Y) m, (} B-Y) m matrix calculation At the time when the C1 matrix circuit 1310 creates the interpolation line h1.
_{Y) m, (B-Y} ) m and interpolation line h2 when creating (R-Y)
_The C2 matrix circuit 1311 performs a matrix operation of _{m + 1} · (BY) _{m + 1} . In the C1 and C2 matrices, the (RY) signal and the (BY) signal are thinned out and output in a time-division manner. FIG. 17 shows an output signal of the digital signal processing circuit 1201 of FIG. 12 which performs the above operation. In FIG. 17, for example, (Y _m-1,1 ) represents the luminance signal of the first pixel on the (m-1) line. FIG. 17A shows an output signal when the interpolation line h1 signal shown in FIG. 15 is created, and FIG. 17B shows an output signal when the interpolation line h2 signal shown in FIG. 16 is created. Are decimated and time-series. In FIG. 10A, the (m-1) line and the m line signal are output, and in FIG. 10B, the m line and the (m + 1) line signal are output.

Next, the output signal of the digital signal processing circuit 1201 in FIG.
2 to 1205 and then stored in the electronic zoom circuit 120
In step 7, an interpolation operation is performed using the above signal from the field memory. The interpolation operation is performed using signals of two lines having a positional relationship of the frame signal. For example, in the case shown in FIG. 15, the signal of the (m-1) line and the m line having the positional relationship of the frame signal is used, and in the case of FIG. 16, the m line and the (m +
1) Interpolation calculation is performed using the signal of the line. FIG. 18 shows a configuration example of this electronic zoom circuit. In FIG.
01 is a switch, 1802 and 1803 are line memories, 1
804 is a line memory control circuit, 1805 is a selector circuit, 1806 and 1807 are vertical line memory control circuits, 1808 and 1809 are multipliers for vertical interpolation, 181
0 is an adder, 1811 is a latch circuit for providing a delay in the horizontal direction, 1812 and 1813 are multipliers for horizontal interpolation, 18
14 is an adder, 1815 is a horizontal interpolation circuit. Hereinafter, the operation thereof will be described focusing on differences from the electronic zoom circuit 720 of FIG. In FIG. 18, Y1, Y for performing the interpolation calculation
2 or signals of two lines C1 and C2 are input, and a vertical interpolation operation is performed using each signal and vertical interpolation data. Therefore, in the vertical line memory control circuits 1806 and 1807, one of the two line memories is used for Wright and the other is used for Re.
It is controlled for ad use.

As described above, according to this embodiment, Y1, Y
By providing a digital signal processing circuit and an electronic zoom circuit for generating the 2 and C1 and C2 signals, an interpolation signal can be generated from two lines of the luminance signal and the color difference signal in a positional relationship of the frame signal with little image quality deterioration. Therefore, it is possible to reduce the deterioration of the sharpness of the horizontal line interpolation signal in the vertical direction and obtain an image with little deterioration in image quality.
Further, since two lines of signals necessary for interpolation are input to the electronic zoom circuit, the number of line memories in the electronic zoom circuit is small, and the circuit scale can be reduced.

In the above embodiment, the drive control of the image pickup device is performed to shift the phases of the three color signals R, G, B. In addition to the drive control, for example, three drive signals for obtaining three color signals are provided. When the solid-state imaging device is bonded and fixed to the color separation prism, the position is shifted in the vertical direction, unlike the conventional case, or the color signal is bent by manipulating the refractive index inside the three-color separation prism to bend the optical path of light. May be shifted. In this case, p1, p2, p3
Is in the range of 0 to less than 1, but is not limited thereto. For example, it is possible to set p1 = 1.5. In this case, the same effect as when p1 = 0.5 is obtained ( The same applies to p2 and p3).

In the above embodiment, the case where the three color signals are the output signals of the solid-state image pickup device has been described. The same applies to the signals stored in the field memory or the frame memory. As means for shifting the phase, memory read control can be performed.

In the above-described embodiment, only the output of the R, G, and B signals has been shown for the image pickup device section.
The configuration of a three-chip imaging device that obtains a G / B signal or a two-chip imaging device that has two imaging devices and obtains a G signal while one imaging device obtains an R signal and a B signal can be considered.

In the first embodiment, the encoder circuit converts the RGB signals from the electronic zoom circuit into NTS signals.
Although the case where the signal is converted into a C signal is shown, and in the other embodiments, the portion up to the point where the RGB signal or the YC signal is output from the electronic zoom circuit is shown, the signal is converted into an NTSC signal or another signal. Is also possible.

In the above embodiment, the three color signals are R, G, and B. However, the present invention is not limited to this. For example, three color signals of yellow, cyan, and magenta can be used. is there.

Further, in the above-described embodiment, only the control of the line memory has been described with respect to the interpolation circuit. However, the control of reading from the solid-state imaging device or the field memory or the like is also required. The control can be performed according to each of the B signal or the Y and C signals, but is not limited thereto.

Further, in the above embodiment, the interpolation processing is shown as a case of linear interpolation. However, the present invention is not limited to this. Higher-order interpolation processing such as secondary interpolation using three or more line memory output signals is performed. Obviously this is possible.

Further, the contents shown in each embodiment can be used in combination with other embodiments. For example, although the false color signal removal has been described only in the first embodiment, it can be used in combination in the second, third, and fourth embodiments.

Further, in the fourth embodiment, the case where the Y1 and Y2 signals and the C1 and C2 signals are output as pseudo frame signals (frame position signals created by interpolation) has been described. A pseudo frame signal of (R1, R2), (G1, G2), (B1, .B2) is output from each of the G and B signals, or the Y signal is a pseudo frame in consideration of the sensitivity to the resolution of human eyes. Signal, C
The signal may output a normal field signal.

[0071]

As described above, according to the present invention, means for obtaining three color signals (for example, a plurality of solid-state image pickup devices) and one of the three color signals are used in the vertical direction of the remaining two color signals. The phase of the direction is a fixed position interval different for each of the two signals
A phase shift unit for only shifted, not phase shifted color
Double the signal and the two color signals phase shifted vertically
A frame calculation circuit for obtaining a signal having a number of lines, Ren
Interpolation circuit and interpolator for obtaining interpolated horizontal line signal at any spatial position
According to this configuration, by interpolating three color signals having different phases when interpolating a horizontal line from the three color signals, the interpolation coefficient is changed, so that the video signal can be viewed as a whole of the three color signals. (Eg, G, R, B
When considering the luminance signal synthesized by matrix operation from the signal), the deterioration of the vertical frequency response characteristic due to the interpolation processing can be reduced, and it is possible with less circuit configuration.
Interpolation processing can be performed at the spatial position of , and high-quality horizontal line interpolation can be performed.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of an imaging device with a horizontal line interpolation function according to a first embodiment of the present invention.

FIG. 2 is an explanatory diagram for explaining frame accumulation drive control of an image sensor according to the first embodiment.

FIG. 3 is an explanatory diagram for explaining field accumulation drive control of the image sensor in the first embodiment.

FIG. 4 is an explanatory diagram for explaining color signal processing in the first embodiment.

FIG. 5 is a block diagram showing an example of the internal configuration of the digital signal processing circuit according to the first embodiment;

FIG. 6 is a block diagram showing an example of the internal configuration of the digital signal processing circuit according to the first embodiment;

FIG. 7 is a block diagram showing a configuration of an interpolation function part in the first embodiment.

FIG. 8 is a block diagram illustrating a configuration of an imaging device with a horizontal line interpolation function according to a second embodiment of the present invention.

FIG. 9 is a block diagram illustrating a configuration of an imaging device with a horizontal line interpolation function according to a third embodiment of the present invention.

FIG. 10 is a block diagram showing a configuration of a vertical electronic zoom circuit according to the third embodiment.

FIG. 11 is a block diagram showing a configuration of a vertical interpolation circuit according to the third embodiment.

FIG. 12 is a block diagram illustrating a configuration of a digital signal processing unit of an imaging device with a horizontal line interpolation function according to a fourth embodiment of the present invention.

FIG. 13 is a block diagram showing a configuration of a digital signal processing circuit according to the fourth embodiment.

FIG. 14 is a first explanatory diagram for explaining horizontal line interpolation in the fourth embodiment;

FIG. 15 is a second explanatory diagram for explaining horizontal line interpolation in the fourth embodiment;

FIG. 16 is a third explanatory diagram for explaining horizontal line interpolation in the fourth embodiment;

FIG. 17 is an explanatory diagram for explaining output signals of a digital signal processing circuit in the fourth embodiment.

FIG. 18 is a block diagram showing an example of the internal configuration of an electronic zoom circuit according to the fourth embodiment.

FIG. 19 is a block diagram showing a configuration of a conventional image stabilization device having an interpolation function.

FIG. 20 is a block diagram illustrating a configuration of a conventional imaging apparatus with an interpolation function.

[Explanation of symbols]

101, 801, 901 image pickup device unit 102,802,902 pickup device driving circuit 103,803,903 drive control circuit 104,804,904 analog signal processing circuit 105,805,905 analog - digital converter 106,806,910, 1201 Digital signal processing circuit 107, 705, 807, 906, 1001 to 100
3, 1202 to 1205 Field memory circuit 108, 808, 907, 1004, 1005, 120
6 Field memory control circuit 109, 809, 1207 Electronic zoom circuit 110, 810 Electronic zoom control circuit 111, 812, 913, 1208 System control circuit 112 Encoder circuit 501, 601, 701, 707-709, 1102-
1104, 1301, 1802, 1803 1H line memory 502, 602, 702, 713, 718, 1108,
1302, 1810, 1814 Adders 503, 603, 703, 1303 1/2 amplifier circuit 504, 604 Luminance signal matrix circuit 505, 605 Color signal matrix circuit 606 Phase adjustment circuit 607, 608, 609 Vertical LPF (VLPF) 704 Phase compensation Circuit 706, 1101, 1801 Switch 710, 1105, 1304, 1805 Selector circuit 711, 712, 716, 717, 1106, 110
7, 1808, 1809, 1812, 1813 Multipliers 714, 908, 1006 to 1008, 1109 Vertical interpolation circuit 715, 1811 Latch circuit 719, 911, 1815 Horizontal interpolation circuit 720 Electronic zoom circuit 811 Vertical interpolation SW. Circuit 909, 1009 , 1010 vertical interpolation control circuit 912 horizontal interpolation control circuit 1011 vertical direction control circuit 1305 to 1307 signal selection circuit 1308 Y1 matrix circuit 1309 Y2 matrix circuit 1310 C1 matrix circuit 1311 C2 matrix circuit 1312 matrix circuit 1804 line memory control circuit 1806, 1807 vertical Line memory control circuit

────────────────────────────────────────────────── ─── Continuation of the front page (56) References JP-A 1-261086 (JP, A) JP-A 5-191811 (JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB name) H04N 9/04-9/11

Claims (16)

(57) [Claims] 1. Three different color signals C1, C2 and C3
An imaging means for obtaining a first vertical phase shift means for phase-shifting by a predetermined position interval p1 (p1 ≠ 0) in the vertical direction of the color signal C2 with respect to the color signal C1, the chrominance signal C1 On the other hand, a second vertical phase shift means for shifting the vertical phase of the color signal C3 by a fixed position interval p2 (p2 色 0), and a color signal whose phase is shifted in the vertical direction with respect to the color signal C1. From C2 and C3 , interpolated horizontal line signals
Signal is created by interpolation processing, and the vertical phase
An arithmetic circuit for compensating for the displacement; and a signal arbitrarily enlarged from the output signal of the arithmetic circuit.
Horizontal line interpolation function imaging apparatus having an enlarged section for obtaining, a. 2. The three different color signals C1, C2 and C3.
2. An imaging apparatus with a horizontal line interpolation function according to claim 1, wherein the imaging means for obtaining the image data comprises a plurality of solid-state imaging devices. 3. The plurality of solid-state imaging devices include a first solid-state imaging device that obtains a color signal C1, a second solid-state imaging device that obtains a color signal C2, and a third solid-state imaging device that obtains a color signal C3. The imaging device with a horizontal line interpolation function according to claim 2. 4. The horizontal line interpolation function according to claim 2, wherein the plurality of solid-state imaging devices include a first solid-state imaging device for obtaining a color signal C1 and a second solid-state imaging device for obtaining color signals C2 and C3. Imaging device. 5. The apparatus according to claim 1, wherein said first vertical phase shift means and said second vertical phase shift means include a drive control circuit for controlling a drive circuit for driving a plurality of solid-state imaging devices. An imaging device with a horizontal line interpolation function according to 2, 3, or 4. 6. The solid-state imaging device according to claim 1, wherein the first vertical phase shift means and the second vertical phase shift means determine a mounting position of the solid-state imaging device for obtaining the color signals C2 and C3 among the plurality of solid-state imaging devices. P1 and p2 spatially with respect to the device
5. The image pickup apparatus with a horizontal line interpolation function according to claim 2, wherein the image pickup apparatus is arranged to be shifted only in the vertical direction. 7. A first vertical phase shift means and a second vertical phase shift means comprises a drive control circuit for controlling the driving circuit for driving a plurality of the solid-state imaging device, among the plurality of solid-color signals Wherein the mounting positions of the solid-state imaging devices for obtaining C2 and C3 are spatially shifted vertically by p1 and p2 with respect to the solid-state imaging device for obtaining the color signal C1. Item 4. The imaging device with a horizontal line interpolation function according to item 2, 3 or 4. 8. The three different color signals C1, C2 and C3.
8. The image pickup apparatus with a horizontal line interpolation function according to claim 1, wherein 3, are three color signals R, G and B. 9. The imaging device with a horizontal line interpolation function according to claim 8, wherein the color signal C1 is a color signal G. 10. The imaging device with a horizontal line interpolation function according to claim 1, wherein p1 and p2 are set such that p1 = p2 . 11. The values of p1 and p2 are 0 <p1 <
2. The method according to claim 1, wherein 1,0 <p2 <1.
An imaging device with a horizontal line interpolation function according to 2, 3, 4, 5, 6, 7, 8, 9, or 10. 12. The apparatus according to claim 1, wherein p1 and p2 are amounts corresponding to one half of the vertical spacing of each color signal line. , 9,10
Or an imaging device with a horizontal line interpolation function according to 11. 13. The color signal C1 is replaced by an interpolation coefficient w (0 ≦ w <1) instead of an arithmetic circuit and an enlarging means.
Performs enlargement processing on the for color signal C2 w and p1
Performs interpolation / enlargement processing using the interpolation coefficient determined from
Interpolation / enlargement processing is performed on the color signal C3 using an interpolation coefficient determined from w and p2, and enlargement is performed in phase with each other.
Vertical interpolation means for obtaining a processed interpolated horizontal line signal
Claim 1,2,3,4,5,6, characterized in that to obtain,
An imaging device with a horizontal line interpolation function according to 7, 8, 9, 10, 11 or 12. 14. An arithmetic circuit according to claim 5 , wherein said arithmetic circuit is provided in a direction perpendicular to said color signal C1.
From the phase shifted color signals C2 and C3,
In-phase interpolated horizontal line signals with twice the number of lines
Signal is created by interpolation processing, and the vertical phase
7. A method for compensating for a deviation and obtaining at least one of a luminance signal and a color difference signal from the interpolated horizontal line signal .
An imaging device with a horizontal line interpolation function according to 7, 8, 9, 10, 11 or 12 . 15. A drive control circuit performs frame accumulation drive control, reads out a signal of a pixel corresponding to one field from one of the plurality of solid-state imaging devices at the same time, and reads the remaining signals. 8. The imaging device with a horizontal line interpolation function according to claim 5, wherein a signal of a pixel corresponding to the other field is read from the solid-state imaging device. 16. A driving control circuit performs field accumulation driving control, reads out signals of pixels corresponding to one field from some of the plurality of solid-state imaging devices at the same time, and reads the remaining signals. 8. The imaging device with a horizontal line interpolation function according to claim 5, wherein a signal of a pixel corresponding to the other field is read from the solid-state imaging device.