JP4029687B2 - Solid-state imaging device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention includes a plurality of CCD image sensors (Charge Coupled Devices), and at least one CCD is spatially shifted, and a high-frequency replacement component is output from the shifted CCD and other CCD output signals. The present invention relates to a solid-state imaging device that employs a spatial pixel shifting method that generates and obtains high resolution by replacing the high-frequency component of each channel with the generated high-frequency replacement component.
[0002]
[Prior art]
Conventionally, in a video camera including a CCD as an image sensor, a spatial pixel shifting method is used to remove a folding signal due to CCD sampling and realize a higher resolution image.
[0003]
FIG. 17 shows a configuration diagram for realizing the conventional spatial pixel shifting method. This example is an example of a spatial pixel shift method in a three-panel video camera. 1G is a CCD for green images, 1B is a CCD for blue images, 1R is a CCD for red images, 2G, 2B, 2R are correlated double sampling Circuit (CDS circuit), 3G is a delay line, 4G, 4B and 4R are A / D converters, 5G, 5B and 5R are low pass filters, 6G and 6R are high pass filters, 7G and 7R are dividers, 8, 9G, 9B and 9R are adders, and 10 is a non-linear processing circuit.
[0004]
Incident light from the subject is divided into R, G, and B by an optical low-pass filter and RGB color separation prism not shown here, and each divided color light is photoelectrically converted by the CCD for each channel To do.
[0005]
Thereafter, A / D conversion is generally performed at the same clock rate as the CCD readout clock, and digital signal processing such as gamma correction processing is performed. For example, a digital signal processing camera having a 520,000 pixel CCD uses a frequency of about 18 MHz, and a HDTV digital signal processing camera having a 2 million pixel CCD uses a clock having a frequency of about 74 MHz. In this case, the limit resolution of the obtained signal is limited to the Nyquist frequency fs / 2 or less which is 1/2 of the clock frequency fs.
[0006]
The optical low-pass filter has a characteristic that removes the above-described component of the Nyquist frequency fs / 2 or higher. In general, however, it is difficult to obtain a characteristic that the optical filter sharply drops at the cutoff frequency fs / 2. Yes, as shown in FIG.
[0007]
Here, the component having the Nyquist frequency fs / 2 or higher that could not be removed by the optical filter becomes a folded component as shown by the hatched line in FIG. 19 due to spatial sampling by the CCD, and a false signal is generated.
[0008]
As shown in FIG. 20, the CCD of each channel is arranged such that 1G, which is a CCD for a green image, is shifted by a half pixel space from 1B, which is a CCD for a blue image, and 1R, which is a CCD for a red image. The CCD is driven using the same clock pulse on all three plates, and the video signal is read out. Therefore, as the output from the CCD, a signal in which only the green signal is spatially shifted by half the pixel pitch S is obtained. The signal output from the CCD is sampled and held by the respective CDS circuits 2G, 2B, and 2R, and thereafter, only the green signal corresponds to a half pixel by the delay line 3G in order to correct a spatial shift of half a pixel. Delay time 1 / (2 × fs).
[0009]
Since the green signal and the red signal are spatially shifted by half a pixel and sampled, the aliasing components near the Nyquist frequency fs / 2 are out of phase with each other. That is, the aliasing component can be canceled out by taking the average of the red component and the green component.
[0010]
The above is the principle that can realize a high-resolution image with few false signals by the spatial pixel shifting method.
[0011]
The high-frequency replacement process is generally performed as follows. A high-frequency low-frequency dividing circuit 11 composed of low-pass filters 5G, 5B, 5R and high-pass filters 6G, 6R divides the low-frequency components GL, BL, RL and high-frequency components GH, RH.
[0012]
Next, a high frequency replacement component generation circuit 12 generates a high frequency replacement component. The green and red high frequency components GH and RH are each halved by the divider and added by the adder 8 to obtain a high frequency replacement component H.
[0013]
H = (GH + RH) / 2 (Formula 1)
Next, in the adders 9G, 9B, and 9R, the high frequency permutation component H is added to the low frequency components GL, CL, and RL of the respective channels to become signals GP, BP, and RP. Thereafter, nonlinear processing such as gamma processing and knee processing is performed in the nonlinear circuit 10, and signals Go, Bo, and Ro are output.
[0014]
Next, the operation of the high-frequency replacement process will be described using the signal waveform shown in FIG. For example, as shown in FIG. 21A, as a signal output from the CCD, the low frequency components Rdc, Bdc, and Gdc of each channel have the same value, and the high frequency component of only a red stripe pattern (amplitude w). If you have
[0015]
Further, the low-pass filters 5G, 5B, 5R, and the high-pass filters 6G, 6R are assumed to have frequency characteristics as shown in FIG. 22, and the frequency component of the striped pattern is higher than the cut-off frequency ft of the filter and from the Nyquist frequency fs / 2. Is also low.
[0016]
The signal input to the non-linear processing circuit 10 is a 12-bit input signal and is converted to 10 bits by a gamma circuit and a knee circuit. In other words, the input ranges from 0 to 4095 in 4096 steps, and the output ranges from 0 to 1023 in 1024 steps.
[0017]
Since there is no green high-frequency component and only a red high-frequency component (amplitude w) is present as shown in FIG. 21A, the high-frequency replacement component H is a signal having an amplitude w / 2. Thereafter, since it is added to the respective low frequency components, Rdc, Gdc, and Bdc, signals GP, BP, and RP having the same high frequency components are obtained as shown in FIG.
[0018]
Thereafter, gamma processing and knee processing are performed in the non-linear processing circuit, and signals Ro, Go, and Bo as shown in (c) are obtained.
[0019]
Here, paying attention to the high-frequency component, the green, blue, and red components are composed of exactly the same signal, so that the color becomes achromatic and a black and white striped pattern is generated. However, in general, human vision has high resolution for light and dark, but low resolution for color. For example, the use band is limited by making the high-frequency component colorless. In some cases, the bandwidth can be used effectively.
[0020]
[Problems to be solved by the invention]
However, when the incident light from the subject has a specific color component that protrudes more than the other color components at high and low frequencies, the following problem occurs.
[0021]
For example, as shown in FIG. 23, as a signal output from the CCD, the red low-frequency component Rdc is larger than the green low-frequency component Gdc and the blue low-frequency component Bdc, and only the red stripe pattern (amplitude w) is high. Consider a case with band components.
[0022]
Here, as shown in FIG. 6A, since there is no green high-frequency component and there is only a red high-frequency component (amplitude w), the high-frequency replacement component H is a signal having an amplitude w / 2. . Thereafter, the signals GP, BP, and RP having the same high frequency components as shown in (b) are obtained because they are added to the respective low frequency components, Gdc, Bdc, and Rdc.
[0023]
Thereafter, gamma processing and knee processing are performed in the non-linear processing circuit, which are processing dependent on the low-frequency components Gdc, Bdc, and Rdc of each color. That is, as shown in (c), the red component with many low-frequency components is compressed by the knee processing, and the red high-frequency component is reduced.
[0024]
In other words, the incident light from the subject should have originally been an achromatic, red, achromatic, and red striped pattern from the left of the screen, but by applying the above high-frequency replacement processing and non-linear processing, It turns into a striped pattern of chromatic, red and achromatic colors. The high-frequency replacement process should replace the red stripe pattern with an achromatic stripe pattern, but it will generate a color component that is different from the original image, resulting in a significant reduction in color reproducibility and degradation in image quality. Will be invited.
[0025]
Further, as the problem solving means, a configuration as shown in FIG. 24 is conceivable. This circuit has a configuration in which the high-frequency and low-frequency division circuit 11, the high-frequency replacement component generation circuit 12, and the adders 9G, 9B, and 9R shown in FIG. This circuit is intended to first perform nonlinear processing on the converted digital signal and then perform high-frequency replacement processing.
[0026]
Similarly, as shown in FIG. 25 (a), as the signal output from the CCD, the red low frequency component is larger than the green and blue components, and in addition, only the red high frequency band is shown (amplitude w). Assuming that the signal has a component, as shown in FIG. 5B, signals GQ, BQ, and RQ in which the red high frequency component is compressed by knee processing are obtained. Here, the blue component and the green component do not have a high frequency. Thereafter, high-frequency replacement is performed by the high-frequency low-frequency division circuit 11, the high-frequency replacement component generation circuit 12, and the adders 9G, 9B, and 9R. Even if the replacement is performed, the output signals Ro, Go, and Bo are signals having almost no high-frequency component, as shown in (c). That is, the above-described problem that the color of the high-frequency component is changed is solved, but the high-frequency component itself is small, so that the image does not have a sense of resolution.
[0027]
[Means for Solving the Problems]
At least one channel of red, blue and green other 2 Spatial to the solid-state image sensor 1 / Solid-state imaging device arranged by shifting 2 pixels,From the solid-state image sensorThe output red, blue and greeneachColor componentA / D converter for analog-digital conversion of signal and signal output from the A / D converterAbout the low frequency component of each color and at least among red, blue and green 2 Outputs the high frequency component of the colorFirst high-frequency low-frequency division circuitAnd at least two color high frequency components output from the first high frequency low frequency dividing circuit are added.First high-frequency replacement component generation circuitAnd a first high-frequency component that outputs the low-frequency component of each color output from the first high-frequency low-frequency dividing circuit and the output of the first high-frequency replacement component generation circuitAn adder;For the output signal of the first adderFrom the non-linear processing circuit that performs non-linear processing and the above non-linear processing circuitOutputsignalAbout the low frequency component of each color and at least among red, blue and green 2 Divide into high frequency components of colorSecond high-frequency low-frequency division circuitAnd at least output from the second high frequency / low frequency dividing circuit 2 Add high frequency component of colorSecond high-frequency replacement component generation circuitAnd a second high-frequency component dividing circuit that adds the low-frequency component of each color output from the second high-frequency component dividing circuit and the output of the second high-frequency replacement component generating circuit.AdderWhenThe solid-state imaging device is characterized in that high-frequency replacement is performed before and after the nonlinear processing circuit.
[0028]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described below.
[0029]
(Embodiment 1)
FIG. 1 is a diagram illustrating a configuration of a solid-state imaging device to which the present invention is applied. In FIG. 1, 1G is a CCD for green image, 1B is a CCD for blue image, 1R is a CCD for red image, 2G, 2B, 2R are correlated double sampling circuits (CDS circuits), 3G is a delay line, 4G, 4B, 4R is an A / D converter, 11 is a first high-frequency / low-frequency dividing circuit including low-pass filters 5G, 5B, 5R, and high-pass filters 6G and 6R, and 12 is a first circuit including dividers 7G and 7R and an adder 8. 1 high-frequency permutation component generation circuit, 9G, 9B and 9R are adders, 10 is a non-linear processing circuit, 18 is a second high-frequency low-frequency band consisting of low-pass filters 13G, 13B and 13R, and high-pass filters 14G and 14R. The dividing circuit 19 is a second high-frequency permutation component generating circuit composed of dividers 15G and 15R and an adder 16, and 17G, 17B and 17R are adders.
[0030]
As in the conventional example, as shown in FIG. 2, as a signal output from the CCD, the red low-frequency component is larger than the green and blue components, and furthermore, only the red has a high-frequency component with a striped pattern (amplitude w). Think about the case.
[0031]
First, the high-frequency replacement component H1 is obtained by the first high-frequency low-frequency division circuit 11 and the first high-frequency replacement component generation circuit 12.
[0032]
H1 = (GH1 + RH1) / 2 (Formula 2)
Thereafter, the above components are added by the adders 9G, 9B, and 9R in common to each channel. As a result, the signals GP, BP, and RP are high frequency components having an amplitude w / 2 as shown in FIG. Will have. Thereafter, gamma processing and knee processing are performed in the non-linear processing circuit, which are processing dependent on the low-frequency components Gdc, Bdc, and Rdc of each color. That is, as shown in FIG. 5C, the red component with many low frequency components is compressed by knee processing, and as a result, the red high frequency component is reduced.
[0033]
Thereafter, the high-frequency replacement is performed again by the second high-frequency low-frequency dividing circuit 18, the second high-frequency component generating circuit 19, and the adders 17G, 17B, and 17R. In other words, the high-frequency replacement component H2 is generated from the red high-frequency component compressed by the knee processing and the green high-frequency component, and added again to each channel.
[0034]
H2 = (GH2 + RH2) / 2 (Formula 3)
Then, as shown in FIG. 4D, the green high-frequency component that has not been compressed in the non-linear processing returns again to the red high-frequency component, and the red high-frequency component is reproduced. . As is clear from comparison with FIG. 23C of the conventional example, since the same high frequency component exists in each channel, an achromatic stripe pattern is reproduced when attention is paid to the high frequency component. Further, as apparent from the comparison with FIG. 25C, since there are many high frequency components, it is possible to obtain a higher resolution image quality than in the conventional example.
[0035]
As described above, when a specific color component is prominent in the signal output from the CCD, color reproduction in the high frequency generated by compressing the high frequency component replaced in the previous stage of the non-linear processing. Degradation and resolution reduction that occurs when high-frequency replacement is performed later in the non-linear processing can be prevented, and higher-quality video can be reproduced.
[0036]
In the present invention, the first high-frequency low-frequency division circuit, the second high-frequency low-frequency division circuit, the first high-frequency replacement component generation circuit, and the second high-frequency replacement component generation circuit are Even if it is not the structure shown to a form, it can implement | achieve also in the different circuit structure which performs the same calculation.
[0037]
Although the present invention is applied to the horizontal space pixel shifting method, it can also be realized in the vertical space pixel shifting method by performing processing between lines.
[0038]
Further, in this embodiment, the green CCD is arranged in a half-pixel spatial manner and the high-frequency replacement component is generated from the red component and the green component. However, a similar pixel shifting method is performed from any combination of channels. May be.
(Embodiment 2).
[0039]
FIG. 3 is a diagram illustrating a configuration of a solid-state imaging device to which the present invention is applied. In FIG. 3, 1G is a CCD for a green image, 1B is a CCD for a blue image, 1R is a CCD for a red image, 2G, 2B, 2R are correlated double sampling circuits (CDS circuits), 3G is a delay line, 4G, 4B, 4R is an A / D converter, 11 is a first high-frequency / low-frequency dividing circuit including low-pass filters 5G, 5B, 5R, and high-pass filters 6G and 6R, and 12 is a first circuit including dividers 7G and 7R and an adder 8. 1 high frequency permutation component generation circuit, 9G, 9B, 9R are adders, 10 is a non-linear processing circuit, 18 is a second high frequency band consisting of low pass filters 13G, 13B, 13R, and high pass filters 14G, 14B, 14R. A low-frequency division circuit 19 is a second high-frequency replacement component generation circuit including a selector 20, and 17G, 17B, and 17R are adders.
[0040]
Hereinafter, the present invention will be described generally and conceptually, but the description of the same parts as those in Embodiment 1 will be omitted.
[0041]
As shown in FIG. 4, a case is considered in which the signal output from the CCD has more red low-frequency components than green and blue components, and in addition, only red has a high-frequency component with a striped pattern (amplitude w).
[0042]
Here, the processing performed before the nonlinear processing is equivalent to that of the first embodiment, and the signals GP, BP, and RP inputted to the nonlinear processing circuit have an amplitude w / 2 as shown in FIG. It will have a high frequency component. Thereafter, gamma processing and knee processing are performed in the non-linear processing circuit to obtain signals GQ, BQ, and RQ shown in (c).
[0043]
Thereafter, the high-frequency replacement is performed again by the second high-frequency low-frequency dividing circuit, the second high-frequency component generating circuit, and the adder. Here, in the second high-frequency low-frequency dividing circuit, a high-pass filter is provided for each channel, and the high-frequency components GH2, BH2, and RH2 filtered by the high-pass filter are input to the selector 20. The low-frequency components GL2, BL2, and RL2 filtered by the other low-pass filters 13G, 13B, and 13R are input to the selector. The selector 20 compares the low frequency components GL2, BL2, and RL2, selects the signal having the lowest value, and outputs the high frequency component of the same channel as the selected channel as the high frequency replacement component H2.
[0044]
For example, in the case of FIG. 4, Rdc is the largest, and Bdc and Gdc are the same and the lowest. That is, in this case, one of the blue high-frequency component BH2 and the green high-frequency component GH2 is output as the high-frequency replacement component H2, and high-frequency replacement is performed.
[0045]
In other words, since the channel having the lowest low-frequency component, which is the signal that is most difficult to be compressed by the knee processing, is selected and high-frequency replacement is performed, a signal having many high-frequency components is always replaced. Compared with, higher resolution image quality can be obtained.
[0046]
As described above, when a specific color component is prominent in the signal output from the CCD, color reproduction in the high frequency generated by compressing the high frequency component replaced in the previous stage of the non-linear processing. Degradation and resolution reduction that occurs when high-frequency replacement is performed later in the non-linear processing can be prevented, and higher-quality video can be reproduced.
[0047]
In the present invention, the first high-frequency low-frequency division circuit, the second high-frequency low-frequency division circuit, the first high-frequency replacement component generation circuit, and the second high-frequency replacement component generation circuit are Even if it is not the structure shown to a form, it can implement | achieve also in the different circuit structure which performs the same calculation.
[0048]
In this embodiment, the selector 20 is switched by the outputs GL2, BL2, and RL2 of the low-pass filters 13G, 13B, and 13R. However, a filter having a different frequency characteristic is separately provided, and switching is performed by using these output signals. Also good.
[0049]
Although the present invention is applied to the horizontal space pixel shifting method, it can also be realized in the vertical space pixel shifting method by performing processing between lines.
[0050]
Further, in this embodiment, the green CCD is arranged in a half-pixel space and the high-frequency replacement component is generated from the red component and the green component. However, a similar pixel shift method is used from any combination of channels. You can go.
(Embodiment 3)
FIG. 5 is a diagram illustrating a configuration of a solid-state imaging device to which the present invention is applied. In FIG. 5, 1G is a CCD for green image, 1B is a CCD for blue image, 1R is a CCD for red image, 2G, 2B, 2R are correlated double sampling circuits (CDS circuits), 3G is a delay line, 4G, 4B, 4R is an A / D converter, 11 is 5G, 5B, 5R is a low-pass filter, 6G and 6R are first high-frequency and low-frequency dividing circuits including high-pass filters, 12 is dividers 7G and 7R, and adder 8 9G, 9B, 9R are adders, 10 is a nonlinear processing circuit, 16 is a second high-frequency low-frequency band consisting of low-pass filters 13G, 13B, 13R, and a high-pass filter 14. The dividing circuit, 20 is a selector, 17G, 17B, and 17R are adders, 21G, 21B, 21R, 22G, 22B, and 22R are delay lines. The second high-frequency permutation component generation circuit 19 includes a selector 20 and a high-pass filter. 14.
[0051]
In the following, the present invention will be described generally and conceptually, but the description overlapping with the second embodiment will be omitted.
[0052]
The present embodiment is different from the second high-frequency low-frequency dividing circuit and the second high-frequency replacement component generating circuit in the second embodiment in that the same effects as those of the second embodiment are obtained. .
[0053]
As in the second embodiment, the non-linear processing circuit 10 performs non-linear processing on the signal subjected to high-frequency replacement by the first high-frequency low-frequency dividing circuit, the first high-frequency replacement component generating circuit, and the adder. Applied. Thereafter, the signals GQ, BQ, and RQ pass through the low-pass filters 13G, 13B, and 13R, and low-frequency components GL2, BL2, and RL2 are generated. The signals GQ, BQ, and RQ are input to the selector 20 after the timing is adjusted by the delay lines 21G, 21B, and 21R. The low frequency components GL2, BL2, RL2 filtered by the other low pass filters 13G, 13B, 13R are input to the selector 20. The selector 20 compares the low frequency components GL2, BL2, and RL2, selects a signal having the lowest value, and outputs a signal of the same channel as the selected channel. Thereafter, filtering is performed by the high-pass filter 14, and a high-frequency replacement component H2 is generated.
[0054]
After the timing adjustment of the low frequency components GL2, BL2, and RL2 is performed in the delay lines 22G, 22B, and 22R, the adders 17G, 17B, and 17R add the low frequency components GL2, BL2, and RL2 to the high frequency replacement component H2. Done and get signals Go, Bo, Ro.
[0055]
In the present embodiment, the selector 20 first selects a channel that is a source of the high-frequency replacement component, and then generates a high-frequency replacement component by a high-pass filter. Compared with the second embodiment, the high-pass filter can be reduced, which leads to a reduction in circuit scale and power can be reduced.
[0056]
In the present invention, the first high-frequency low-frequency division circuit, the second high-frequency low-frequency division circuit, the first high-frequency replacement component generation circuit, and the second high-frequency replacement component generation circuit are Even if it is not the structure shown to a form, it can implement | achieve also in the different circuit structure which performs the same calculation.
[0057]
In this embodiment, the selector 20 is switched by the outputs GL2, BL2, and RL2 of the low-pass filters 13G, 13B, and 13R. However, a filter having a different frequency characteristic is separately provided, and switching is performed by using these output signals. Also good.
[0058]
Although the present invention is applied to the horizontal space pixel shifting method, it can also be realized in the vertical space pixel shifting method by performing processing between lines.
[0059]
Further, in this embodiment, the green CCD is arranged in a half-pixel space and the high-frequency replacement component is generated from the red component and the green component. However, a similar pixel shift method is used from any combination of channels. You can go.
(Embodiment 4)
FIG. 6 is a diagram illustrating a configuration of a solid-state imaging device to which the present invention is applied. 1G is CCD for green image, 1B is CCD for blue image, 1R is CCD for red image, 2G, 2B, 2R is correlated double sampling circuit (CDS circuit), 3G is delay line, 4G, 4B, 4R is A / D converter 11 is a high-frequency low-frequency dividing circuit composed of low-pass filters 5G, 5B and 5R and high-pass filters 6G and 6R, 12 is a high-frequency permutation component generating circuit composed of dividers 7G and 7R and adder 8, 9G, 9B, and 9R are adders, 10 is a nonlinear processing circuit, 23 and 25 are multipliers, 24 is a gain determination circuit, and 17G, 17B, and 17R are adders.
In the following, the present invention will be described generally and conceptually, but the description overlapping with the first, second, and third embodiments will be omitted.
The signal A / D-converted by the A / D converter is divided into the low-frequency components GL, BL, RL and the high-frequency components GH, RH by the 11 high-frequency low-frequency dividing circuits to generate high-frequency replacement components The circuit 12 generates a high frequency component shown in (Expression 1) from the divided high frequency components GH and RH. Thereafter, the multiplier 23 multiplies the coefficient m determined by the gain determination circuit, and the high frequency permutation component H1 is expressed by the following equation.
H1 = m * H = m (GH + RH) / 2 (Formula 4)
Thereafter, the adders 9G, 9B, and 9R perform addition with the low-frequency components GL, BL, and RL to obtain signals GP, BP, and RP. Thereafter, nonlinear processing is performed by the nonlinear circuit 10 to obtain signals GQ, BQ, and RQ.
The high-frequency component H generated by the high-frequency replacement component generation circuit 12 is also input to the multiplier 25 and is similarly multiplied by the coefficient n determined by the gain determination circuit to generate the high-frequency replacement component H2.
[0060]
H2 = n * H = n (GH + RH) / 2 (Formula 5)
Thereafter, addition with the signals GQ, BQ, and RQ having undergone nonlinear processing is performed by the adders 17G, 17B, and 17R.
That is, the high-frequency replacement component H1 added at the previous stage of the nonlinear processing circuit depends on the coefficient m, and the high-frequency replacement component H2 added at the subsequent stage of the nonlinear processing circuit depends on the coefficient n.
Here, FIG. 7 is a block diagram of a gain determination circuit, which includes a comparison circuit 31, a subtracter 32, a divider 33, and a subtractor 34. The comparison circuit 31 receives the low-frequency components GL, BL, and RL of each channel generated by the high-frequency low-frequency division circuit, and has the highest value among the low-frequency components GL, BL, and RL by the comparison circuit. The channel and the channel with the lowest value are selected and output as a and b, respectively.
Thereafter, the subtracting circuit 32 determines t which is the difference between a and b.
[0061]
t = a-b (Formula 6)
After division is performed in the divider, the coefficient n is output.
[0062]
n = t / 4095 (Formula 7)
The output of the divider is subtracted from 1 by the subtractor 34 and output as a coefficient m.
[0063]
m = 1- t / 4095 (Formula 8)
Next, a description will be given using the signal waveforms shown in FIG. Consider a case where the low-frequency components Gdc, Bdc, and Rdc of green, blue, and red have the same value as signals output from the CCD, and each channel has a high-frequency component with a striped pattern (amplitude w). In this case, since the low frequency components GL, BL, and RL input to the gain determination circuit have the same value, m = 1, n = 0, and (t = 0) are obtained as a result of the calculation. That is, the high-frequency replacement components are H1 = H and H2 = 0, respectively, which means that the high-frequency component H is added before the nonlinear processing, and no addition is performed after the nonlinear processing. Therefore, the signals GP, BP, and RP are signals having a high-frequency component with an amplitude w as shown in FIG. 5B, and thereafter, nonlinear processing is performed by the nonlinear processing circuit, so that the signals GQ, Get BQ, RQ.
[0064]
In the multiplier 25, since H2 = 0, no addition is performed, and the signals GQ, BQ, and RQ are output as signals Go, Bo, and Ro as they are. In this case, the same operation as the conventional example of FIG. 17 in which high-frequency replacement is performed before the nonlinear processing is performed.
Further, as shown in FIG. 9, as a signal output from the CCD, a case is considered in which the red low-frequency component is larger than the green and blue components, and in addition, only the red has a high-frequency component with a striped pattern (amplitude w). . For example, if the difference d between Rdc, Gdc, and Bdc is d = 4095 × 0.8, the gain determination circuit outputs m = 0.2 and n = 0.8 (t = 0.8). Then, since the high frequency component of H1 = 0.2H is added before the non-linear processing, the signals GP, BP, and RP are signals having a high frequency of 0.2 w as shown in FIG. Thereafter, non-linear processing is performed, and only the red color having a high low frequency component Rdc is compressed by gamma to obtain signals GQ, BQ, and RQ shown in FIG. Thereafter, the multiplier 25 multiplies the high frequency permutation component H and the coefficient n to obtain the high frequency permutation component H2 = 0.8H, and the adders 15G, 15B, and 15R add the signals GQ, BQ, and RQ. Done. As a result, signals Go, Bo, and Ro as shown in (d) are obtained.
[0065]
In other words, if the difference between the low-frequency components of each channel is small, the high-frequency replacement component to be added before the nonlinear processing is increased. Conversely, if the difference is small, the high-frequency component to be added after the nonlinear processing is increased. Increase the range replacement component. In this way, by changing the ratio of high-frequency addition in the first and second stages of nonlinear processing for the difference in low-frequency components between channels, a high-frequency component close to achromatic color is achieved, It is possible to prevent a decrease in color reproducibility and a decrease in resolution, and to reproduce a higher quality video.
As a problem solving method of the conventional example, in this embodiment, m = 0 and n = 1 are fixed, and a solution that always replaces the high frequency after the nonlinear processing can be considered. When there is an achromatic high-frequency component near the black level as shown in FIG. 8 (a), the high-frequency component does not pass through the non-linear processing circuit, so the gamma processing shown in FIG. Since expansion is not performed, resolution near the black level cannot be obtained. That is, in the present invention, as described above, when there is no difference in the low frequency component between the channels, the high frequency is added before the non-linear processing, and then the gamma processing is performed. The signal is expanded to obtain a resolution near the black level.
Further, the gain determining circuit is not limited to the above-described configuration, and the coefficients m and n may not be obtained by the relational expressions represented by (Expression 7) and (Expression 8). For example, FIG. 10 is a configuration diagram of a different gain determination circuit, which includes a comparison circuit 31, a subtracter 32, and a memory unit 35.
The comparison circuit 31 receives the low-frequency components GL, BL, and RL generated by the high-frequency low-frequency division circuit, and the channel having the highest value and the lowest value among the low-frequency components GL, BL, and RL. Are selected and output as a and b, respectively. Thereafter, the subtraction circuit 32 obtains the difference t between a and b shown in (Formula 6) and inputs the difference to the memory unit 35 as an address. Arbitrary data is input in advance to each address of the memory, and the data stored at the designated address is output. This makes it possible to realize an arbitrary conversion formula using data input in advance.
[0066]
As described above, the present invention realizes a high-frequency component close to an achromatic color by changing the ratio of high-frequency addition in the first and second stages of nonlinear processing with respect to the difference in low-frequency component between channels. It is possible to prevent a decrease in color reproducibility and resolution in a region, and to reproduce a higher quality image.
[0067]
In the present invention, the high-frequency low-frequency dividing circuit and the high-frequency replacement component generating circuit can be realized not only with the configuration shown in the present embodiment but also with different circuit configurations that perform the same calculation.
[0068]
In this embodiment, the outputs GL, BL, and RL of the low-pass filters 5G, 5B, and 5R are used as input signals to the gain determination circuit. However, filters having different frequency characteristics are separately provided and these signals are used as input signals. Also good.
[0069]
Although the present invention is applied to the horizontal space pixel shifting method, it can also be realized in the vertical space pixel shifting method by performing processing between lines.
[0070]
Further, in this embodiment, the green CCD is arranged in a half-pixel spatial manner and the high-frequency replacement component is generated from the red component and the green component. However, a similar pixel shifting method is performed from any combination of channels. May be.
(Embodiment 5)
FIG. 11 is a diagram illustrating a configuration of a solid-state imaging device to which the present invention is applied. 1G is CCD for green image, 1B is CCD for blue image, 1R is CCD for red image, 2G, 2B, 2R is correlated double sampling circuit (CDS circuit), 3G is delay line, 4G, 4B, 4R is A / D converter, 11 is 5G, 5B, 5R is a low-pass filter, 6G, 6R is a high-frequency low-frequency dividing circuit consisting of a high-pass filter, 12 is a high-frequency permutation component generator consisting of dividers 7G, 7R, and adder 8 Circuits 9G, 9B and 9R are adders, 10 is a non-linear processing circuit, 41 and 43 are coring circuits, 42 is a coring value determining circuit, and 17G, 17B and 17R are adders.
[0071]
In the following, the present invention will be described generally and conceptually, but the description overlapping with the fourth embodiment will be omitted.
[0072]
In the present embodiment, the gain determination circuit and the multiplier in the fourth embodiment are replaced with a coring value determination circuit 42 and coring circuits 41 and 43, and instead of multiplying a high frequency component by a coefficient, a coring operation is performed. The same effect is obtained.
[0073]
As shown in FIG. 12, a case is considered in which the signal output from the CCD has more red low-frequency components than green and blue components, and in addition, only red has a high-frequency component with a striped pattern (amplitude w).
[0074]
Similarly to the fifth embodiment, the low-frequency components GL, BL, and RL are divided by the high-frequency and low-frequency dividing circuit and input to the adders 9G, 9B, and 9R. Further, a high frequency component H (Equation 1) is generated from the high frequency components GH and RH divided by the high frequency replacement component generation circuit. In this case, the high frequency component H becomes a signal having an amplitude w / 2 as shown in FIG. 13A and is then input to the coring circuit 41. The constant u determined by the coring value determination circuit is input to the other side of the coring circuit 41.
[0075]
The coring circuit has characteristics as shown in FIG.
[0076]
y = 0 (-u <x <+ u), y = x-u (x> + u), y = x + u (x <-u) (Formula 9)
As shown in FIG. 13 (a), the high frequency component H input to the coring circuit 41 is hollowed out in the width 2u portion indicated by hatching, and the high frequency replacement component H1 is shown in FIG. The signal is as shown. Thereafter, H1 is added to the low frequency components GL, BL, and RL by adders 9G, 9B, and 9R to obtain signals GP, BP, and RP.
[0077]
Thereafter, nonlinear processing is performed by the nonlinear processing circuit 10 to obtain signals GQ, BQ, and RQ shown in FIG. Further, the high frequency component H is also input to the coring circuit 43, and as shown in FIG. 14, coring is performed by the constant v similarly determined by the coring determination circuit, and the high frequency replacement component H2 Get. Thereafter, the signals are added to the signals GP, BP, and RP by the adders 17G, 17B, and 17R to obtain the signals Go, Bo, and Ro.
Here, FIG. 16 is a configuration diagram of the coring value determination circuit, which includes a comparison circuit 31, a subtracter 32, and a subtracter 34. The low frequency components GL, BL, and RL generated by the high frequency and low frequency division circuit are input to the comparison circuit 31, and the channel having the highest value and the lowest value among the low frequency components GL, BL, and RL are input. Are selected and output as a and b, respectively. Thereafter, the subtraction circuit 32 obtains the difference t between a and b shown in (Formula 6), and outputs it as a constant u.
[0078]
u = t (Formula 10)
It becomes. In addition, t is also input to the subtractor 34, subtracted from 4095, and output as a constant v.
[0079]
v = 4095-t (Formula 11)
That is, as the difference t between the low frequency components of each channel is smaller, the coring amount u decreases and v increases, and the high frequency permutation component added in the previous stage of nonlinear processing becomes dominant.
From the above, when the difference between the low-frequency components of each channel is small, the high-frequency replacement component to be added is increased before the nonlinear processing, and conversely, when the difference is small, it is added after the nonlinear processing. Increase the high frequency component. In this way, for the difference in low-frequency components between channels, the high-frequency component close to achromatic color is realized by changing the ratio of high-frequency addition in the upstream and downstream of nonlinear processing by coring operation. Thus, it is possible to prevent both the color reproducibility degradation and the resolution degradation in the high frequency range, and it is possible to reproduce a higher quality image.
[0080]
In the present invention, the high-frequency low-frequency dividing circuit and the high-frequency replacement component generating circuit can be realized not only with the configuration shown in the present embodiment but also with different circuit configurations that perform the same calculation.
[0081]
In this embodiment, the outputs GL, BL, and RL of the low-pass filters 5G, 5B, and 5R are used as the input signals of the corin value determining circuit. However, filters having different frequency characteristics are separately provided, and these signals are input signals. It is also good.
[0082]
Although the present invention is applied to the horizontal space pixel shifting method, it can also be realized in the vertical space pixel shifting method by performing processing between lines.
[0083]
Further, in this embodiment, the green CCD is arranged in a half-pixel space and the high-frequency replacement component is generated from the red component and the green component. However, a similar pixel shift method is used from any combination of channels. You can go.
[0084]
Further, as shown in FIG. 10 shown in the fourth embodiment, the coring value determination circuit according to the present invention can be realized not only with the configuration shown in this embodiment but also with different circuit configurations that perform the same calculation.
[0085]
【The invention's effect】
As is clear from the above description, in a solid-state imaging device that employs the spatial pixel shifting method, when a specific color component is larger than other color components, the high-frequency component that is replaced in the previous stage of nonlinear processing However, it is possible to prevent deterioration in color reproducibility in the high frequency region caused by compression by knee processing and resolution degradation caused by adding the high frequency after the non-linear processing. It is possible to reproduce the video.
[Brief description of the drawings]
FIG. 1 is a circuit diagram of a solid-state imaging device according to Embodiment 1 of the present invention.
FIG. 2 is a diagram showing signals in the first embodiment of the present invention.
FIG. 3 is a circuit diagram of a solid-state imaging device according to Embodiment 2 of the present invention.
FIG. 4 is a diagram showing signals in the second embodiment of the present invention.
FIG. 5 is a circuit diagram of a solid-state imaging device according to Embodiment 3 of the present invention.
FIG. 6 is a circuit diagram of a solid-state imaging device according to Embodiment 4 of the present invention.
FIG. 7 is a circuit diagram of a gain determination circuit according to the fourth embodiment of the present invention.
FIG. 8 is a diagram showing signals in the fourth embodiment of the present invention.
FIG. 9 is a diagram showing signals in the fourth embodiment of the present invention.
FIG. 10 is a circuit diagram of a gain determination circuit according to the fourth embodiment of the present invention.
FIG. 11 is a circuit diagram of a solid-state imaging device according to Embodiment 5 of the present invention.
FIG. 12 is a diagram showing signals in the fifth embodiment of the present invention.
FIG. 13 is a diagram showing signals in the fifth embodiment of the present invention.
FIG. 14 is a diagram showing signals in the fifth embodiment of the present invention.
FIG. 15 is a diagram showing a coring characteristic in the fifth embodiment of the present invention.
FIG. 16 is a circuit diagram of a coring value determination circuit according to a fifth embodiment of the present invention.
FIG. 17 is a circuit diagram of a solid-state imaging device in a conventional example.
FIG. 18 is a diagram illustrating the characteristics of an optical low-pass filter.
FIG. 19 is a diagram illustrating the folding characteristics of an optical low-pass filter.
FIG. 20 is a layout diagram of CCDs in the spatial pixel shifting method.
FIG. 21 is a diagram showing signals in a conventional example.
FIG. 22 is a diagram illustrating characteristics of a low-pass filter and a high-pass filter.
FIG. 23 is a diagram showing signals in a conventional example.
FIG. 24 is a circuit diagram of a solid-state imaging device in a conventional example.
FIG. 25 is a diagram showing signals in a conventional example.
[Explanation of symbols]
CCD for 1G green image
1B CCD for blue image
1R CCD for red image
2G, 2B, 2R correlated double sampling circuit (CDS circuit)
3G delay line
4G, 4B, 4R A / D converter
11 First high-frequency low-frequency division circuit
5G, 5B, 5R low-pass filter
6G, 6R high-pass filter
12 First high-frequency replacement component generation circuit
7G, 7R divider
8 Adder
9G, 9B, 9R adder
10 Nonlinear processing circuit
18 Second high-frequency low-frequency division circuit
13G, 13B, 13R Low-pass filter
14G, 14R high-pass filter
19 Second high-frequency replacement component generation circuit
15G, 15R divider
16 Adder
17G, 17B, 17R Adder

Claims (4)

A solid-state image sensor in which at least one channel of red, blue, and green is spatially shifted by 1/2 pixel with respect to the other two-channel solid-state image sensors;
An A / D converter for analog-digital conversion of signals of each color component of red, blue, and green output from the solid-state imaging device;
A first high-frequency low-frequency dividing circuit that outputs a low-frequency component of each color and a high-frequency component of at least two colors of red, blue, and green for the signal output from the A / D converter;
A first high-frequency replacement component generation circuit for adding high-frequency components of at least two colors output from the first high-frequency low-frequency dividing circuit;
A first adder for adding the low-frequency component of each color output from the first high-frequency low-frequency dividing circuit and the output of the first high-frequency replacement component generating circuit;
A non-linear processing circuit that performs non-linear processing on the output signal of the first adder;
A second high-frequency low-frequency dividing circuit that divides the signal output from the nonlinear processing circuit into a low-frequency component of each color and a high-frequency component of at least two colors of red, blue, and green;
A second high-frequency replacement component generating circuit for adding high-frequency components of at least two colors output from the second high-frequency low-frequency dividing circuit;
A second adder for adding the low-frequency component of each color output from the second high-frequency component dividing circuit and the output of the second high-frequency replacement component generating circuit, and preceding the non-linear processing circuit And a solid-state imaging device characterized by performing high-frequency replacement in a subsequent stage.   2. The solid-state imaging device according to claim 1, wherein the first high-frequency low-frequency dividing circuit and the second high-frequency low-frequency dividing circuit according to claim 1 include a low-pass filter and a high-pass filter.   The first high-frequency permutation component generation circuit according to claim 1 includes a first divider, a second divider, and an adder, and the adder includes the first divider and the second divider. The solid-state imaging device according to claim 1, wherein the output of the divider is input to perform addition averaging.   The second high-frequency permutation component generation circuit according to claim 1 includes a first divider, a second divider, and an adder, and the adder includes the first divider and the second divider. The solid-state imaging device according to claim 1, wherein the output of the divider is input to perform addition averaging.

Images