JP4043197B2 - Single plate color camera color separation circuit - Google Patents

Description

【０１６７】
上記式（８）は、次式（８）’のように簡略化することが可能である。
【０１６８】
【数２】

【０１６９】
なお、次式（９）に示すように、水平方向相関値Ｓｈを算出する場合において、垂直方向に平均化した上で垂直相関値を求め、垂直方向相関値Ｓｖを算出する場合において、水平方向に平均化した上で垂直相関値を求めると、より精度が上がる。
【０１７０】
【数３】

【０１７１】
上記式（９）のＳｈの算出式において、Ｇ３３、Ｂ２２、Ｂ３４にかける係数を１とするとともに、Ｓｖの算出式において、Ｇ３３、Ｒ２３、Ｒ４３にかける係数を１とすることにより、各式の除数を２のｎ乗である８とし、割算回路をビットシフトのみで構成できるようにしてもよい。
【０１７２】
上記式（９）は、次式（９）’のように簡略化することが可能である。
【０１７３】
【数４】

【０１７４】
有彩色画像では、被写体色によって、モザイク配列色フィルタのＣＣＤから出力されるＲＧＢそれぞれの信号比率が異なる。そのため被写体色による妨害が発生し、ＣＣＤ全画素信号を使用した相関方向判別結果の精度は悪くなる。そこで、画面各部の色差信号の積算結果から、入力画像の色の強度を判定し、有彩色画像では、Ｇ信号のみに基づいて相関値Ｓｈ、Ｓｖを算出する。
【０１７５】
つまり、処理対象ブロックが有彩色である場合には、相関値Ｓｈ、Ｓｖは、次式（１０）に基づいて算出される。
【０１７６】
【数５】

【０１７７】
上記式（１０）は、次式（１０）’のように簡略化することが可能である。
【０１７８】
【数６】

【０１７９】
〔４−２〕 処理対象ブロックが、図１４の下側に示されているように、奇数フィールドの偶数番目のブロックである場合の相関値Ｓｈ、Ｓｖの算出方法について説明する。
【０１８０】
処理対象ブロックが無彩色である場合には、相関値Ｓｈ、Ｓｖは、次式（１１）に基づいて算出される。
【０１８１】
【数７】

【０１８２】
上記式（１１）は、次式（１１）’のように簡略化することが可能である。
【０１８３】
【数８】

【０１８４】
なお、次式（１２）に示すように、水平方向相関値Ｓｈを算出する場合において、垂直方向に平均化した上で垂直相関値を求め、垂直方向相関値Ｓｖを算出する場合において、水平方向に平均化した上で垂直相関値を求めると、より精度が上がる。
【０１８５】
【数９】

【０１８６】
上記式（１２）のＳｈの算出式において、Ｂ３３、Ｇ３２、Ｇ３４にかける係数を１とするとともに、Ｓｖの算出式において、Ｂ３３、Ｇ２３、Ｇ４３にかける係数を１とすることにより、各式の除数を２のｎ乗である８とし、割算回路をビットシフトのみで構成できるようにしてもよい。
【０１８７】
上記式（１２）は、次式（１２）’のように簡略化することが可能である。
【０１８８】
【数１０】

【０１８９】
処理対象ブロックが有彩色である場合には、相関値Ｓｈ、Ｓｖは、次式（１３）に基づいて算出される。
【０１９０】
【数１１】

【０１９１】
上記式（１３）は、次式（１３）’のように簡略化することが可能である。
【０１９２】
【数１２】

【０１９３】
なお、処理対象ブロックが有彩色である場合には、相関値Ｓｈ、Ｓｖとしては、第１の実施の形態と同様に、次式（１４）に基づいて算出してもよい。
【０１９４】
Ｓｈ＝｜Ｇ３２−Ｇ３４｜
Ｓｖ＝｜Ｇ２３−Ｇ４３｜ …（１４）
【０１９５】
なお、処理対象ブロックが偶数フィールドのブロックである場合においても、Ｇ信号のパターンは同じであるから同様の処理を行えばよい。ただし、処理対象ブロックが奇数フィールドの奇数番目のブロックである場合の演算式と処理対象ブロックが偶数フィールドの偶数番目のブロックである場合の演算式とが同じであり、処理対象ブロックが奇数フィールドの偶数番目のブロックである場合の演算式と処理対象ブロックが偶数フィールドの奇数番目のブロックである場合の演算式とが同じである。
【０１９６】
〔４−３〕 相関値Ｓｈ、Ｓｖの算出方法の変形の説明
無彩色の場合に適した演算式（例えば、上記（８）’、（１１）’）を用いて算出される相関値をＳｈ１、Ｓｖ１とし、有彩色の場合に適した演算式（例えば、上記（１０）’、（１３）’）を用いて算出される相関値をＳｈ２、Ｓｖ２とし、処理対象ブロックの色の強度に応じて、それらを加重加算することによって、相関値Ｓｈ、Ｓｖを求めるようにしてもよい。
【０１９７】
先ず、図１６に示すように、予め設定された複数の彩度検出エリアのうち、処理対象ブロックを含む彩度検出エリアｋ内の全画素信号から、画像の色の付き具合を、次式（１５）に基づいて、彩度積算値Ｃ_k として算出する。
【０１９８】
【数１３】

【０１９９】
次に、図１７に示すような、彩度積算値Ｃ_k と有彩色／無彩色判定値Ｒ_k との関係から、上記式（１５）によって算出された彩度積算値Ｃ_k に対応する有彩色／無彩色判定値Ｒ_k を求める。なお、図１７における２つの閾値Ｔｈ１、Ｔｈ２は、システム化する際に不可欠な光学ＬＰＦの特性も関係するため、実験的に求められる。
【０２００】
そして、次式（１６）で示すように、得られた有彩色／無彩色判定値Ｒ_k に基づいて、無彩色の場合に適した演算式を用いて算出される相関値Ｓｈ１、Ｓｖ１と、有彩色の場合に適した演算式を用いて算出される相関値をＳｈ２、Ｓｖ２とを加重加算する。
【０２０１】
Ｓｈ＝Ｒ_k ×Ｓｈ１＋（１−Ｒ_k ）×Ｓｈ２
Ｓｖ＝Ｒ_k ×Ｓｖ１＋（１−Ｒ_k ）×Ｓｖ２ …（１６）
【０２０２】
もちろん、有彩色／無彩色判定を行なわずに、常に有彩色として、式（１０）と式（１３）のみを使用するようにしてもよい。
【０２０３】
〔５〕第４の実施の形態の説明
図１の原色相関色分離部２４で用いられている原色相関色分離方式では、除数が０のときには、計算結果が無限大となる。そこで、原色相関色分離方式で用いられる演算式における除数が０になる場合には、カメラの自動利得制御手段の利得が小さい場合でも、図１の色差相関色分離部２５によって算出された色信号を用いるようにすることが好ましい。
【０２０４】
ここでは、”除数が０になると場合”としたが、原色相関色分離方式では除数が小さくなると誤差が増えるので、除数が一定値以下になった場合に、図１の色差相関色分離部２５によって算出された色信号を用いるようにしてもよい。
【０２０５】
具体的には、第１の実施の形態において、図１８に示すように、Ｌ０、Ｌ１、Ｌ２を入力とし、処理対象ブロック（第１の実施の形態では３×３画素の大きさの処理対象ブロック、第２の実施の形態では５×５画素の大きさの処理対象ブロック）内に画素値が０の画素が含まれているか否かを判定する判定回路４０を設ける。
【０２０６】
そして、判定回路４０は、処理対象ブロック内に画素値が０の画素が含まれていると判定した場合には、３６に色差相関色分離部２５によって算出された色信号のみを出力させるための指令を出力する。
【０２０７】
なお、第２の実施の形態においても、同様な制御を行なうことが好ましい。また、第３の実施の形態における適応相関色分離部２４も図１の原色相関色分離部２４と同様に除数が０のときには計算結果が無限大となるので、第３の実施の形態においても、同様な制御を行なうことが好ましい。
【０２０８】
【発明の効果】
以上述べたように、本発明によれば、偽色信号が発生しない色分離回路を提供することができ、画質の改善がはかれる。
【０２０９】
また、垂直または水平の高域成分をそのまま出力することが可能となり、高解像度な色分離を提供することができる。
【図面の簡単な説明】
【図１】本発明の構成を示すブロック図である。
【図２】本発明で利用されるＣＣＤを示す図である。
【図３】本発明の構成の各信号を示す説明図である。
【図４】図１中の原色相関色分離部の構成を示すブロック図である。
【図５】図１中の色差相関色分離部の構成を示すブロック図である。
【図６】演算処理回路の構成を示すブロック図である。
【図７】処理対象の画素の選択を説明する説明図である。
【図８】補間処理を示す説明図である。
【図９】相関検出を示す説明図である。
【図１０】発明の効果を説明するための説明図である。
【図１１】補間処理を説明する説明図である。
【図１２】発明の効果を説明するための説明図である。
【図１３】第２の実施の形態における、水平補間回路２６及び垂直補間回路２７の動作を説明するための説明図である。
【図１４】第２の実施の形態における、水平補間回路２９及び垂直補間回路３０の動作を説明するための説明図である。
【図１５】第３の実施の形態における、水平補間回路２６及び垂直補間回路２７の動作を説明するための説明図である。
【図１６】予め設定された複数の彩度検出エリアを示す模式図である。
【図１７】彩度積算値Ｃ_k と有彩色／無彩色判定値Ｒ_k との関係を示すグラフである。
【図１８】第４の実施の形態の構成を示すブロック図である。
【図１９】偽色信号の発生を説明する説明図である。
【符号の説明】
１ ＣＣＤ（固体撮像デバイス）
２２ 補間処理手段
２３ 相関検出手段
３２ 係数算出手段
３６ 加重加算回路[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a color separation circuit for a single-plate color video camera using a solid-state imaging device.
[0002]
[Prior art]
In a single-plate color camera using a solid-state imaging device such as a CCD, a color filter of a specific color (for example, one of three colors of R, G, and B) corresponding to each pixel of the solid-state imaging device. Is provided. And the signal from each pixel of the solid-state imaging device corresponding to a specific color is processed, color separation is performed, and R, G, and B video signals are generated.
[0003]
Conventionally, color separation in a color camera represents a process of separating and generating a luminance signal and a color signal from a CCD signal, but at present, it often represents a process of generating RGB signals at all pixel positions from a CCD signal. became. For this reason, color separation from CCD signals is often called color signal interpolation or pixel interpolation.
[0004]
FIG. 19 shows a conventional color separation process.
[0005]
For example, as shown in FIG. 19B, a case is considered where G, R, G, R... And G and R filters are alternately arranged in the horizontal direction. In such a case, for example, since a G signal cannot be obtained from a pixel to which an R filter is arranged, the G signal corresponding to this pixel is obtained by interpolating with a signal from an adjacent pixel. . That is, the G signal corresponding to the second R pixel can be obtained by calculating the average of the signal from the first G pixel and the signal from the third G pixel in FIG. As is well known, this method uses the correlation in the horizontal direction.
[0006]
However, with such a simple interpolation method, it is known that a false color signal is generated at a boundary portion (edge portion) where white and black portions are adjacent to each other on the screen. First, the generation of the false color signal will be briefly described.
[0007]
Assume that the solid-state imaging device having the filter arrangement shown in FIG. 19B is supplied with image light that suddenly changes from white to black as shown in FIG. At this time, it is assumed that the signal output value from each pixel is a value as shown in FIG. That is, the signal output values from the pixels corresponding to the white portion are all 1.0, and the signal output values from the pixels corresponding to the black portion are all 0.
[0008]
When the G signal for the pixel to which the R filter is arranged and the R signal for the pixel to which the G filter is arranged are interpolated by the above-described interpolation method, the G signal and R signal after the interpolation processing are shown in ( As shown in d) and (e). That is, it can be seen that a false color signal is generated at the boundary between white and black. The optimum G signal for the pixel to which the R filter is arranged and the optimum R signal for the pixel to which the G filter is arranged must be as shown in (f) and (g) of FIG.
[0009]
As countermeasures against the generation of such false color signals, a low-pass filter is used to make the false color signals less noticeable, or the color signals at the boundary portions (edge portions) as described above are suppressed. A method has been considered.
[0010]
However, in the method using the low-pass filter, the level of the false color signal can be lowered. However, since the area where the false color signal is generated is enlarged, it is not a complete solution. Further, in the method of suppressing the color signal at the boundary portion, even the color signal that should be originally is suppressed, and there is a possibility that the color at the boundary portion is lost.
[0011]
In order to solve this problem, in Japanese Patent No. 2931520, since the local change of the color signal is small with respect to the change of the luminance, the ratio of the G signal to the R signal or the B signal is substantially equal between adjacent pixels. And the interpolation process, that is, the primary color correlation color separation process, or the locality of the color signal, using the property that the ratio of the low frequency component to the high frequency component of the specific color component is the same in other color components. Therefore, the difference between the G signal and the R signal or the B signal is almost equal between adjacent pixels, and the difference between the low frequency component and the high frequency component of a specific color component is almost the same in other color components. Interpolation processing, that is, color-difference correlated color separation processing is performed using the property of being equal.
[0012]
[Problems to be solved by the invention]
However, in the primary color correlated color separation processing, when a large video signal having no correlation with other parts, for example, noise, is input to the input video signal, interpolation processing is performed via a division circuit, so that the noise is emphasized and the image The dot noise of R, G, and B generated in the dark part of cannot be eliminated.
[0013]
Further, in the color difference correlated color separation processing, when an object of the same color is illuminated with illumination, a false color signal due to a luminance change occurring at the boundary between the object and a shadow caused by illumination cannot be eliminated.
[0014]
The present invention, in a single-plate color video camera using a solid-state imaging device, prevents generation of false color signals in color separation processing for generating a plurality of color signal components, and has high sharpness and excellent color reproducibility. A color separation circuit is provided.
[0015]
[Means for Solving the Problems]
The color separation circuit of the single-plate color camera according to the present invention adjusts the gain of a signal from a solid-state imaging device in which a plurality of types of color filters having different spectral sensitivity characteristics corresponding to each pixel are arranged in a mosaic pattern. In a color separation circuit of a single-plate color camera that processes through an automatic gain control circuit, to generate a plurality of color signal components in an arbitrary processing target pixel based on the color signal components of the processing target pixel and surrounding pixels And two or more color separation means having different color signal component generation methods, and a synthesis means for synthesizing each color signal component generated by each color separation means based on a signal from a solid-state imaging device. It is characterized by that.
[0016]
As the color separation means, for example, a plurality of color signal components in an arbitrary pixel to be processed are used based on the correlation of the color ratios in the horizontal direction or the vertical direction based on the color signal components of the pixel to be processed and surrounding pixels. And a plurality of color signal components in an arbitrary pixel to be processed based on the color signal components of the pixel to be processed and the surrounding pixels based on the color difference between the horizontal direction and the vertical direction. Color difference correlated color separation means for generating using correlation is provided.
[0017]
In this case, as the synthesizing unit, for example, when the signal level from the solid-state imaging device is large, each color generated by the both color separating unit is set so that each color signal component generated by the primary color correlated color separating unit becomes large. When the signal components are combined and the signal level from the solid-state imaging device is low, the color signal components generated by the both color separation means are increased so that the color signal components generated by the color difference correlated color separation means become large. What is synthesized is used.
[0018]
Alternatively, as the combining means, for example, when the gain of the automatic gain adjustment circuit corresponding to the magnitude of the signal level from the solid-state imaging device is large, each color signal component generated by the color difference correlated color separation means is increased. The color signal components generated by the color separation means are combined, and when the gain of the automatic gain adjustment circuit is small, the color separation is performed so that the color signal components generated by the primary color correlation color separation means are large. What synthesize | combines each color signal component produced | generated by the means is used.
[0019]
Correlation value detection means for obtaining a correlation value in the horizontal direction and the vertical direction in the processing target pixel is provided, and the primary color correlation color separation means and the color difference correlation color separation means are color signal components suitable for cases where the correlation in the horizontal direction is strong. Horizontal direction processing means for generating color signal components and vertical direction processing means for generating color signal components suitable for a case where the correlation in the vertical direction is strong. The primary color correlation color separation means and the color difference correlation color separation means include correlation values. Processing is performed by weighted addition of each color signal component generated by the horizontal direction processing means and each color signal component generated by the vertical direction processing means in accordance with the correlation value in the horizontal direction and the vertical direction detected by the detection means. A plurality of color signal components in the target pixel may be obtained.
[0020]
As the color separation means, for example, a plurality of color signal components in an arbitrary processing target pixel are correlated with a color ratio correlation in the horizontal direction or the vertical direction based on the color signal components of the processing target pixel and surrounding pixels. The color difference in the horizontal direction or the vertical direction based on the color signal components of the processing target pixel and the surrounding pixels based on the primary color correlated color separation means generated by using the color signal components of the processing target pixel and the surrounding pixels. And a color difference correlated color separation unit that uses the correlation of the color difference.
[0021]
In this case, as the combining means, for example, when the signal level from the solid-state imaging device is large, the color signal components generated by the adaptive color separation means are increased so that each color signal component is generated. When each color signal component is synthesized and the signal level from the solid-state imaging device is small, each color signal component generated by the both color separation means is increased so that each color signal component generated by the color difference correlated color separation means becomes large. Is used.
[0022]
Alternatively, as the combining means, for example, when the gain of the automatic gain adjustment circuit corresponding to the magnitude of the signal level from the solid-state imaging device is large, each color signal component generated by the color difference correlated color separation means is increased. The color signals generated by the color separation means are combined, and the color signals generated by the adaptive correlated color separation means are increased when the gain of the automatic gain adjustment circuit is small. What synthesize | combines each color signal component produced | generated by the isolation | separation means is used.
[0023]
Correlation value detection means for obtaining a correlation value in the horizontal direction and vertical direction in the processing target pixel is provided, and the adaptive correlation color separation means and the color difference correlation color separation means are color signals suitable for cases where the horizontal correlation is strong. A horizontal direction processing means for generating a component and a vertical direction processing means for generating a color signal component suitable when the correlation in the vertical direction is strong, and the adaptive correlation color separation means and the color difference correlation color separation means include: By performing weighted addition of each color signal component generated by the horizontal direction processing means and each color signal component generated by the vertical direction processing means in accordance with the correlation values in the horizontal direction and the vertical direction detected by the correlation value detection means. A plurality of color signal components in the processing target pixel may be obtained.
[0024]
BEST MODE FOR CARRYING OUT THE INVENTION
Embodiments of the present invention will be described below with reference to the drawings.
[0025]
[1] Description of the first embodiment
FIG. 1 shows the configuration of a CCD and a color separation circuit.
[0026]
The CCD 1 is provided with an imaging unit 2 and horizontal transfer units 7 and 8. On the front surface of the CCD 1, an iris 1 a serving as a diaphragm for adjusting the amount of light incident on the imaging unit 2 is provided.
[0027]
FIG. 2 shows the configuration of the CCD 1 and its driving circuit.
The imaging unit 2 includes photodiodes 4, 4, 4,... That perform photoelectric conversion and vertical transfer CCDs 5, 5, 5,. Each photodiode 4, 4, 4 is provided with a color filter arranged in R, G, B. The vertical transfer CCD 5 is driven by a vertical drive circuit 6 outside the CCD.
[0028]
The horizontal transfer unit has a dual channel structure with the first horizontal transfer CCD 7 and the second horizontal transfer CCD 8 so that signals of two lines can be obtained simultaneously. The horizontal transfer units 7 and 8 are driven by a horizontal drive circuit 9 outside the CCD. The present invention is not limited to the dual channel structure, and may be a single channel structure.
[0029]
The signals from the horizontal transfer units 7 and 8 are processed by the CDSs 10 and 11 (correlated double sampling circuit) and the AGCs 12 and 13 (automatic gain control circuit), respectively, and then converted into digital signals by the A / D conversion circuits 14 and 15, respectively. Is converted to
[0030]
The signal D0 output from one A / D conversion circuit 14 is sent to the first selection circuit 19 in the selection circuit 18 and also sent to the first 1H delay circuit 16. The signal D1 output from the other A / D conversion circuit 15 is sent to the first and second selection circuits 19 and 20 in the selection circuit 18 and to the second 1H delay circuit 17.
[0031]
Each 1H delay circuit 16 and 17 is a memory capable of storing 1H (one horizontal period) of the input signal, and each 1H delay circuit 16 and 17 outputs a signal obtained by delaying the input signal by 1H. The Note that signal writing to and reading from the 1H delay circuit are performed in synchronization with the horizontal transfer of the CCD 1. Therefore, the timing generation circuit 71 that controls the operation of the CCD 1 and the synchronization signal generation circuit 70 operate in synchronization.
[0032]
The timing generation circuit 71 outputs a timing pulse for driving the CCD, an odd pixel / even pixel identification signal, and a clock signal. The sync signal generation circuit 70 outputs HD (horizontal sync signal), VD (vertical sync signal), and a field identification signal. The operation of other circuit parts is controlled by the output signal from the synchronization signal generating circuit 70.
[0033]
The output D2 of the first 1H delay circuit 16 is sent to the second and third selection circuits 20 and 21 in the selection circuit 18. The output D3 of the second 1H delay circuit 17 is sent to the third selection circuit 21 in the selection circuit 18.
[0034]
The selection circuit 18 selects and outputs a digital signal for 3 lines from the digital signals for 4 lines depending on whether the field is an odd field or an even field. Control of the selection circuits 19, 20, and 21 in the selection circuit 18 is performed based on a field identification signal indicating the type of field output from the synchronization signal generation circuit 70.
[0035]
That is, D1, D2, and D3 signals are selected in the odd field, and D0, D1, and D2 signals are selected in the even field. More specifically, when the field identification signal indicates an odd field (for example, an L level signal), the first selection circuit 19 selects D1, the second selection circuit 20 selects D2, and the third selection. The circuit 21 selects D3. Conversely, when the field identification signal indicates an even field (for example, H level), the first selection circuit 19 selects D0, the second selection circuit 20 selects D1, and the third selection circuit 21 selects D2. . In this way, the signals L0, L1, and L2 for three lines corresponding to the even and odd fields are output from the selection circuit 18.
[0036]
FIG. 3 shows the operation of the selection circuit 18.
[0037]
A signal (a) in FIG. 3 indicates a horizontal synchronization signal (HD) in an odd field. The signal (f) is a field identification signal, which is an odd field here, and is therefore at the L level.
[0038]
Signals (b) to (e) in FIG. 3 indicate input signals D3, D2, D1, and D0 to the selection circuit 18 in the odd field. When the field of the input image is an odd field, the signals D3, D2, and D1 are output from the selection circuit 18 as signals L2, L1, and L0.
[0039]
In the odd field, when the fourth line is input as D0, D3 is the first line, D2 is the second line, D1 is the third line, and D0 is the fourth line. The signals of the second line and the third line are selected. Next, D3 is the third line, D2 is the fourth line, D1 is the fifth line and D0 is the sixth line, and the signals of the third, fourth and fifth lines are selected. The
[0040]
A signal (g) in FIG. 3 indicates a horizontal synchronization signal (HD) in an even field. The signal (l) is a field identification signal, and here, since it is an even field, it is at the H level.
[0041]
Signals (h) to (k) in FIG. 3 indicate the input signals D3, D2, D1, and D0 to the selection circuit 18 in the even field. When the field of the input image is an even field, the signals D2, D1, and D0 are output from the selection circuit 18 as signals L2, L1, and L0.
[0042]
In the even field, when the fourth line is input as D0, D3 is the first line, D2 is the second line, D1 is the third line, and D0 is the fourth line. The signals of the third line and the fourth line are selected. Next, D3 is the 3rd line, D2 is the 4th line, D1 is the 5th line and D0 is the 6th line, and the signals of the 4th, 5th and 6th lines are selected. The
[0043]
The outputs L0, L1, and L2 of the selection circuit 18 are supplied to the interpolation processing unit 22 and the correlation detection unit 23, respectively.
[0044]
The interpolation processing means 22 includes a primary color correlation color separation unit 24 that performs an interpolation process using the correlation of the color ratios in the horizontal direction or the vertical direction, and an interpolation using the correlation of the color differences in the horizontal direction or the vertical direction A color difference correlated color separation unit 25 for performing processing is provided.
[0045]
Further, the correlation detection means 23 is provided with a horizontal direction correlation detection means 23a and a vertical direction correlation detection means 23b. The horizontal direction correlation detection means 23a outputs the horizontal direction correlation value Sh, and the vertical direction correlation detection means 23b outputs the vertical direction correlation value Sv. These two outputs Sh and Sv are given to the coefficient calculation means 32. The coefficient calculation means 32 calculates a horizontal coefficient Kh and a vertical coefficient Kv based on both correlation values Sh and Sv. Details of the correlation value and coefficient calculation method will be described later.
[0046]
FIG. 4 shows the configuration of the primary color correlated color separation unit 24.
The primary color correlated color separation unit 24 includes an intra-field horizontal interpolation circuit 26, an intra-field vertical interpolation circuit 27, and weighted addition means 28.
[0047]
The intra-field horizontal interpolation circuit 26 performs interpolation processing suitable for the case where the correlation in the horizontal direction is strong, based on the output signals from the CCD 1 for three lines, and outputs the color signals Gh, Rh, and Bh. The intra-field vertical interpolation circuit 27 performs interpolation processing suitable for the case where the correlation in the vertical direction is strong, based on the output signals from the CCD 1 for three lines, and outputs the color signals Gv, Rv, Bv. A specific interpolation method by the interpolation circuits 26 and 27 in the primary color correlation color separation unit 24 will be described later.
[0048]
The case where the correlation in the horizontal direction is strong refers to a case where the correlation in the horizontal direction is strong and there is almost no correlation in the vertical direction as in a fine horizontal stripe image. The case where the correlation in the vertical direction is strong refers to a case where the correlation in the vertical direction is strong and there is almost no correlation in the horizontal direction as in a fine vertical stripe image.
[0049]
The color signals Gh, Rh, Bh and Gv, Rv, Bv output from the interpolation circuits 26, 27 are supplied to the weighted addition means 28. The weighted addition means 28 is also supplied with a horizontal coefficient Kh and a vertical coefficient Kv output from the coefficient calculation means 32. The weighted addition means 28 multiplies the color signals Gh, Rh, Bh by the coefficient Kh, and after multiplying the color signals Gv, Rv, Bv by the coefficient Kv, the same color signals (Gh and Gv, Rh and Rv, Bh). And Bv) are added to output the final color signals G1, R1, and B1.
[0050]
FIG. 5 shows the configuration of the color difference correlated color separation unit 25.
The color difference correlated color separation unit 25 includes an intra-field horizontal interpolation circuit 29, an intra-field vertical interpolation circuit 30, and weighted addition means 31. A specific interpolation method by the interpolation circuits 29 and 30 in the color difference correlated color separation unit 25 will be described later.
[0051]
The color difference correlated color separation unit 25 and the above-described primary color correlated color separation unit 24 differ only in the interpolation processing method in the in-field horizontal direction interpolation circuit 29 and the in-field vertical interpolation circuit 30. Color signals G2, R2, and B2 are finally output from the weighted addition means 31 of the color difference correlated color separation unit 25.
[0052]
The primary color correlated color separation method and the color difference correlated color separation method will be described. First, the primary color correlated color separation method will be described. The in-field horizontal interpolation circuit 26 and the in-field vertical interpolation circuit 27 are basically composed of digital processing circuits composed of blocks as shown in FIG. Reference numerals 33 and 34 in FIG. 6 denote delay means having a delay time equal to the time required for transmission of one pixel, and reference numeral 35 denotes an arithmetic means. In this example, each of the interpolation circuits 26 and 27 generates three color signals corresponding to the position of one central pixel based on the signals of nine pixels in a 3 × 3 pixel block.
[0053]
That is, the signals L 0, L 1, L 2 output from the selection circuits 19, 20, 21 are input to the delay means 33, 34, so that signals for nine pixels are simultaneously supplied to the calculation means 35. Then, interpolation processing is performed by calculation in the calculation means 35.
[0054]
FIG. 7 shows the relationship between the array of pixels on the CCD 1 and the selected pixels.
As described above, in the odd field, first, the D1, D2, and D3 line signals are selected. Therefore, the odd-numbered 3 × 3 pixel block is as shown in FIG. The evenly processed pixels are as shown in FIG.
[0055]
On the other hand, in the even field, first, the line signals D0, D1, and D2 are selected, so that the odd-numbered block is processed as shown in FIG. As shown in FIG.
[0056]
FIG. 8 shows an arithmetic expression used by the horizontal interpolation circuit 26 and the vertical interpolation circuit 27 when the processing target block is an odd-numbered block or an even-numbered block in an odd field.
[0057]
When the processing target block is an odd-numbered block in the odd field, the horizontal interpolation circuit 26 calculates Gh, Bh, and Rh for the central pixel (processing target pixel) G22 as follows.
[0058]
That is, the signal G22 of the processing target pixel G22 is used as it is as Gh for the processing target pixel G22. Bh for the processing target pixel G22 is obtained by calculating the average of the signals B21 and B23 of the two left and right pixels B21 and B23 sandwiching the processing target pixel G22 (using a horizontal correlation). Rh for the processing target pixel G22 is a little complicated. That is, G12 which is the G signal at the position of the pixel R12 is calculated as an average value of G11 and G13 (that is, a horizontal correlation is used). And Rh is calculated | required from the ratio of G12 calculated | required by calculation, and G22 actually obtained, and R12.
[0059]
This is because the local change of the color signal is small with respect to the change of the luminance, so that the ratio of the G signal to the R signal or the B signal is considered to be approximately equal between adjacent pixels. That is, R22 (= Rh) is obtained on the assumption that G12: R12 = G22: R22.
[0060]
When the processing target block is an odd-numbered block in the odd field, the vertical interpolation circuit 27 calculates Gv, Bv, and Rv for the central pixel (processing target pixel) G22 as follows.
[0061]
That is, as Gv for the processing target pixel G22, the signal G22 for the processing target pixel G22 is used as it is. The Rv for the processing target pixel G22 is obtained by calculating the average of the signals R12 and R32 of the upper and lower pixels R12 and R32 sandwiching the processing target pixel G22 (using the vertical correlation). Bv for the processing target pixel G22 is obtained in the same manner as Rh described above. That is, G21, which is a G signal in the pixel B21, is calculated as an average value of G11 and G31 (a vertical correlation is used). Then, Bv is calculated from the ratio of G21 and G22 obtained by calculation and B21.
[0062]
When the processing target block is an even-numbered block in the odd field, the horizontal interpolation circuit 26 calculates Gh, Bh, and Rh for the central pixel (processing target pixel) B22 as follows.
[0063]
That is, the signal B22 of the processing target pixel B22 is used as it is as Bh for the processing target pixel B22. Gh for the processing target pixel B22 is obtained by calculating an average value of G21 and G23. For Rh for the processing target pixel B22, first, R12 is obtained by obtaining the average of R11 and R13, and Rh is obtained from the ratio of Gh (= G22) and G12 and R12.
[0064]
When the processing target block is an even-numbered block in an odd field, the vertical interpolation circuit 27 calculates Gv, Bv, and Rv for the central pixel (processing target pixel) B22 as follows.
[0065]
That is, for Bv for the processing target pixel B22, the signal B22 of the processing target pixel B22 is used as it is. Gv for the processing target pixel B22 is obtained by calculating an average value of G12 and G32. As for Rv for the processing target pixel B22, first, R21 is obtained by obtaining the average of R11 and R31, and Rv is obtained from the ratio of Gh (= G22) and G21 and R21.
[0066]
Even when the processing target block is an even field block, Rh, Gh, Bh, Rv, Gv, and Bv signals can be obtained by performing the same processing as that shown in FIG. That is, when the processing target block is an odd-numbered block in the even field, the pixel arrangement is the even-numbered block in the odd field, as can be seen from (c) and (d) of FIG. Since R and B are interchanged in the pixel array in the case of a block, R and B may be exchanged in an arithmetic expression used when the processing target block is an even-numbered block in an odd field.
[0067]
If the processing target block is an even-numbered block in an even-numbered field, the pixel arrangement is an odd-numbered block in an odd-numbered field, as can be seen from FIGS. 7B and 7E. In this case, R and B are interchanged in the pixel array in the case of the above. Therefore, R and B may be exchanged in an arithmetic expression used when the processing target block is an odd-numbered block in an odd field.
[0068]
The horizontal direction correlation detection circuit 23a and the vertical direction correlation detection circuit 23b are basically composed of a digital processing circuit including blocks as shown in FIG. Reference numerals 33 and 34 in FIG. 6 denote delay means having a delay time equal to the time required for transmission of one pixel, and reference numeral 35 denotes an arithmetic means.
[0069]
That is, the signals L 0, L 1, L 2 output from the selection circuits 19, 20, 21 are input to the delay means 33, 34, so that signals for nine pixels are simultaneously supplied to the calculation means 35. Then, correlation value detection processing is performed by calculation in the calculation means 35.
[0070]
FIG. 9 shows arithmetic expressions used by the horizontal direction correlation detection circuit 23a and the vertical direction correlation detection circuit 23b.
[0071]
Each of the correlation value detection circuits 23a and 23b calculates correlation values Sh and Sv by using the G signal that is contained most in the 3 × 3 pixel block. When the processing target block is an odd-numbered block in the odd field, the vertical correlation value Sv is obtained by calculating the absolute value of the difference between G12 and G32. Since G12 and G32 do not actually exist, G12 is calculated from G11 and G13, and G32 is calculated from G31 and G33.
[0072]
The horizontal correlation value Sh when the block to be processed is an odd-numbered block in the odd-numbered field is obtained by calculating the absolute value of the difference between G21 and G23. Since G21 and G23 do not actually exist, G21 is calculated from G11 and G31, and G23 is calculated from G13 and G33.
[0073]
When the processing target block is an even-numbered block in an odd field, the vertical correlation value Sv is obtained by calculating the absolute value of the difference between G12 and G32. Further, the horizontal correlation value Sh is obtained by calculating the absolute value of the difference between G21 and G23.
[0074]
Even when the block to be processed is an even field block, the G signal pattern is the same, and thus the same processing may be performed. However, the arithmetic expression when the processing target block is an odd-numbered block in the odd field is the same as the arithmetic expression when the processing target block is an even-numbered block in the even field, and the processing target block is in the odd field. The arithmetic expression for the even-numbered block is the same as the arithmetic expression for when the processing target block is the odd-numbered block in the even field.
[0075]
Note that the smaller the horizontal correlation value Sh, the stronger the horizontal correlation. Similarly, the smaller the correlation value Sv in the vertical direction, the stronger the correlation in the vertical direction.
[0076]
The coefficient calculation means 32 calculates horizontal and vertical coefficients Kh and Kv from the correlation values Sh and Sv calculated by the correlation value detection circuits 23a and 23b. Each coefficient Kh, Kv is calculated | required by following Formula (1). Note that a relationship of Kh + Kv = 1 is established between Kh and Kv.
[0077]
Kh = Sv / (Sh + Sv)
Kv = Sh / (Sh + Sv) (1)
[0078]
Therefore, the horizontal coefficient Kh becomes large when the horizontal correlation is stronger than the vertical correlation (when Sh is smaller than Sv). The vertical coefficient Kv is a large value when the vertical correlation is stronger than the horizontal correlation (when Sv is smaller than Sh).
[0079]
As described above, R, G, and B signal components corresponding to one pixel in the CCD are calculated by using the outputs from the surrounding pixels (9 in total). The calculation is performed using two methods, a processing method suitable for a case where the correlation between the pixels is strong and a processing method suitable for a case where the correlation in the vertical direction is strong. In addition, the calculation results by the above two methods are weighted and added.
[0080]
Next, the operation of the primary color correlated color separation unit 24 will be described with a specific example.
[0081]
Consider a case where white incident light as shown in FIG. 10A is given to a processing target block. In the example of FIG. 10A, an example of an edge portion where there is no color component and only the luminance level is changed is shown. Therefore, as a correct result, the R, G, and B levels of the processing target pixel (center pixel) must all be the same.
[0082]
FIG. 10B schematically shows the output level from each pixel. Two types are shown because the pattern of the color is different between the odd-numbered processing and the even-numbered processing as described above. The Rh, Rv, Gh, Gv, Bh, Bv and the weighting coefficients Kh, Kv of the processing target pixel (center pixel) calculated according to FIGS. 8 and 9 and the R of the processing target pixel (center pixel) taking the weighting coefficient into account, G and B (Ro, Go, Bo) are as shown in (c) and (d). The component levels of R, G, B at the center position are all equal to 0.2.
[0083]
Next, the color difference correlated color separation method will be described.
[0084]
The difference between the color difference correlated color separation unit 25 and the above-described primary color correlated color separation unit 24 is as follows. That is, the interpolation processing in the primary color correlated color separation unit 24 has a property that the ratio of the G signal to the R signal or the B signal is substantially equal between adjacent pixels because the local change in the color signal is small with respect to the change in luminance. The ratio of the low-frequency component to the high-frequency component of a specific color component is utilized in the other color components. On the other hand, in the interpolation processing in the color difference correlated color separation unit 25, since the local change of the color signal is small, the difference between the G signal and the R signal or the B signal is substantially equal between adjacent pixels, The difference is that the difference between the low frequency component and the high frequency component of the color component uses the property that the other color components are substantially equal.
[0085]
FIG. 11 shows arithmetic expressions used by the horizontal interpolation circuit 29 and the vertical interpolation circuit 30 of the chrominance correlated color separation unit 25 when the processing target block is an odd-numbered block or an even-numbered block in an odd field. .
[0086]
When the processing target block is an odd-numbered block in the odd field, the horizontal interpolation circuit 29 calculates Gh, Bh, and Rh for the central pixel (processing target pixel) G22 as follows.
[0087]
That is, the signal G22 of the processing target pixel G22 is used as it is as Gh for the processing target pixel G22. Bh for the processing target pixel G22 is obtained by calculating the average of the signals B21 and B23 of the two left and right pixels B21 and B23 sandwiching the processing target pixel G22. Regarding Rh for the processing target pixel G22, a property that the difference between the G signal and the R signal or the B signal is substantially equal between adjacent pixels is used. That is, the following relational expression (2) is established.
[0088]
R22-G22 = R12-G12 = R32-G32
= {(R12-G12) / 2} + {(R32-G32) / 2} (2)
[0089]
Accordingly, Rh (= R22) for the processing target pixel G22 is obtained by the following equation (3).
[0090]
Rh = {(R12−G12) / 2} + {(R32−G32) / 2} + G22
= {(R12 + R32) / 2}-{(G12 + G32) / 2} + G22
= {(R12 + R32) / 2}-{(G11 + G13 + G31 + G33) / 4} + G22 (3)
[0091]
Even when the processing target block is an even-numbered block in an odd field, the horizontal interpolation circuit 29 obtains Gh, Rh, and Bh by performing the same calculation.
[0092]
The method of obtaining Gv, Rv, and Bv in the vertical interpolation circuit 30 is basically the same as the interpolation processing of the horizontal interpolation circuit 29, and thus the description thereof is omitted.
[0093]
As described above, multiplication and division occupy the arithmetic processing in the arithmetic expression in FIG. 8. However, in the arithmetic expression in FIG. 11, multiplication and division are replaced with addition and subtraction, and the circuit configuration is particularly complicated. Since division is not used, the configuration of the arithmetic processing circuit can be greatly simplified.
[0094]
Further, according to the color difference correlated color separation method interpolation processing, a high-quality image with few false color signals can be obtained for an image close to monotone.
[0095]
The color signals G1, R1, B1, G2, R2, and B2 output from the color separation units 24 and 25 described above are supplied to a weighted addition circuit 36 serving as a combining unit. The weighted addition circuit 36 outputs final color signals G0, R0, B0.
[0096]
The color signals G0, R0, B0 output from the weighted addition circuit 36 are supplied to the AGC control circuit 37. The AGC control circuit 37 adjusts the opening degree of the iris 1a or the gains of the AGCs 12 and 13 from the color signals G0, R0, and B0 in order to control the luminance level of the captured image. Specifically, the adjustment is basically performed by the iris 1a, and when the opening of the iris 1a reaches the maximum, an instruction is given to control the gain of the AGC to be controlled further. At this time, the gain instruction to the AGCs 12 and 13 is also supplied to the weighted addition circuit 36.
[0097]
In the weighted addition circuit 36, the color signals G 1, R 1, B 1 from the primary color correlation color separation unit 24 and the color signals G 2, R 2, B 2 from the color difference correlation color separation unit 25 according to the magnitude of the gain of the AGC 12, 13 Operates to add weights. Specifically, if the gains of the AGCs 12 and 13 are small, the color signals G1, R1, and B1 are added so as to be larger than the color signals G2, R2, and B2. If the gains are large, the gains are larger than those of the color signals G1, R1, and B1. The color signals G2, R2, and B2 are added so as to increase, and the color signals G0, R0, and B0 are output.
[0098]
In FIG. 10, it is assumed that the correlation direction is erroneously detected due to the influence of noise or the like. Then, the values of Go, Ro, and Bo become the values of Gh, Rh, and Bh. As a result, an error occurs with respect to the value (0.2) that should be taken. The error is 0.133 for R in the odd-numbered (upper diagram in FIG. 10B), 0.4 in B, and is 0. 0 for G in the even-numbered (lower diagram in FIG. 10B). 4, R is 1.6, and the sum of these errors is 2.533.
[0099]
On the other hand, when interpolation is performed by the color difference correlated color separation unit 25, as shown in FIG. 12, even if the correlation direction is detected incorrectly, the total error is only 2.2. Although this is an example, since division is used in the primary color correlation, when the divisor (denominator) approaches 0, the accuracy deteriorates and the error increases, resulting in an increase in noise. Particularly in the dark, there is a high probability that the divisor approaches 0.
[0100]
In this embodiment, if the gains of the AGCs 12 and 13 are small, the color signals G1, R1, and B1 are added so as to be larger than the color signals G2, R2, and B2, and if the gains are large, the color signals G1, R1, and B1 are added. Since the color signals G2, R2, and B2 are added so as to be larger, noise enhancement and false color included in the image caused by the primary color correlated color separation processing can be suppressed, and the resolution can be improved. .
[0101]
In the above embodiment, the primary color correlated color separation method and the color difference correlated color separation method are weighted and added in accordance with the gains of the AGCs 12 and 13, but the present invention is not limited to this, and the image contrast and Weighted addition may be performed according to the signal level.
[0102]
Further, in the above-described primary color correlated color separation method and color difference correlated color separation method, calculation is performed in units of 3 × 3 pixels. However, the present invention is not limited to this, and for example, if calculation is performed in units of 5 × 5 pixels, further false The color is suppressed and the resolution can be improved. In the above embodiment, the case where the present invention is applied to an interlace signal has been described. However, the present invention can also be applied to a progressive signal.
[0103]
[2] Description of the second embodiment
[0104]
In the second embodiment, the operations of the intra-field horizontal interpolation circuit 26 and the intra-field vertical interpolation circuit 27 in the primary color correlation color separation unit 24 are different from those of the first embodiment. The operations of the intra-field horizontal interpolation circuit 29 and the intra-field vertical interpolation circuit 30 in the color difference correlated color separation unit 25 are different from those of the first embodiment.
[0105]
In the second embodiment, each of the interpolation circuits 26, 27, 29, and 30 corresponds to one central pixel (processing target pixel) based on signals of 25 pixels in a 5 × 5 pixel block. Three color signals are created.
[0106]
[2-1] Description of Interpolation Circuits 26 and 27 in the Primary Color Correlated Color Separation Unit 24
First, operations of the intra-field horizontal interpolation circuit 26 and the intra-field vertical interpolation circuit 27 in the primary color correlated color separation unit 24 will be described. These interpolation circuits 26 and 27 calculate the color signals R, G, and B of the processing target pixel by the primary color correlated color separation method.
[0107]
FIG. 13 shows arithmetic expressions used by the horizontal interpolation circuit 26 and the vertical interpolation circuit 27 when the processing target block is an odd-numbered block or an even-numbered block in an odd field.
[0108]
Here, the primary color correlated color separation method will be described by taking the operation of the vertical interpolation circuit 27 when the processing target block is an odd-numbered block in an odd-numbered field as an example.
[0109]
In the primary color correlated color separation method, a strong correlation direction of an image is detected in a local region, and two units on the same line are first filtered by line-by-line filtering only in the strong correlation direction (in this case, the vertical direction or the horizontal direction). A low-frequency color signal is calculated.
[0110]
In the vertical interpolation circuit 27, it is assumed that the processing target pixel G33 of the upper 5 × 5 pixel block in FIG. 13 has a strong correlation in the vertical direction, and R and G are calculated from the vertical line including the processing target pixel G33. Low frequency signal R _VLPF3 , G _VLPF3 Is calculated.
[0111]
In the slowly changing signal, the ratio of the color signal of each pixel is substantially equal to the ratio of the low-frequency signal. Therefore, when the correlation in the vertical direction is strong, Rv ( = R33) / G33 = R _VLPF3 / G _VLPF3 Holds. Therefore, the two low-frequency color signals R on the vertical line including the processing target pixel G33. _VLPF3 , G _VLPF3 , And the color signal G33 of the pixel to be processed, another color signal Rv existing on the vertical line can be obtained.
[0112]
As Gv for the processing target pixel G33, the signal G33 of the processing target pixel G33 is used as it is.
[0113]
Only the color signals (R and G in this example) existing on the target line (vertical line in this example) can be separated from one vertical and horizontal line. Therefore, the color signal that does not exist on the line is obtained from the signal of the adjacent line and the signal of the pixel to be processed, that is, the color ratio, that is, the color ratio in the direction where the image correlation is low (here, the horizontal direction). The color signal Bv that does not exist on the target line is interpolated using the characteristic that “the change of the signal is small”, that is, the characteristic of Bv (= B33) / G33 = B32 / G32 = B34 / G34.
[0114]
Here, since the horizontal direction is a direction in which the correlation is low, as shown in the following equation (4), the low correlation is compensated by averaging the ratios of the color signals of both horizontal adjacent pixels.
[0115]
Bv = [{(B32 / G32) + (B34 / G34)} / 2] × G33 (4)
[0116]
Instead of the formula (4), a formula obtained by simplifying the formula (4), Bv = (B32 / G32) × G33, may be used. In FIG. 13, an expanded version of this simplified expression is shown as Bv.
[0117]
It is desirable that the filter characteristics for obtaining the low frequency components of the signal used here have frequency characteristics as close as possible. In order to match the filter characteristics of signals having different sampling phases, it is necessary to considerably widen the tap length of the filter, which is not practical because it involves a significant increase in circuit scale.
[0118]
Therefore, here, focusing on the low-frequency signal component in the direction of strong correlation, focusing on the low-frequency part up to 1/2 of the Nyquist frequency, the frequency characteristics of the low-frequency part are close. In order to reduce the number of filter taps, the following equation (5) is used.
[0119]
G _VLPF3 = (G13 + 6 × G33 + G53) / 8
R _VLPF3 = (R23 + R43) / 2 (5)
[0120]
The operation of the vertical interpolation circuit 27 in the case where the processing target block is an odd-numbered block in the odd field has been described. However, when the processing target block is an even-numbered block in the odd field, the same operation is performed. Rv, Gv, and Bv of the target pixel can be obtained.
[0121]
Further, the horizontal interpolation circuit 26 can obtain Rh, Gh, and Bh of the processing target pixel by assuming that the correlation in the horizontal direction is strong.
[0122]
Even when the processing target block is an even field, color separation can be performed by the same method.
[0123]
In the primary color correlated color separation method, division is performed to obtain the ratio of color signals. Therefore, when the divisor is close to 0, the error becomes large. When the divisor is 0, computation is impossible. Therefore, when the divisor is 0, exception processing is performed such that the constant is forcibly set to 1. For this reason, when the screen is dark or when there is noise in a dark portion of the image, the noise is enhanced and dot noise is generated.
[0124]
That is, the primary color correlated color separation method exhibits good characteristics when the state of the input video is good, that is, when there is sufficient input light.
[0125]
[2-2] Description of Interpolation Circuits 29 and 30 in the Color Difference Correlated Color Separation Unit 25
[0126]
The operation of the intra-field horizontal interpolation circuit 29 and the intra-field vertical interpolation circuit 30 in the color difference correlated color separation unit 25 will be described. These interpolation circuits 29 and 30 calculate the color signals R, G, and B of the pixel to be processed by the color difference correlated color separation method.
[0127]
FIG. 14 shows arithmetic expressions used by the horizontal interpolation circuit 29 and the vertical interpolation circuit 30 when the processing target block is an odd-numbered block or an even-numbered block in an odd field.
[0128]
Here, the color difference correlated color separation method will be described by taking the operation of the vertical interpolation circuit 30 when the processing target block is an odd-numbered block in an odd-numbered field as an example.
[0129]
In the color difference correlation color separation method, by utilizing the color difference correlation that “the change in the color difference signal in the local region is small”, the direction in which the image correlation is strong in the local region is detected, and the direction in which the correlation is strong (here, First, two low-frequency color signals on the same line are calculated by filtering in units of lines only in the vertical direction or the horizontal direction.
[0130]
In the vertical interpolation circuit 30, it is assumed that the correlation in the vertical direction is strong in the processing target pixel G33 of the upper 5 × 5 pixel block in FIG. 14, and from the vertical line including the processing target pixel G33, R and G Low frequency signal R _VLPF3 , G _VLPF3 Is calculated.
[0131]
In the slowly changing signal, the difference between the color signals of the pixels and the difference between the low-frequency signals are almost equal. Therefore, if the correlation in the vertical direction is strong, Rv ( = R33) -G33 = R _VLPF3 -G _VLPF3 Holds. Therefore, the two low-frequency color signals R on the vertical line including the processing target pixel G33. _VLPF3 , G _VLPF3 And another color signal Rv existing on the vertical line can be obtained from the color signal G33 of the pixel to be processed.
[0132]
As Gv for the processing target pixel G33, the signal G33 of the processing target pixel G33 is used as it is.
[0133]
Only the color signals (R and G in this example) existing on the target line (vertical line in this example) can be separated from one vertical and horizontal line. Therefore, the color signal that does not exist on the line is correlated with the color difference from the signal of the adjacent line and the signal of the processing target pixel, that is, the color difference even in the direction where the image correlation is low (here, the horizontal direction). The color signal Bv that does not exist on the target line is interpolated using the characteristic that the change of “is small”, that is, the characteristic of Bv (= B33) −G33 = B32−G32 = B34−G34.
[0134]
Here, since the horizontal direction is a direction in which the correlation is low, as shown in the following equation (6), the low correlation is compensated by averaging the color difference signals of both horizontal adjacent pixels.
[0135]
Bv = [{(B32−G32) + (B34−G34)} / 2] + G33 (6)
[0136]
Instead of the formula (6), a formula obtained by simplifying the formula (6), Bv = (B32−G32) + G33, may be used. In FIG. 14, an expanded version of this simplified expression is shown as Bv.
[0137]
The operation of the vertical interpolation circuit 30 in the case where the processing target block is an odd-numbered block in the odd field has been described. However, when the processing target block is an even-numbered block in the odd field, the same operation is performed. Rv, Gv, and Bv of the target pixel can be obtained.
[0138]
Further, the horizontal interpolation circuit 29 can obtain Rh, Gh, and Bh of the processing target pixel by assuming that the correlation in the horizontal direction is strong.
[0139]
Even when the processing target block is an even field, color separation can be performed by the same method.
[0140]
In the color difference correlated color separation method, a color signal can be obtained only by addition and subtraction without performing division. For this reason, there is a feature that an error hardly occurs and that it is resistant to noise.
[0141]
Therefore, in a strong noise situation, that is, as the gain of the automatic gain control means of the camera increases, the color signal calculated by the color difference correlated color separation method is used more frequently, and when the gain of the automatic gain control means of the camera is small Many color signals calculated by the primary color correlated color separation method are used.
[0142]
[3] Description of the third embodiment
[0143]
In the third embodiment, the operations of the intra-field horizontal interpolation circuit 26 and the intra-field vertical interpolation circuit 27 in the primary color correlation color separation unit 24 are different from those of the first embodiment. The operations of the intra-field horizontal interpolation circuit 29 and the intra-field vertical interpolation circuit 30 in the color difference correlated color separation unit 25 are different from those of the first embodiment.
[0144]
In the third embodiment, each of the interpolation circuits 26, 27, 29, and 30 corresponds to one central pixel (processing target pixel) based on signals of 25 pixels in a 5 × 5 pixel block. Three color signals are created.
[0145]
In the third embodiment, the intra-field horizontal interpolation circuit 29 and the intra-field vertical interpolation circuit 30 in the chrominance correlated color separation unit 25 in FIG. 1 are similar to the second embodiment in the chrominance correlated color separation method. Thus, R, G, and B of the processing target pixel are obtained.
[0146]
In the third embodiment, instead of the primary color correlation color separation unit 24 in FIG. 1, a color separation unit (hereinafter referred to as an adaptation) for obtaining R, G, and B of the processing target pixel by an adaptive correlation color separation method. Correlated color separation unit) is used. The intra-field horizontal interpolation circuit 26 and the intra-field vertical interpolation circuit 27 in the adaptive correlation color separation unit 24 obtain R, G, and B of the processing target pixel by the adaptive correlation color separation method.
[0147]
The operation of the intra-field horizontal interpolation circuit 26 and the intra-field vertical interpolation circuit 27 in the adaptive correlation color separation unit 24 will be described. These interpolation circuits 26 and 27 calculate the color signals R, G, and B of the processing target pixel by the adaptive correlation color separation method.
[0148]
FIG. 15 shows an arithmetic expression used by the horizontal interpolation circuit 26 and the vertical interpolation circuit 27 in the adaptive correlation color separation unit 24 when the processing target block is an odd-numbered block or an even-numbered block in an odd field. Yes.
[0149]
Here, the adaptive correlation color separation method will be described by taking the operation of the vertical interpolation circuit 27 as an example when the processing target block is an odd-numbered block in an odd-numbered field.
[0150]
In the adaptive correlation color separation method, a strong correlation direction of an image is detected in a local region, and filtering is performed in units of lines only in a strong correlation direction (in this case, a vertical direction or a horizontal direction in this case). A low-frequency color signal is calculated.
[0151]
In the vertical interpolation circuit 27, assuming that the correlation in the vertical direction is strong in the processing target pixel G33 of the upper 5 × 5 pixel block in FIG. 15, from the vertical line including the processing target pixel G33, R and G Low frequency signal R _VLPF3 , G _VLPF3 Is calculated.
[0152]
In the slowly changing signal, the difference between the color signals of the pixels and the difference between the low-frequency signals are almost equal. Therefore, if the correlation in the vertical direction is strong, Rv ( = R33) -G33 = R _VLPF3 -G _VLPF3 Holds. Therefore, the two low-frequency color signals R on the vertical line including the processing target pixel G33. _VLPF3 , G _VLPF3 And another color signal Rv existing on the vertical line can be obtained from the color signal G33 of the pixel to be processed. Rv is obtained in the same manner as the color difference correlated color separation method.
[0153]
As Gv for the processing target pixel G33, the signal G33 of the processing target pixel G33 is used as it is.
[0154]
Only the color signals (R and G in this example) existing on the target line (vertical line in this example) can be separated from one vertical and horizontal line. Therefore, the color signal that does not exist on the line is obtained from the signal of the adjacent line and the signal of the pixel to be processed, that is, the color ratio, that is, the color ratio in the direction where the image correlation is low (here, the horizontal direction). The color signal Bv that does not exist on the target line is interpolated using the characteristic that “the change of the signal is small”, that is, the characteristic of Bv (= B33) / G33 = B32 / G32 = B34 / G34. The method for obtaining Bv is the same as in the primary color correlation color separation method.
[0155]
Here, since the horizontal direction is a direction in which the correlation is low, the low correlation is compensated by averaging the color difference signals of both horizontal adjacent pixels as shown in the following equation (7).
[0156]
Bv = [{(B32 / G32) + (B34 / G34)} / 2] × G33 (7)
[0157]
Instead of the formula (7), a formula obtained by simplifying the formula (7), Bv = (B32 / G32) × G33, may be used. In FIG. 15, an expanded version of this simplified expression is shown as Bv.
[0158]
The operation of the vertical interpolation circuit 27 in the case where the processing target block is an odd-numbered block in the odd field has been described. However, when the processing target block is an even-numbered block in the odd field, the same operation is performed. Rv, Gv, and Bv of the target pixel can be obtained.
[0159]
Further, the horizontal interpolation circuit 26 can obtain Rh, Gh, and Bh of the processing target pixel by assuming that the correlation in the horizontal direction is strong.
[0160]
Even if the processing target block is an even field, color separation can be performed by the same method.
[0161]
[4] Description of Modification of Correlation Detection Unit 23
Hereinafter, modifications of the correlation detection unit 23 will be described. This modification can be applied not only to the second or third embodiment but also to the first embodiment.
[0162]
As the horizontal direction correlation detection unit 23a and the vertical direction correlation detection unit 23b in the correlation detection unit 23, those that calculate the horizontal direction correlation value Sh and the vertical direction correlation value Sv as follows may be used.
[0163]
The horizontal direction correlation detection means 23a and the vertical direction correlation detection means 23b are based on the signals of 25 pixels in the 5 × 5 pixel block, and the correlation values Sh and Sv corresponding to one central pixel (processing target pixel). Is calculated.
[0164]
[4-1] First, a method of calculating correlation values Sh and Sv when the processing target block is an odd-numbered block in an odd-numbered field as shown in the upper side of FIG.
[0165]
When the processing target block is achromatic, the correlation values Sh and Sv are calculated based on the following equation (8).
[0166]
[Expression 1]

[0167]
The above equation (8) can be simplified as the following equation (8) ′.
[0168]
[Expression 2]

[0169]
As shown in the following equation (9), when calculating the horizontal correlation value Sh, the vertical correlation value is obtained after averaging in the vertical direction, and when calculating the vertical correlation value Sv, the horizontal direction If the vertical correlation value is obtained after averaging the values, the accuracy increases.
[0170]
[Equation 3]

[0171]
In the calculation formula of Sh in the above formula (9), the coefficient applied to G33, B22, B34 is set to 1, and in the calculation formula of Sv, the coefficient applied to G33, R23, R43 is set to 1, so that The divisor may be 8 which is 2 to the power of n, and the division circuit may be configured only by bit shift.
[0172]
The above equation (9) can be simplified as the following equation (9) ′.
[0173]
[Expression 4]

[0174]
In a chromatic color image, the RGB signal ratios output from the CCD of the mosaic arrangement color filter differ depending on the subject color. For this reason, interference due to the subject color occurs, and the accuracy of the correlation direction discrimination result using the CCD all pixel signal is deteriorated. Therefore, the intensity of the color of the input image is determined from the result of integration of the color difference signals in each part of the screen, and for chromatic images, correlation values Sh and Sv are calculated based only on the G signal.
[0175]
That is, when the processing target block is a chromatic color, the correlation values Sh and Sv are calculated based on the following equation (10).
[0176]
[Equation 5]

[0177]
The above equation (10) can be simplified as the following equation (10) ′.
[0178]
[Formula 6]

[0179]
[4-2] A method of calculating correlation values Sh and Sv when the processing target block is an even-numbered block in an odd field as shown in the lower side of FIG. 14 will be described.
[0180]
When the processing target block is achromatic, the correlation values Sh and Sv are calculated based on the following equation (11).
[0181]
[Expression 7]

[0182]
The above equation (11) can be simplified as the following equation (11) ′.
[0183]
[Equation 8]

[0184]
As shown in the following equation (12), when calculating the horizontal correlation value Sh, the vertical correlation value is obtained after averaging in the vertical direction, and when calculating the vertical correlation value Sv, the horizontal direction If the vertical correlation value is obtained after averaging the values, the accuracy increases.
[0185]
[Equation 9]

[0186]
In the calculation formula of Sh in the above formula (12), the coefficient applied to B33, G32, G34 is set to 1, and in the calculation formula of Sv, the coefficient applied to B33, G23, G43 is set to 1, so that The divisor may be 8 which is 2 to the power of n, and the division circuit may be configured only by bit shift.
[0187]
The above equation (12) can be simplified as the following equation (12) ′.
[0188]
[Expression 10]

[0189]
When the processing target block is a chromatic color, the correlation values Sh and Sv are calculated based on the following equation (13).
[0190]
## EQU11 ##

[0191]
The above equation (13) can be simplified as the following equation (13) ′.
[0192]
[Expression 12]

[0193]
When the processing target block is a chromatic color, the correlation values Sh and Sv may be calculated based on the following equation (14), as in the first embodiment.
[0194]
Sh = | G32-G34 |
Sv = | G23−G43 | (14)
[0195]
Even when the processing target block is an even-field block, the G signal pattern is the same, and the same processing may be performed. However, the arithmetic expression when the processing target block is an odd-numbered block in the odd field is the same as the arithmetic expression when the processing target block is an even-numbered block in the even field, and the processing target block is in the odd field. The arithmetic expression for the even-numbered block is the same as the arithmetic expression for when the processing target block is the odd-numbered block in the even field.
[0196]
[4-3] Description of modification of method for calculating correlation values Sh and Sv
The correlation values calculated using arithmetic expressions suitable for achromatic colors (for example, (8) ′ and (11) ′ above) are Sh1 and Sv1, and arithmetic expressions suitable for chromatic colors (for example, The correlation values calculated using (10) ′ and (13) ′) are set to Sh2 and Sv2, and the correlation values Sh and Sv are obtained by performing weighted addition according to the color intensity of the block to be processed. You may do it.
[0197]
First, as shown in FIG. 16, from all the pixel signals in the saturation detection area k including the processing target block among a plurality of preset saturation detection areas, the degree of coloration of the image is expressed by the following formula ( 15), the saturation integrated value C _k Calculate as
[0198]
[Formula 13]

[0199]
Next, as shown in FIG. _k And chromatic / achromatic color judgment value R _k Saturation saturation value C calculated by the above equation (15) _k Chromatic / achromatic color judgment value R corresponding to _k Ask for. It should be noted that the two threshold values Th1 and Th2 in FIG. 17 are experimentally determined because the characteristics of the optical LPF, which are indispensable for systemization, are also involved.
[0200]
Then, as shown by the following equation (16), the obtained chromatic / achromatic color determination value R _k Based on the above, the correlation values Sh1 and Sv1 calculated using an arithmetic expression suitable for an achromatic color and the correlation values Sh2 and Sv2 calculated using an arithmetic expression suitable for a chromatic color are weighted to add.
[0201]
Sh = R _k × Sh1 + (1-R _k ) X Sh2
Sv = R _k × Sv1 + (1-R _k ) × Sv2 (16)
[0202]
Of course, only the expressions (10) and (13) may be used as the chromatic colors without performing the chromatic / achromatic color determination.
[0203]
[5] Description of the fourth embodiment
In the primary color correlated color separation method used in the primary color correlated color separation unit 24 of FIG. 1, when the divisor is 0, the calculation result is infinite. Therefore, when the divisor in the arithmetic expression used in the primary color correlated color separation method is 0, the color signal calculated by the color difference correlated color separation unit 25 in FIG. 1 even when the gain of the automatic gain control means of the camera is small. Is preferably used.
[0204]
Here, “when the divisor becomes 0” is described. However, in the primary color correlated color separation method, the error increases as the divisor becomes smaller. Therefore, when the divisor becomes a certain value or less, the color difference correlated color separation unit 25 in FIG. The color signal calculated by the above may be used.
[0205]
Specifically, in the first embodiment, as shown in FIG. 18, L0, L1, and L2 are input, and a processing target block (a processing target having a size of 3 × 3 pixels in the first embodiment). A determination circuit 40 for determining whether or not a pixel having a pixel value of 0 is included in a block (a processing target block having a size of 5 × 5 pixels in the second embodiment).
[0206]
When the determination circuit 40 determines that a pixel having a pixel value of 0 is included in the processing target block, the determination circuit 40 outputs only the color signal calculated by the color difference correlation color separation unit 25 to 36. Outputs a command.
[0207]
In the second embodiment, it is preferable to perform the same control. Similarly to the primary color correlation color separation unit 24 in FIG. 1, the adaptive correlation color separation unit 24 in the third embodiment has an infinite calculation result when the divisor is 0. Therefore, in the third embodiment, too. It is preferable to perform similar control.
[0208]
【The invention's effect】
As described above, according to the present invention, it is possible to provide a color separation circuit that does not generate a false color signal, thereby improving the image quality.
[0209]
Further, it becomes possible to output vertical or horizontal high-frequency components as they are, and to provide high-resolution color separation.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of the present invention.
FIG. 2 is a diagram showing a CCD used in the present invention.
FIG. 3 is an explanatory diagram showing each signal of the configuration of the present invention.
4 is a block diagram illustrating a configuration of a primary color correlation color separation unit in FIG. 1. FIG.
FIG. 5 is a block diagram illustrating a configuration of a color difference correlated color separation unit in FIG. 1;
FIG. 6 is a block diagram illustrating a configuration of an arithmetic processing circuit.
FIG. 7 is an explanatory diagram illustrating selection of a pixel to be processed.
FIG. 8 is an explanatory diagram showing an interpolation process.
FIG. 9 is an explanatory diagram showing correlation detection.
FIG. 10 is an explanatory diagram for explaining the effect of the invention.
FIG. 11 is an explanatory diagram illustrating interpolation processing.
FIG. 12 is an explanatory diagram for explaining the effect of the invention.
FIG. 13 is an explanatory diagram for explaining operations of a horizontal interpolation circuit and a vertical interpolation circuit 27 in the second embodiment.
FIG. 14 is an explanatory diagram for explaining operations of a horizontal interpolation circuit 29 and a vertical interpolation circuit 30 in the second embodiment;
FIG. 15 is an explanatory diagram for explaining operations of a horizontal interpolation circuit and a vertical interpolation circuit 27 in the third embodiment;
FIG. 16 is a schematic diagram showing a plurality of preset saturation detection areas.
FIG. 17: Saturation integrated value C _k And chromatic / achromatic color judgment value R _k It is a graph which shows the relationship.
FIG. 18 is a block diagram showing a configuration of the fourth exemplary embodiment.
FIG. 19 is an explanatory diagram illustrating generation of a false color signal.
[Explanation of symbols]
1 CCD (solid-state imaging device)
22 Interpolation processing means
23 Correlation detection means
32 Coefficient calculation means
36 Weighted addition circuit

Claims (6)

A single-plate color camera that processes a signal from a solid-state imaging device arranged in a mosaic pattern through an automatic gain control circuit that adjusts the gain of multiple types of color filters with different spectral sensitivity characteristics corresponding to each pixel In the color separation circuit of
Two or more types of color separation means having different color signal component generation methods provided to generate a plurality of color signal components in an arbitrary processing target pixel based on the color signal components of the processing target pixel and surrounding pixels And combining means for combining each color signal component generated by each color separation means based on a signal from the solid-state imaging device,
As color separation means, a plurality of color signal components in an arbitrary pixel to be processed are generated based on the color signal components of the pixel to be processed and surrounding pixels, using the correlation of the color ratios in the horizontal direction or the vertical direction. The primary color correlation color separation means and a plurality of color signal components in an arbitrary processing target pixel are used based on the color difference correlation in the horizontal direction or the vertical direction based on the color signal components of the processing target pixel and surrounding pixels. Color difference correlation color separation means to be generated,
The synthesizing unit synthesizes each color signal component generated by the both color separation unit so that each color signal component generated by the primary color correlated color separation unit becomes large when the signal level from the solid-state imaging device is large, When the signal level from the solid-state imaging device is low, the color signal components generated by the both color separation means are synthesized so that the color signal components generated by the color difference correlated color separation means become large. Color separation circuit for single-panel color cameras. A single-plate color camera that handles multiple pixels of different spectral sensitivity characteristics corresponding to each pixel through an automatic gain control circuit that performs gain adjustment on signals from solid-state imaging devices arranged in a mosaic pattern In the color separation circuit of
Two or more types of color separation means having different color signal component generation methods provided to generate a plurality of color signal components in an arbitrary processing target pixel based on the color signal components of the processing target pixel and surrounding pixels And
Comprising color synthesizing means for synthesizing each color signal component generated by each color separation means based on a signal from the solid-state imaging device;
As color separation means, a plurality of color signal components in an arbitrary pixel to be processed are generated based on the color signal components of the pixel to be processed and surrounding pixels, using the correlation of the color ratios in the horizontal direction or the vertical direction. The primary color correlation color separation means and a plurality of color signal components in an arbitrary processing target pixel are used based on the color difference correlation in the horizontal direction or the vertical direction based on the color signal components of the processing target pixel and surrounding pixels. Color difference correlation color separation means to be generated,
When the gain of the automatic gain adjustment circuit corresponding to the magnitude of the signal level from the solid-state image pickup device is large, the combining means separates the two color separations so that each color signal component generated by the color difference correlated color separation means becomes large. The color signal components generated by the means are combined, and when the gain of the automatic gain adjustment circuit is small, the color signal components generated by the primary color correlated color separation means are generated so as to increase. A color separation circuit for a single-panel color camera, wherein each color signal component is synthesized. Correlation value detection means for obtaining a correlation value in the horizontal direction and vertical direction in the pixel to be processed is provided, and the primary color correlation color separation means and the color difference correlation color separation means are color signal components suitable for cases where the horizontal correlation is strong. Horizontal direction processing means for generating color signal components and vertical direction processing means for generating color signal components suitable for a case where the correlation in the vertical direction is strong. The primary color correlation color separation means and the color difference correlation color separation means include correlation values. Processing is performed by weighted addition of each color signal component generated by the horizontal direction processing means and each color signal component generated by the vertical direction processing means in accordance with the correlation value in the horizontal direction and the vertical direction detected by the detection means. 3. A color separation circuit for a single-plate color camera according to claim 1, wherein a plurality of color signal components in the target pixel are obtained. Corresponding to each pixel, the spectral plurality of types of color filters having different sensitivity characteristics is a single-chip color for processing through the automatic gain control circuitry for performing signal gain adjusted from the solid-state imaging device disposed in a mosaic pattern In the color separation circuit of the camera,
Two or more types of color separation means having different color signal component generation methods provided to generate a plurality of color signal components in an arbitrary processing target pixel based on the color signal components of the processing target pixel and surrounding pixels And
Comprising color synthesizing means for synthesizing each color signal component generated by each color separation means based on the signal from the solid-state imaging device
As a color separation means, a plurality of color signal components in an arbitrary processing target pixel are determined based on the color signal components of the processing target pixel and surrounding pixels, and the color difference is determined in the horizontal direction or the vertical direction having a strong correlation. And a correlation type color separation means that generates a correlation using a color ratio in a weak correlation direction, and a plurality of color signal components in an arbitrary processing target pixel, the processing target pixel and the surroundings Color difference correlated color separation means for generating using the correlation of the color difference in the horizontal direction or the vertical direction based on the color signal component of the pixel of
The synthesizing unit synthesizes each color signal component generated by the two-color separation unit so that each color signal component generated by the adaptive correlated color separation unit becomes large when the signal level from the solid-state imaging device is large. When the signal level from the solid-state imaging device is small, the color signal components generated by the two color separation means are synthesized so that the color signal components generated by the color difference correlated color separation means become large. Color separation circuit for single-plate color camera. A single-plate color camera that handles multiple pixels of different spectral sensitivity characteristics corresponding to each pixel through an automatic gain control circuit that performs gain adjustment on signals from solid-state imaging devices arranged in a mosaic pattern In the color separation circuit of
Two or more types of color separation means having different color signal component generation methods provided to generate a plurality of color signal components in an arbitrary processing target pixel based on the color signal components of the processing target pixel and surrounding pixels And
Comprising color synthesizing means for synthesizing each color signal component generated by each color separation means based on a signal from the solid-state imaging device;
As a color separation means, a plurality of color signal components in an arbitrary processing target pixel are determined based on the color signal components of the processing target pixel and surrounding pixels, and the color difference is determined in the horizontal direction or the vertical direction having a strong correlation. And a correlation type color separation means that generates a correlation using a color ratio in a weak correlation direction, and a plurality of color signal components in an arbitrary processing target pixel, the processing target pixel and the surroundings Color difference correlated color separation means for generating using the correlation of the color difference in the horizontal direction or the vertical direction based on the color signal component of the pixel of
When the gain of the automatic gain adjustment circuit corresponding to the magnitude of the signal level from the solid-state image pickup device is large, the combining means separates the two color separations so that each color signal component generated by the color difference correlated color separation means becomes large. The color signal components generated by the means are combined, and when the gain of the automatic gain adjustment circuit is small, the color signal components generated by the adaptive correlation color separation means are generated by the both color separation means so as to increase. A color separation circuit for a single-plate color camera, wherein the color signal components are combined. Correlation value detection means for obtaining a correlation value in the horizontal direction and vertical direction in the processing target pixel is provided, and the adaptive correlation color separation means and the color difference correlation color separation means are color signals suitable for cases where the correlation in the horizontal direction is strong. A horizontal direction processing means for generating a component and a vertical direction processing means for generating a color signal component suitable when the correlation in the vertical direction is strong, and the adaptive correlation color separation means and the color difference correlation color separation means include: By performing weighted addition of each color signal component generated by the horizontal direction processing means and each color signal component generated by the vertical direction processing means in accordance with the correlation values in the horizontal direction and the vertical direction detected by the correlation value detection means. 6. A color separation circuit for a single-plate color camera according to claim 4, wherein a plurality of color signal components in the processing target pixel are obtained.

Images