JP2006217416A - Imaging apparatus and control method thereof - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To allow output of an image signal adapted to enlargement/reduction processing of an image. <P>SOLUTION: A motion determination circuit 41 determines a motion of an image within a first image signal on the basis of a first image signal output from a CCD 1 and a delayed delay image signal held in a field memory 5, and a line interpolation circuit 42 synthesizes the first image signal and the first delay image signal in accordance with the determination results to generate an interpolation signal, and a scan conversion circuit 43 generates a second image signal on the basis of the first image signal and the first interpolation signal, and an enlargement/reduction circuit 61 performs frequency control of the second image signal on the basis of a set enlargement/reduction rate and the determination results. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an imaging apparatus to which, for example, an IP conversion technique is applied and a control method thereof.

  Conventionally, there is a so-called IP conversion technique in which an interlaced image is interpolated to generate a progressive image in which scanning lines between the interlaced images are filled. For example, this IP conversion technology is incorporated into progressive DVD players and televisions that are currently popular.

  In the IP conversion, the moving part and the stationary part are usually determined from the correlation between the current field and the field images before and after the current field. An intra-field interpolation process generated from the current field is applied to the motion part, and an inter-field interpolation process generated from the preceding and succeeding fields is applied to the stationary part.

  Also, in recent imaging devices such as digital video cameras, image quality is improved by reducing the size of captured moving and still images using an imaging device that exceeds the number of pixels required for television formats such as NTSC. Have been made. By applying the IP conversion technology to such an image pickup apparatus, further improvement in image quality has been achieved (see, for example, Patent Document 1 and Patent Document 2).

JP-A-6-233184 Japanese Patent No. 3157706

  In Patent Document 1, it is proposed to control a band limiting filter at the time of reduction based on motion information detected by an IP conversion unit. However, the image quality of the reduced image is greatly influenced by the interpolation process, and it is necessary to appropriately process the deterioration of the image quality due to the IP conversion.

  Patent Document 2 proposes controlling a band limiting filter based on detected motion information for an interlaced signal before IP conversion. However, the vertical sampling frequency is half before IP conversion, and further, processing is performed independently for each field, so that processing for signals after IP conversion is not necessarily equivalent. Furthermore, in the image enlargement process, a misjudgment in motion determination that occurs during IP conversion is also enlarged, which causes image quality degradation.

  SUMMARY OF THE INVENTION An object of the present invention is to enable output of a video signal adaptive to image enlargement / reduction processing.

  The imaging apparatus of the present invention includes an imaging unit that images a subject, a memory for delaying a first video signal output from the imaging unit, the first video signal output from the imaging unit, and the Based on the delayed video signal that is the delayed first video signal held in the memory, a motion determination unit that determines a motion of the image in the first video signal, and a determination result by the motion determination unit In response, the first video signal and the delayed video signal are combined to generate an interpolation signal, and an interpolation signal generation means for generating an interpolation signal, and a second video signal is generated based on the first video signal and the interpolation signal And a frequency control means for controlling the frequency of the second video signal based on the set enlargement / reduction ratio and the determination result.

  An imaging apparatus control method according to the present invention is an imaging apparatus control method comprising: imaging means for imaging a subject; and a memory for delaying a first video signal output from the imaging means. The movement of the image in the first video signal is determined based on the first video signal output from the means and the delayed video signal that is the delayed first video signal held in the memory A motion determination step, an interpolation signal generation step of combining the first video signal and the delayed video signal in accordance with a determination result of the motion determination step to generate an interpolation signal, the first video signal, and the A video signal generating step for generating a second video signal based on the interpolation signal, and a frequency control for controlling the frequency of the second video signal based on the set scaling ratio and the determination result Characterized in that it comprises a step.

  A program according to the present invention causes a computer to execute the control method of the imaging apparatus.

  The computer-readable recording medium of the present invention is characterized in that the program is recorded.

  According to the present invention, the frequency control of the video signal is performed based on the set enlargement / reduction ratio and the motion determination result, and the video signal adaptive to the image enlargement process can be output.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments to which the invention is applied will be described in detail with reference to the accompanying drawings.

<First Embodiment>
First, a first embodiment of the present invention will be described. FIG. 1 is a block diagram showing a configuration of a signal processing system of a digital video camera according to the first embodiment of the present invention.

  Although various things can be considered regarding the number of pixels of the CCD 1 that is an image sensor, in the present embodiment, a larger number of pixels than the required number of pixels required from the recorded moving image format can be output at a predetermined field rate. Take what is driven as an example.

  The interlaced video signal photoelectrically converted by the CCD 1 is converted into a digital signal by the A / D converter 2 and input to the camera signal processing circuit 3. The camera signal processing circuit 3 performs color separation, white balance, gamma processing, and the like, and outputs video signals of luminance signals (hereinafter referred to as Y signals) and color difference signals (hereinafter referred to as Cb signals and Cr signals), for example, 4: Output in 2: 2 band.

  In FIG. 1, the output cur of the camera signal processing circuit 3 is one line, and the Y signal, the Cb signal and the Cr signal are multiplexed. Here, the Y signal, the Cb signal and the Cr signal are two lines. You may be independent. The following description is also the same.

  The output cur of the camera signal processing circuit 3 is input to the IP conversion circuit 4, and the output pre of the field memory 5 is further input to the IP conversion circuit 4.

  The IP conversion circuit 4 interpolates a line corresponding to the output of the field memory 5 from the output cur of the camera signal processing circuit 3 and the output pre of the field memory 5 based on the motion determination result, and converts it into a progressive video signal. At the same time, a motion determination value k for the interpolation line is output.

  The enlargement / reduction circuit 61 changes the interpolation processing based on the motion determination result, and outputs the progressive video signal after being enlarged / reduced in the horizontal and vertical directions.

  Here, as shown in FIG. 2, the CCD 1 performs so-called interlaced reading in which charges in the vertical direction are added on the vertical transfer path of the CCD 1 and the combination of addition is shifted depending on the even number (even) and odd number (odd) of the reading field. Do.

  Further, the field memory 5 has a capacity capable of delaying the output of the camera signal processing circuit 3 by at least one field, and the delayed field video signal pre is supplied to the output cur of the camera signal processing circuit 3 as shown in FIG. Outputs with matching timing so as to be adjacent line positions.

  Next, the operation of the IP conversion circuit 4 will be described in detail. The output cur of the camera signal processing circuit 3 and the delayed field video signal pre input to the IP conversion circuit 4 are input to a motion determination circuit 41 and a line interpolation circuit 42 inside the IP conversion circuit 4.

  The motion determination circuit 41 determines the motion of the interpolation line position from the output cur of the camera signal processing circuit 3 and the delayed field video signal pre. For example, as shown in FIG. 4, the line positional relationship at this time sandwiches the cur0 obtained by delaying the output cur of the camera signal processing circuit 3 by one line and the field video signal pre delayed by the current input line cur1. Like that. The interpolation line in this case is the position of the delayed field video signal pre, and the interpolation pixel position (target pixel position) is Pi.

  The motion determination circuit 41 analyzes the in-field high-frequency component and the inter-field high-frequency component of each of the current field video signal cur and the delayed field video signal pre for the target pixel position Pi.

  If the high-frequency component in the field is dominant, the pixel position is likely to be stationary, so it is judged that the pixel is stationary. If the inter-field high-frequency component is dominant, the pixel position may be moving. Since it is highly probable, it is judged to be close to movement.

  The result of this motion determination is output as a multi-value expression value k so as to continuously represent from complete stillness determination to complete motion determination. For example, k is an 8-bit digital signal value. When k = 0, it is completely stationary, and when k = 255, it is completely moved.

  The motion determination value k obtained in this way is input to the line interpolation circuit 42, and the line interpolation circuit 42 indicates the delay field video signal pre if k is small, and presents if k is large. The field video signal cur is weighted so as to be used for synthesis and output as an interpolation line ipr.

  The scan conversion circuit 43 converts the input current field video signal cur and the interpolation line ipr to the progressive video signal Sp by alternately outputting the current line video signal cur and the interpolation line ipr so as to be sequentially scanned.

  Next, the operation of the enlargement / reduction circuit 61 will be described in detail. The enlargement / reduction circuit 61 receives the progressive video signal Sp, and the motion determination value k output from the motion determination circuit 41 in the IP conversion circuit 4 and the set enlargement / reduction ratio. The enlargement / reduction circuit 61 is configured as shown in FIG. 5, for example. The input signal Sp of the enlargement / reduction circuit 61 is input to the narrowband interpolation circuit 611 and the wideband interpolation circuit 612.

  The narrowband interpolation circuit 611 is an interpolation circuit having frequency adjustment and interpolation characteristics so as to output a narrowband signal. For example, the narrowband interpolation circuit 611 has characteristics as shown in FIG. The broken line in the figure indicates the baseband band of Sp, and the blacked area indicates the frequency characteristic of the output of the narrowband interpolation circuit 611.

  The wideband interpolation circuit 612 is an interpolation circuit having frequency adjustment and interpolation characteristics for outputting a wideband signal, and has characteristics as shown in FIG. 7, for example. The broken line in the figure indicates the baseband band of Sp, and the blacked area indicates the frequency characteristic of the output of the wideband interpolation circuit 612.

  The synthesizing circuit 613 receives the video signals interpolated by the respective interpolation methods, weights and synthesizes the two signals according to the value of the motion determination value k, and outputs an interpolated signal output Sf.

  When the motion determination value k is large and there is a high possibility that the pixel of interest is moving, the IP conversion circuit 4 generates an interpolation line containing a large amount of the current field signal, so that the original signal band is narrow. If the band is further limited by an interpolation circuit for such a signal, the image blurs more than necessary.

  In addition, when the motion determination value k is small and there is a high possibility that the pixel of interest is stationary, the IP conversion circuit 4 generates an interpolation line including a large amount of delayed field signals, so that the original signal band becomes wide. Yes. If appropriate band limitation is not performed on such a signal by the interpolation circuit, image folding back and movement determination fluctuations will occur more than necessary.

  Accordingly, the synthesis circuit 613 weights and synthesizes the narrowband interpolation value when the motion determination value k is small and weights the two signals so as to include more wideband interpolation values as the motion determination value k increases. FIG. 8 is a diagram illustrating the frequency characteristics of the synthesis circuit 613. The broken lines in FIG. 8 indicate the baseband band of Sp, and the black portions indicate the frequency characteristics of the output of the synthesis circuit 613.

  Here, when the enlargement ratio of the image increases, there is a problem that the fluctuation of the motion determination value is enlarged and becomes conspicuous as an obstruction. Therefore, the synthesis circuit 63 weights and synthesizes the two signals so as to include many narrowband interpolation values so as to suppress high-frequency components in the band as the enlargement ratio increases. The interpolated video signal Sf obtained in this way is output as a very good interpolated video signal because it has been subjected to interpolation processing taking into account the frequency characteristics during IP conversion processing.

  In this embodiment, the frequency adjustment based on the motion determination value and the enlargement / reduction ratio is configured by an interpolation circuit having two bands. However, the interpolation algorithm itself may be different as long as the frequency adjustment is possible. There may be two or more candidate interpolation values.

<Second Embodiment>
Next, a second embodiment of the present invention will be described. FIG. 9 is a block diagram showing a configuration of a signal processing system of a digital video camera according to the second embodiment of the present invention. As shown in FIG. 9 and the like, the configuration of the signal processing system of the present embodiment and the configuration of the signal processing system of the first embodiment are different in the configuration of the enlargement / reduction circuit 61 and the enlargement / reduction circuit 62, and the other parts. Is the same. Therefore, only the configuration of the enlargement / reduction circuit 6 will be described, and the description of the other parts will be omitted.

  The operation of the enlargement / reduction circuit 62 of this embodiment will be described in detail. The scaling circuit 62 receives the progressive video signal Sp, and inputs the motion judgment value k output from the motion judgment circuit 41 in the IP conversion circuit 4 and the scaling ratio determined from the output video signal size. Is done.

  The interpolation coefficient generation circuit 620 generates a magnification coefficient z indicating the enlargement / reduction ratio and an interpolation coefficient α for each pixel for each interpolation position from the set enlargement / reduction ratio. The input signal Sp of the enlargement / reduction circuit 62 is input to the band limiting circuit 621, and the motion determination value k and the magnification coefficient z are input to the band limiting circuit 621 at the same time.

  In the band limiting circuit 621, for example, frequency characteristic control as shown in FIG. 10 is performed, and when the magnification coefficient z indicates the reduction ratio, the pass band is limited according to the reduction ratio. That is, the band limiting circuit 621 performs control so as to widen the pass band as the reduction ratio decreases.

  Furthermore, the band limiting circuit 621 limits the pass band according to the motion determination value k. When the motion determination value k is large and there is a high possibility that the pixel of interest is moving, the IP conversion circuit 4 generates an interpolation line containing a large amount of the current field signal, so that the original signal band is narrow. If the band limiting circuit 621 further limits the band for such a signal, the image blurs more than necessary.

  In addition, when the motion determination value k is small and there is a high possibility that the pixel of interest is stationary, the IP conversion circuit 4 generates an interpolation line including a large amount of delayed field signals, so that the original signal band becomes wide. Yes. If appropriate band limitation is not performed on such a signal, the image aliasing is more than necessary. Accordingly, the band limiting circuit 621 controls the narrow band when the motion determination value k is small and the wide band as the motion determination value k increases, and outputs the video signal Ss.

  However, when the enlargement ratio of the image increases, the fluctuation of the motion determination value is enlarged and becomes more noticeable as an obstruction. Therefore, the band limiting circuit 621 controls the narrow band when the enlargement ratio increases, and outputs the video signal Ss.

  The output signal Ss of the band limiting circuit 621 is input to the interpolation circuit 622. A conceptual diagram of the interpolation calculation performed by the interpolation circuit 622 is shown in FIG. Here, Pn indicates the pixel position of the video signal Ss, and Pi indicates the interpolation point. Xn represents the distance between the interpolation point Pi and the video signal Pn. Here, assuming that the time function is f (Xn), Pi can be obtained by the following Equation 1.

  The time function f (Xn) is as shown in FIG. 15, and the function curve can be freely changed. When the time characteristic of the thick line in FIG. 15 is expressed by the frequency characteristic, a flat characteristic without a peak is shown as shown by the thick line in FIG. In the interpolation circuit 622, for example, frequency characteristic control as shown in FIG. 11 is performed according to the motion determination value k.

  When the motion determination value k is large and there is a high possibility that the pixel of interest is moving, the IP conversion circuit 4 generates an interpolation line containing a large amount of the current field signal, so that the original signal band is narrow. If such a signal suppresses a high-frequency component by the interpolation circuit 622, the image blurs more than necessary.

  In addition, when the motion determination value k is small and there is a high possibility that the target pixel is stationary, the IP conversion circuit 4 generates an interpolation line including a lot of delay field signals, so that the original signal band becomes wide. Yes. Unless appropriate suppression of high-frequency components is performed on such a signal, image folding interference will occur more than necessary. Therefore, the interpolation circuit 622 suppresses the high frequency component when the motion determination value k is small, and controls to compensate the high frequency component as the motion determination value k increases, and outputs the video signal Si. At this time, the time function f (Xn) may be changed gently as shown by the thick line in FIG.

  However, when the enlargement ratio of the image increases, the fluctuation of the motion determination value is enlarged and becomes more noticeable as an obstruction. Therefore, the interpolation circuit 622 controls to reduce the compensation of the high frequency component as the enlargement ratio increases, and outputs the video signal Si. At this time, the time function f (Xn) may be changed abruptly as shown by a thin line in FIG.

  The output signal Si from the interpolation circuit 622 is input to the contour compensation circuit 623. In the contour compensation circuit 623, for example, frequency characteristic control as shown in FIGS. 12 and 13 is performed according to the motion determination value k. When the motion determination value k is large and there is a high possibility that the pixel of interest is moving, the IP conversion circuit 4 generates an interpolation line containing a large amount of the current field signal, so that the original signal band is narrow. If the high frequency component is not sufficiently compensated for such a signal by the contour compensation circuit 623, the blur of the image is not improved as expected.

  In addition, when the motion determination value k is small and there is a high possibility that the target pixel is stationary, the IP conversion circuit 4 generates an interpolation line including a lot of delay field signals, so that the original signal band becomes wide. Yes. If the high frequency component is excessively compensated for such a signal, the image aliasing is more than necessary. Accordingly, the contour compensation circuit 623 suppresses the high frequency component when the motion determination value k is small, and controls to compensate the high frequency component as the motion determination value k increases, and outputs the video signal Sf.

  Contour compensation control methods include changing the characteristics of the contour extraction filter as shown in FIG. 12, or changing the contour compensation gain as shown in FIG. However, when the enlargement ratio of the image increases, the fluctuation of the motion determination value is enlarged and becomes more noticeable as an obstruction. Therefore, the interpolation circuit 622 controls to reduce the degree of compensation of the high-frequency component when the enlargement ratio increases, and outputs the video signal Sf.

  The interpolated video signal Sf obtained in this way is output as a very good interpolated video signal because it has been subjected to interpolation processing taking into account the frequency characteristics during IP conversion processing.

  As described above, according to the above-described embodiment, it is possible to adaptively control the output image quality according to the motion determination result and the enlargement / reduction ratio when the IP converted progressive video signal is enlarged / reduced. .

  Another object of the present invention is to supply a storage medium storing software program codes for realizing the functions of the above-described embodiments to a system or apparatus, and the computer (or CPU or MPU) of the system or apparatus stores the storage medium. Needless to say, this can also be achieved by reading and executing the program code stored in.

  In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the program code itself and the storage medium storing the program code constitute the present invention.

  As a storage medium for supplying the program code, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.

  Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an OS (basic system or operating system) running on the computer based on the instruction of the program code. Needless to say, a case where the functions of the above-described embodiment are realized by performing part or all of the actual processing and the processing is included.

  Further, after the program code read from the storage medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, the function is determined based on the instruction of the program code. It goes without saying that the CPU or the like provided in the expansion board or function expansion unit performs part or all of the actual processing and the functions of the above-described embodiments are realized by the processing.

1 is a block diagram illustrating a configuration of a signal processing system of a digital video camera according to a first embodiment of the present invention. It is a figure for demonstrating the charge read-out operation | movement from CCD. It is a figure which shows the perpendicular | vertical phase of the input signal to an IP converter circuit. It is a figure for demonstrating the motion determination operation | movement of a motion determination circuit. It is a figure which shows the structure of the expansion / contraction circuit in the 1st Embodiment of this invention. It is a figure which shows the frequency characteristic of a narrow-band interpolation circuit. It is a figure which shows the frequency characteristic of a wideband interpolation circuit. It is a figure which shows the frequency characteristic of a wideband interpolation circuit. It is a block diagram which shows the structure of the signal processing system of the digital video camera which concerns on the 2nd Embodiment of this invention. It is a figure which shows the frequency characteristic of a band-limiting circuit. It is a figure which shows the frequency characteristic of an interpolation circuit. It is a figure which shows the frequency characteristic of a contour compensation circuit. It is a figure which shows the gain characteristic of a contour compensation circuit. It is a figure which shows the conceptual diagram of an interpolation circuit. It is a figure which shows the time characteristic of an interpolation circuit.

Explanation of symbols

1 CCD
2 A / D converter 3 Camera signal processing circuit 4 IP conversion circuit 41 Motion determination circuit 42 Line interpolation circuit 43 Scan conversion circuit 5 Field memory 61, 62 Enlargement / reduction circuit 611 Narrow band interpolation circuit 612 Wide band interpolation circuit 613 Synthesis circuit 620 Interpolation coefficient Generating circuit 621 Band limiting circuit 622 Interpolating circuit 623 Contour compensation circuit 7 TG (timing generator)

Claims (12)

Imaging means for imaging a subject;
A memory for delaying the first video signal output from the imaging means;
The motion of the image in the first video signal based on the first video signal output from the imaging means and the delayed video signal which is the delayed first video signal held in the memory Movement determination means for determining
An interpolation signal generating means for combining the first video signal and the delayed video signal according to a determination result by the motion determining means to generate an interpolation signal;
Video signal generating means for generating a second video signal based on the first video signal and the interpolation signal;
An imaging apparatus comprising: a frequency control unit that performs frequency control of the second video signal based on a set enlargement / reduction ratio and the determination result. The frequency control means includes
A narrowband signal output unit for controlling the frequency of the second video signal to a narrowband signal and outputting the narrowband signal;
A wideband signal output unit for frequency-controlling and outputting the second video signal to a wideband signal;
And a combined signal output unit configured to combine and output the output from the narrowband signal output unit and the output from the wideband signal output unit according to the enlargement / reduction ratio and the determination result. Item 2. The imaging device according to Item 1. As the enlargement ratio indicated by the enlargement / reduction ratio increases, the combined signal output unit
The imaging apparatus according to claim 2, wherein outputs of the narrowband signal output unit and the wideband signal output unit are combined so as to include a large amount of output from the narrowband signal output unit.   The imaging apparatus according to claim 1, wherein the frequency control unit includes a band limiting unit that allows the second video signal to pass in a frequency band corresponding to the enlargement / reduction ratio and the determination result.   The imaging apparatus according to claim 4, wherein the band limiting unit controls a frequency band through which the second video signal is passed to a narrow band as an enlargement ratio indicated by the enlargement / reduction ratio increases. .   6. The frequency control unit according to claim 1, further comprising an interpolation unit that controls frequency characteristics of the second video signal in accordance with the enlargement / reduction ratio and the determination result. The imaging device described in 1.   The imaging apparatus according to claim 6, wherein the interpolation unit performs control to reduce compensation of a high-frequency component of the second video signal as an enlargement ratio indicated by the enlargement / reduction ratio increases.   The frequency control means includes a contour compensation unit that controls frequency characteristics of the second video signal in accordance with the enlargement / reduction ratio and the determination result. The imaging device according to any one of the above.   The imaging apparatus according to claim 8, wherein the contour compensation unit performs control to reduce compensation of a high-frequency component of the second video signal as an enlargement ratio indicated by the enlargement / reduction ratio increases. An imaging apparatus control method comprising: an imaging unit that images a subject; and a memory for delaying a first video signal output from the imaging unit,
The motion of the image in the first video signal based on the first video signal output from the imaging means and the delayed video signal which is the delayed first video signal held in the memory A motion determination step for determining
An interpolation signal generating step of generating an interpolation signal by combining the first video signal and the delayed video signal according to the determination result of the motion determination step;
A video signal generation step of generating a second video signal based on the first video signal and the interpolation signal;
And a frequency control step of performing frequency control of the second video signal based on a set enlargement / reduction ratio and the determination result.   A program for causing a computer to execute the control method of the imaging apparatus according to claim 10.   The computer-readable recording medium which recorded the program of Claim 11.

Images