JP2005086731A - Imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To take a high-resolution still image without using a mechanical shutter. <P>SOLUTION: An image of a photo object obtained through a lens is converted into electric signals by a solid-state imaging device, given analog processing and A/D conversion in an AFE. Image signals converted into digital signals are then written in a memory by field by field. Motion detection is executed on the basis of 4 consecutive field images written in the memory and output from the AFE, and first motion information D1 and second motion information D2 are output. When generating a frame image, a frame composition unit executes an interpolation processing with respect to a motion part between an A field and a B field based on the first motion information D1 and the second motion information D2, and then writes in the memory. A camera signal processing unit executes processings necessary for a camera image such as luminance, color signal generation, γ processing and contour corrections to an image obtained from image data output from the AFE and the frame image stored in the memory. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an imaging apparatus having a solid-state imaging device, such as a video camera or a digital still camera.

  In an imaging apparatus having a solid-state imaging device such as a video camera, when shooting a still image, the solid-state imaging device is exposed for a certain period and then the mechanical shutter is closed, and it takes more than one field period to exceed the signal amount of the field from the solid-state imaging device. In general, a still image is generated by reading the above signal. Normally, during moving image shooting, readout from the solid-state imaging device reads out the signal amount of the field within the field period, and therefore moving images cannot be shot during still image shooting.

  To capture a still image while shooting a moving image, there is a method of generating a single frame image by combining two continuous field images of the moving image. When the subject moves between fields, the moving part is If it is a frame image, it will be “blurred”. A method described in Japanese Patent Application Laid-Open No. 11-187306 is known as a method for generating a still image by correcting this blur.

  FIG. 8 is a configuration diagram of a conventional imaging apparatus. An image passing through a lens 81 is photoelectrically converted by a solid-state imaging device 82, a signal component is sampled by an IC 83 in which CDS / AGC / ADC is made into one chip, and an appropriate amplitude is obtained. The adjusted signal is digitally converted. The digital signal processing IC 87 includes a camera signal processing unit 87a, a digital auto unit 87c, a synchronization signal generation unit 87d, a microcomputer interface unit 87e, a clock generation unit 87f, a frequency conversion unit 87g, and an electronic zoom unit 87b, and a signal output from the IC 83. Is subjected to luminance signal processing such as γ processing, contour correction, and color signal processing for generating a color difference in the camera signal processing unit 87a. The IC 84 generates a driving pulse for the solid-state image sensor 82 and a sampling pulse at the IC 83. 85a is each horizontal speed sensor, 85b is each vertical speed sensor, and detects camera shake. Reference numerals 88a and 88b denote field memories.

  The operation of the conventional imaging apparatus configured as described above will be described below.

  In the camera shake detection, the horizontal angular velocity sensor 85a and the vertical angular velocity sensor 85b are used as the angular velocity sensors, and the microcomputer 86 performs integration calculation or the like to obtain camera shake correction data. In the camera shake correction in the vertical direction, a timing generator 84 that drives the CCD image sensor 82 controls the number of high-speed vertical transfer pulses to cut out an arbitrary signal in the vertical direction. In addition, in the horizontal direction image stabilization, an arbitrary signal in the horizontal direction is cut out while performing frequency conversion using a dual-port line memory built in the frequency conversion unit 87g. The electronic zoom in the vertical direction uses external field memories 88a and 88b, and the electronic zoom in the horizontal direction is interpolated using a line memory built in the electronic zoom unit 87b.

  Also, during still image recording, motion detection is performed by calculating the correlation of data for two fields, and so-called clear frame processing is performed. As output, moving images are interfaced in a digital VTR format and output to a digital VTR recording system, and still images are interfaced in a digital still format and output to a JPEG (Joint Photographic Coding Experts Group) recording system.

As described above, the conventional imaging apparatus performs motion detection by calculating the correlation of data for two fields stored in the field memories 88a and 88b during still image recording, and performs so-called clear frame processing.
JP-A-11-187306 (FIG. 1)

  However, in the conventional imaging apparatus, a still image is generated by clear frame processing, but motion detection is performed by calculating the correlation of data for two fields during clear frame processing. When motion is detected in two fields, if the subject is such that there are shades for each line in the state of the frame, the two fields constituting the frame have different values, so the motion is erroneously determined despite the stationary subject. However, there is a problem that a high-definition image cannot be generated during clear frame processing.

  In view of the above problems, the present invention provides an imaging apparatus that can generate a high-definition still image without using a mechanical shutter function because it can generate a high-definition frame still image from continuous two-field images. For the purpose.

  In order to achieve this object, an imaging apparatus of the present invention converts a lens, a solid-state imaging device that converts light passing through the lens into an electrical signal, and converts the output signal of the solid-state imaging device into a digital signal after analog processing. Means, a rewritable memory for storing the digital signal, a motion detection unit for detecting a motion part of an image, and a frame synthesis unit for generating a frame image by synthesizing two field images, the motion detection unit Detects and outputs the motion level of the moving part using the image stored in the memory and the continuous four-field image based on the digital signal, and the frame composition unit continuously outputs the continuous four-field image. The frame image is synthesized by applying different interpolation processing to the two-field image to be processed according to the motion level detected by the motion detection unit. And characterized in that it forms output.

  Since the imaging apparatus of the present invention can generate a high-definition frame still image from continuous two-field images, it can capture a high-definition still image without using a mechanical shutter function.

  According to the first to fifth aspects of the present invention, a lens, a solid-state image sensor that converts light passing through the lens into an electric signal, and an output signal of the solid-state image sensor are converted into a digital signal after analog processing. Conversion means, rewritable storage means for accumulating the digital signal, motion detection means for detecting a motion part of the image, and frame synthesis means for synthesizing two field images to generate a frame image, The motion detection means detects and outputs a motion level of a motion part using a plurality of field images accumulated in the storage means, and the frame composition means outputs at least two consecutive fields of the plurality of field images. A frame image is generated and output by performing different interpolation processing on the image according to the motion level detected by the motion detection means and combining the images. , And the thereby has the effect that it is possible to take a high-resolution still image without a mechanical shutter function it is possible to generate a high-definition frame still image from two successive field images.

  Hereinafter, embodiments of the present invention will be described with reference to FIGS.

(Embodiment 1)
FIG. 1 is a block diagram showing a configuration of Embodiment 1 of an imaging apparatus according to the present invention, in which 1 is a lens, 2 is an image formed from an optical signal from a lens 1, converted into an electrical signal, and output. An element 3 is an AFE that performs analog signal processing and analog / digital conversion (AD conversion) on an electrical signal from the solid-state imaging device 2, 4 is a memory, 5 is a video signal from the AFE 3, and a video signal from the memory 4. 6 is a motion detection unit that detects motion of a subject image, and 6 is a frame synthesis unit that synthesizes a frame image from field images from the AFE 3 and the memory 4 based on motion information from the motion detection unit 5. And 7 for frame image data obtained by alternately reading out the image data output from the AFE 3 and the image written in the memory 4 for each line. Brightness, color signal generation, gamma processing, a camera signal processing unit is a camera signal processing means for performing processing necessary for the camera image, such as contour correction.

  The operation of the first embodiment of the present invention configured as described above will be described below.

  The subject image via the lens 1 is converted into an electric image signal by the solid-state imaging device 2 and read out by interlace. After analog processing by the AFE 3, it is converted into a digital signal. The image signal converted into the digital signal is written in the memory 4 for each field.

  Here, assuming that the image signal is output from the solid-state imaging device 2 in the order of fields of A0, B0, A1, and B1, the motion detection unit 5 writes the A0, B0, and A1 field images written in the memory 4. And the motion detection is performed based on the continuous four field images of the B1 field image output from the AFE 3, and the first motion information D1 and the second motion information D2 are output.

  The frame synthesizing unit 6 generates a frame image from the B1 field image and the paired A1 field image. At this time, the A1 field and the B1 field are generated based on the first motion information D1 and the second motion information D2. Interpolation processing is performed on the moving parts between them, and A1 ′ and B1 ′ are written in the memory 4.

  The camera signal processing unit 7 generates luminance, color signal, and γ for frame image data obtained by alternately reading image data output from the AFE 3 and A1 ′ and B1 ′ written in the memory 4 for each line. Processing necessary for the camera image such as processing and contour correction is performed.

  FIG. 2 shows a configuration of the motion detection unit 5 according to the first embodiment. In the figure, reference numerals 21a and 21b denote subtractors, and 22a and 22b denote absolute value parts which are absolute value parts for converting an input value into an absolute value. 23a, 23b, 24a, 24b are comparators, and 25, 26 are OR circuits.

  The operation of the motion detector 5 configured as described above will be described below.

The subtractors 21a and 21b calculate the differences between the fields in the same spatial position for the continuous four field images A0, B0, A1 and B1, and the absolute values 22a and 22b convert the respective differences into absolute values. , SubB. (Formula 1A, Formula 1B)
subA = | A1-A0 | (Formula 1A)
subB = | B1-B0 | (Formula 1B)
The comparator 23a compares the difference subA of the A field converted into the absolute value with the comparison value a1, and outputs “1” when subA ≧ comparison value a1, and outputs “0” otherwise.
The comparator 24a compares the difference subA of the A field converted into the absolute value with the comparison value a2, and outputs “1” when subA ≧ comparison value a2, and outputs “0” otherwise.
The comparator 23b compares the difference subB of the B field converted to the absolute value with the comparison value b1, and outputs “1” when subB ≧ comparison value b1, and outputs “0” otherwise.
The comparator 24b compares the difference subB of the B field converted into an absolute value with the comparison value b2, and outputs “1” when subB ≧ comparison value b2, and outputs “0” otherwise.

  In the above processing, when comparing fields at the same spatial position, the stationary part has the same value, and the moving part has a different value, so the difference between the fields at the same spatial position is larger. Since there is a high possibility that the pixel is moving, whether the output value of the comparator is 1 or 0 indicates whether the pixel is a moving part pixel.

  The OR circuit 25 takes the OR of the outputs of the comparators 23a and 23b and outputs it as the first motion information D1, and the OR circuit 26 takes the OR of the outputs of the comparators 24a and 24b and outputs it as the second motion information D2. To do. Here, the comparison value a1, the comparison value a2, the comparison value b1, and the comparison value b2 are rewritable arbitrary values. By setting the comparison value a1 ≧ the comparison value a2 and the comparison value b1 ≧ the comparison value b2, D1 is The information is more likely to move than D2.

  FIG. 3 shows the configuration of the frame synthesizing unit 6 in the first embodiment. In FIG. 3, reference numeral 31 denotes a first delay unit which is first delay means for delaying image data by one line, and 32 denotes an input. Is a first interpolation unit that is a first interpolation unit that performs interpolation processing on two pieces of data, 33 is a selector that is a selection unit, and 34 is a second delay unit that delays image data by one line. A second delay device 35 and 36 are second and third interpolation units which are second and third interpolation means for performing interpolation processing using the three input data. The numbers in parentheses) indicate the relative line positions of the frame images generated at A1 and B1, and the operation will be described below.

  Two consecutive field images A1 and B1 are input to the frame synthesizing unit 6, and B1 is a field image whose spatial position is one line above A1. The first interpolation unit 32 interpolates and generates an image at the line position B1 (2) using A1 (3) and A1 (1) delayed by one line, and designates this as B1h.

  The selector 33 selects B1h generated by interpolation of A1 when the first motion information D1 is 1, and selects B1 (2) itself when D1 is 0 to select B1 after the first motion correction. Output as image mB1.

  The second delay unit 34 delays the motion-corrected B1 image mA1 by one line, and the second interpolation unit 35 performs mB1 image (0) delayed by one line and A1 image (1) delayed by one line. When the second motion information D2 is “1” for the continuous three-line image of the mB1 image (2), the filter process is performed, and when D2 is 0, the image is delayed by one line. A1 itself is output, and this is defined as a motion-corrected A1 image A1 ′.

  The third interpolation unit 36 sets the second motion information D2 to 1 for the A1 image delayed by one line, (1), the mB1 image, (2), and the three consecutive lines of the A1 image (3). In some cases, a filter process is performed and output. When D2 is “0”, the mB1 image itself delayed by one line is output, and this is defined as a motion-corrected B1 image B1 ′.

  FIG. 4 shows an image of the image in the above-described operation. FIG. 4A shows the state of the subject between four consecutive fields, and o1 is a subject that repeats light and shade in the vertical direction that is stationary. , O2 is a subject moving in the direction of the arrow. (B0), (b1), (b2), and (b3) are images of A0, B0, A1, and B1, which are four consecutive fields, and the subjects described in o1 and o2 (a). (CA) and (cB) show motion detection information between A0 and A1 and motion detection information between B0 and B1, respectively, and mA and mB show the respective motion parts. As described with reference to FIG. 2, since motion detection is performed based on the difference between the same fields (subA and subB in FIG. 2), there is no difference in the stationary part (subject o1), and only the part of the moving part (subject o2) is detected. Detected. (D) is an OR of (cA) and (cB), and mAB indicates a moving part between four fields. (R) is a frame image obtained by synthesizing A1 and B1 as they are, and the moving part (subject o2) becomes a blurred image. (E) is a frame image obtained by synthesizing A1 and B1, and the mAB portion in (d) is generated by interpolating A1 without using B1, and therefore the subject o2 that has become a blurred image in (r). The stationary part (subject o1) is generated as a high-definition frame image.

  With the above operation, a high-definition frame still image can be generated from two continuous field images, so a high-definition still image can be taken without using the mechanical shutter function.

  In the above description, the motion detection unit performs motion detection using the absolute value of the difference obtained by the subtractors 1a and 1b. If it is configured to perform the motion detection by quantification, it is possible to reduce the influence of random noise and suppress malfunctions.

(Embodiment 2)
FIG. 5 is a configuration diagram of Embodiment 2 of the apparatus of the present invention, in which 1 is a lens, 2 is a solid-state imaging device, 3 is an AFE that performs analog processing and AD conversion, and 54 is a storage means for temporarily storing signals. Reference numeral 5 denotes a motion detection unit which is a motion detection unit, 56 is a frame synthesis unit which is a frame synthesis unit, and 7 is a camera signal processing unit which is a camera signal processing unit. In the figure, 1, 2, 3, 5, and 7 are the same as those in FIG. The difference from FIG. 1 is that the operation contents of the memory 54 and the frame synthesizing unit 56 have changed, and the operation will be described below.

  First, the second motion information D <b> 2 detected by the motion detector 5 is once written in the memory 54. Based on the first motion information D1, the frame synthesizing unit 56 performs an interpolation process on the motion part for the A1 field, and writes it as A1 'in the memory 54. The processing so far is referred to as sequence 1.

  Next, A1 ′ written in sequence 1 is read from memory 54 (memA1 ′), B1 is read, second motion information D2 written in sequence 1 is read (D2mem), and after frame synthesis, the value of D2mem is used as a basis. Is again written in the memory 54 as frame data AB ′ subjected to interpolation processing and motion correction. This process is referred to as sequence 2.

  FIG. 6 shows the configuration of the frame synthesis unit 56 in the second embodiment. In the figure, 61 is a line memory capable of recording one line of image data, and 62 interpolates two input data. A first interpolation unit as a first interpolation unit, 63 is a selector, 64 is a memory capable of recording image data for one line, 65 and 66 are interpolation processes using three input data The second and third interpolation units are second and third interpolation units for performing the above operations, and 67 is a selector which is a selection unit. The numbers in parentheses in the figure indicate the relative line positions of the frame images generated by the field data A1 and B1, and the operation will be described below.

  First, in the process of sequence 1, continuous two field images A1 and B1 are input to the frame synthesis unit 56, and B1 is a field image whose spatial position is one line above A1. The first interpolation unit 62 interpolates and generates an image at the line position B1 (2) between A1 (3) and A1 (1) delayed by one line by the line memory 61, and designates this as B1h. The selector 63 selects B1h generated by the interpolation of A1 when the first motion information D1 is “1”, and selects B1 (2) itself when D1 is “0”. The corrected B1 image B1 ′ is output.

  In the process of sequence 2, memB1 '(2), A1 (3), and D2mem are read from the memory 64 by repeating the same line twice. The line memories 61 and 64 perform the writing process every other line, and the output data of the line memories 61 and 64 is always read out the data one line before the A1 and memB1 ′, respectively, A1h (0) and memB1′h ( 1).

  The second interpolation unit 65 performs the following operation on the memB1′h image delayed by one line, (0), the A1 image delayed by one line, (1), and the three-line image of the memB1 ′ image (2). When the second motion information memD2 read from the memory 54 is “1”, it is filtered and output. When memD2 is “0”, A1 itself delayed by one line is output, and this is motion-corrected. A1 image hA1.

  The third interpolating unit 66 reads the second line read from the memory 54 for the A1 image delayed by one line, the (1), the memB1 ′ image, (2), and the three lines of the A1 image (3). If the motion information memD2 is “1”, it is filtered and output. If D2 is “0”, the memB1 ′ image itself delayed by one line is output, which is set as a motion-corrected B1 image hB1. The selector 7 outputs hA1 and hB1 as a frame image AB ′ subjected to motion correction by switching and outputting hA1 and hB1 for each line.

  FIG. 7 is a schematic diagram of the timing of the signals described in the frame synthesis unit 56, where □ represents B1 input, A1 input, memB1 ′ input, line memory 1 output, and line memory 2 output memD2 input, respectively. The numerical value in □ is the order of the lines in the frame.

  With the above operation, a high-definition frame still image can be generated from two continuous field images, so a high-definition still image can be taken without using the mechanical shutter function.

  The image pickup apparatus according to the present invention can be applied to uses such as a video camera and a digital camera having a solid-state image pickup device by generating a high-definition still image from a moving image being shot without using a mechanical shutter.

1 is a block diagram showing a configuration showing an image pickup apparatus in Embodiment 1 of the present invention. The block diagram which shows the structure which showed the motion detection part in Embodiment 1 of this invention. The block diagram which shows the structure which showed the frame synthetic | combination part in Embodiment 1 of this invention Schematic diagram showing a configuration showing an image image by the operation of the imaging device in Embodiment 1 of the present invention The block diagram which shows the structure which showed the imaging device in Embodiment 2 of this invention The block diagram which shows the structure which showed the flame | frame synthetic | combination part in Embodiment 2 of this invention. Schematic diagram showing operations in the same embodiment by sequence Block diagram showing the configuration of a conventional imaging device

Explanation of symbols

1 Lens 2 Solid-state image sensor 3 AFE
4 Memory 5 Motion detection unit 6 Frame composition unit 7 Camera signal processing unit

Claims (5)

A lens, a solid-state imaging device that converts light passing through the lens into an electrical signal, a conversion unit that converts an output signal of the solid-state imaging device into an analog signal, and then converts the digital signal into a digital signal; Storage means; motion detection means for detecting a moving part of the image; and frame synthesis means for generating a frame image by synthesizing two field images, wherein the motion detection means includes a plurality of motion detection means stored in the storage means. A motion level of a motion part is detected and output using an image of a field, and the frame synthesis means detects the motion level detected by the motion detection means for at least two consecutive field images among the images of the plurality of fields. An image pickup apparatus characterized in that different interpolation processes are performed in accordance with and combined to generate and output a frame image. A lens, a solid-state imaging device that converts light passing through the lens into an electrical signal, a conversion unit that converts an output signal of the solid-state imaging device into an analog signal, and then converts the digital signal into a digital signal; Storage means; motion detection means for detecting a moving part of the image; and frame synthesis means for synthesizing two field images to generate a frame image, wherein the motion detection means includes an image stored in the storage means, The motion level of a moving part is detected and output using a continuous four-field image based on the digital signal, and the frame synthesizing means performs the motion on the continuous two-field image among the continuous four-field image. Depending on the motion level detected by the detection means, a different interpolation process is applied and combined to generate and output a frame image. Imaging device according to symptoms. The motion detection means obtains the difference A between the A field images and the difference B between the B field images in the continuous four-field images, and the motion is assumed to be higher as the difference A or the difference B is larger. The imaging apparatus according to claim 2, wherein the imaging apparatus is configured to increase a level. The frame synthesizing means synthesizes a field interpolation process for interpolating and synthesizing the other field image without using one of the two consecutive field images when generating the frame image, and synthesizing the two consecutive field images. Depending on the motion level, a frame interpolation process for performing a filtering process in a vertical direction so that the spatial position of the image does not change with respect to the frame image, and a process for leaving the frame image obtained by combining the continuous two-field images as they are The imaging apparatus according to claim 2, wherein the frame image is generated by switching between the two. The frame synthesizing means synthesizes a field interpolation process for interpolating and synthesizing the other field image without using one of the consecutive two field images when generating the frame image, and synthesizing the two consecutive field images. A frame interpolation process for performing a filtering process in the vertical direction so that the spatial position of the image does not change with respect to the frame image, and a process for leaving the frame image obtained by synthesizing the continuous two-field images as the motion level. The imaging apparatus according to claim 2, wherein the frame image is generated by switching in response, and the generated frame image is divided and output.

Images