JP2619039B2 - Imaging device - Google Patents

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup apparatus called a video camera or the like which is configured to be small and light so that it can be picked up and picked up. .

[Conventional technology]

2. Description of the Related Art Conventionally, as shown in, for example, JP-A-58-56581 or JP-A-58-157263, an image pickup apparatus which obtains a high-quality image by using a plurality of field images has been used.

FIG. 9 is a block diagram showing a basic configuration of a conventional imaging device. In FIG. 9, reference numeral 901 denotes a solid-state imaging device;
Is an actuator that controls the position of the solid-state image sensor, 903
Is an actuator drive circuit, 904 is a synchronization signal generation circuit, 9
05 is a signal processing circuit.

The operation of the conventional recording / imaging apparatus configured as described above will be described below.
This will be described with reference to FIG. In FIG. 10, P ₁₁ , P ₁₂ , ‥‥, P
Reference numeral ₂₅ denotes a pixel of the solid-state imaging device 901. The actuator drive circuit 903 drives the actuator 902 at a period (one field period) defined by the oscillation frequency of the synchronization signal generation circuit 904, so that the solid-state imaging device 901 moves horizontally by one pixel every one field period. Move in both directions along. Therefore, the odd field period in the NTSCTV signal, the pixel P ₁₁ to P ₂₅ of the solid-state imaging device 901 is in the position indicated by the solid line, the even field period, that pixel
P ₁₁ to P ₂₅ is in the position indicated by reference numeral P _'11 ~P' ₂₅ was moved a distance H of about 1 pixel in the horizontal direction, respectively.
As a result, a signal whose apparent number of pixels is doubled and whose horizontal resolution is doubled is obtained.

FIG. 11 shows a basic configuration of another conventional imaging apparatus. In FIG. 11, reference numeral 1001 denotes an analog / digital converter (hereinafter referred to as “A / D converter”);
Is the image memory, 1002 is the A / D converter 1001 output signal and memory 1
An adder 1003 adds the 003 output signal, 1004 is a memory control circuit for controlling the memory 1003, and 1005 is a synchronization signal generation circuit.

In this imaging apparatus, the input video signal of the first field is converted into a digital signal by an A / D converter 1001 and input to a memory 1003 via an adder 1002. Next, the video signal of the second field is converted into a digital signal by the A / D converter 1001, added to the video signal of the first field recorded in the memory 1003 by the adder 1002, and recorded in the memory 1003. Hereinafter, the video signals in the third and subsequent fields are processed in the same manner.
A video signal reconstructed into a field is obtained. As a result, for example, the sensitivity at low illuminance can be improved, and the signal-to-noise ratio can be improved.

[Problems to be solved by the invention]

However, in the above-described configuration, the resolution is improved by using the video signal of a plurality of fields, and the sensitivity and the signal-to-noise ratio are improved. When the position of the image pickup apparatus body relative to the subject changes, the positional relationship of the subject in the video signal between a plurality of fields is disturbed, and thus the desired effect described above cannot be sufficiently obtained.

An object of the present invention is to provide an image pickup apparatus capable of reliably achieving an improvement in resolution or sensitivity and an improvement in signal-to-noise ratio.

[Means for solving the problem]

According to a first aspect of the present invention, there is provided an image pickup apparatus configured to reconstruct an input image signal of a plurality of fields into an image signal of one field or one frame, wherein a relative position between an optical image and an image pickup element is changed. Position change means, storage means for storing an image sensor output signal of at least one field, a motion detection circuit for detecting a displacement of the image pickup apparatus corresponding to the same position state in the position change means, and an output of the motion detection circuit. And a motion correction circuit for generating correction data for correcting a change in the video signal due to the displacement of the imaging device body, and correcting the input video signal stored in the storage means based on the correction data from the motion correction circuit. Alms,
A memory control circuit for outputting from the storage means a corrected video signal obtained by correcting the displacement of the video signal due to the displacement of the imaging device main body.

According to a second aspect of the present invention, there is provided an image capturing apparatus, comprising: a position changing unit configured to change a relative position between the light image and the image sensor;
Storage means for storing a field image sensor output signal;
A motion detection circuit for detecting a displacement of the imaging device main body, a motion correction circuit for creating correction data for correcting a change in a video signal due to the displacement of the imaging device main body from the output of the motion detection circuit, and the motion correction circuit A memory control circuit that corrects the input video signal stored in the storage unit based on the correction data from the storage unit, and outputs from the storage unit a corrected video signal obtained by correcting the displacement of the video signal due to the displacement of the imaging device body; A moving image output circuit for outputting a means output as a moving image mode; and a still image output circuit for outputting a storage means output as a video signal of a plurality of fields having different spatial positions by the position changing means in a still image mode. I do.

(Operation)

According to the configuration of the present invention, the movement detection circuit detects the movement of the imaging apparatus main body corresponding to the same position state in the position changing means that can change the relative position of the optical image and the imaging element, thereby obtaining the imaging apparatus. Data for correcting a change in the video signal due to the displacement of the main body is created by the motion correction circuit. The input video signal of one field stored in the storage means is subjected to correction based on the correction data by the memory control means, and the corrected video signal thus corrected is stored under the control of the memory control means. Output from the means.

In this way, the corrected video signal output from the storage means removes the change in the video signal due to the displacement of the imaging device main body, so that the images of a plurality of fields are not shifted from each other in one field. It can be reconstructed into a video signal. Therefore, even when, for example, the imaging device body is displaced due to camera shake of the operator and the imaging position is slightly changed, the reconstruction of the image is favorably performed. Thereby, in the imaging device, for example, a configuration having a configuration for improving resolution by superimposing images of a plurality of fields, and a configuration for improving resolution and further improving sensitivity and signal-to-noise ratio at low illuminance are provided. In each of the above-described devices, each function can be operated well.

In addition, by providing a moving image output circuit and a still image output circuit, moving image mode processing and still image mode processing can be performed. It is possible to obtain a high-resolution still image by combining a plurality of field images by combining position movement and camera shake correction.

〔Example〕

FIG. 1 is a block diagram showing a basic configuration of an imaging apparatus according to one embodiment of the present invention. In FIG. 1, reference numeral 101 denotes a memory circuit as storage means for storing an input video signal V _IN ;
Is a memory control circuit that controls the memory circuit 101, and 103 is a field memory 1 that stores an output signal of the memory circuit 101.
N and 104 are addition circuits for adding output signals of the memory circuit 101 and the field memory 103, and 105 is a field memory 1
03 is a memory control circuit that controls 03; 106 is a motion detection circuit that detects the amount of displacement of the imaging device main body due to camera shake, etc. 107 is a motion correction circuit that obtains motion correction data from the output of the motion detection circuit 106; It is a system control circuit for controlling.

The input video signal V _IN is stored in the memory circuit 101 in the form of a one-field video signal, and the motion detection circuit 106
Is input to The motion detection circuit 106 detects a motion amount (displacement amount of the imaging device) between one field by comparing with a video signal inputted one field before. Using the motion amount data, the motion correction circuit 107 creates correction data.
The memory control circuit 102 uses the correction data to
From 01, a corrected video signal obtained by correcting a motion component that is a change in the video signal due to a change in the position of the imaging apparatus body is obtained. The corrected video signal subjected to the motion correction is input to the adding circuit 104 and sequentially stored in the field memory 103. The adding circuit 104 adds and outputs a predetermined number of corrected video signals. The memory control circuit 105 controls the field memory 103 at this time.

For example, a continuous three-field input video signal V _IN
2 (1) to (3) are changed, and the image of the first field shown in FIG. 2 (1) is changed to the second image shown in FIG. 2 (2). Both the change to the image of the field and the change from the image of the second field to the image of the third field shown in FIG. 2 (3) are caused by the displacement of the imaging apparatus main body due to the hand shake of the operator or the like. I do. FIGS. 2 (1) to 2 (3) simultaneously show an input video signal V _IN corresponding to a scanning line indicated by reference numeral 1 _N of each video.
Is shown. In FIG. 2, the change in luminance is indicated by oblique lines.

Wherein in the image of the first field, the input video signal V _IN corresponding to the scanning line l _N, compared to the level A to the period of the scanning start after the time t ₁ through T ₁ for the scanning line l _N, the 2, the level a to the duration of each with the video time t ₂ through t _2, the time t ₃ through t ₃ of the third field. Such image shift due to the displacement of the imaging apparatus main body superimposed images of the first to third field that occurs is picture shown in FIG. 2 (4), a video signal corresponding to the scanning line l _N is Are shown at the same time. That is, since images cannot be accurately superimposed, the sensitivity is improved and the signal-to-noise ratio (hereinafter “S / N
Ratio. " ) Is hindered.

FIGS. 3 (1) to 3 (3) show the above-mentioned FIGS.
(3) The video signal corresponding to the video shown in FIG.
And a video corresponding to the corrected video signal output from the memory circuit 101 after being corrected by the memory control circuit 102. These videos are stored in, for example, the field memories I to III (103). Is done. Figure 3 (1) to (3) there is shown a waveform of a video signal corresponding to the scanning line indicated by reference symbol l _M of each picture at the same time, these video signals, scanning start after with respect to the scanning lines Time t _{0 to} T ₀
Is equal to the level A during the period of. In the addition circuit 104, the third
The video signals corresponding to the images shown in FIGS. 1 (1) to 3 (3) are added, and thus the image shown in FIG. 3 (4) is obtained. Figure 3 (4) are also shown a video signal corresponding to the scanning line l _M is the level of the video signal period time t ₀ through T ₀ is a 3A, extremely sensitive is increased It is understood that there is.

In this way, according to this embodiment, the video signals of a plurality of fields can be added while removing the change in the position of the imaging device main body, which is advantageous in improving the sensitivity of the imaging device particularly at low illuminance. is there.

FIG. 4 is a block diagram showing a basic configuration of an imaging apparatus according to a second embodiment of the present invention. In FIG. 4, parts that are the same as the parts shown in FIG. 1 are given the same reference numerals. In this embodiment, a division circuit 408 is provided at a stage subsequent to the addition circuit 104. In the division circuit 408, the video signal from the addition circuit 104 is divided by the number of added fields.

Accordingly, for example, if the signal level of the video signal of one field is S ₀ and the noise level is N ₀ , the signal level S ₂ when the video signal of two fields is added becomes the video level of the video signal due to the displacement of the imaging device body. Since the motion component is corrected, S ₂ = 2S ₀ ‥‥ (1) The noise level N ₂ is And the S / N ratio at this time is Becomes Thus, it is understood that the S / N ratio is significantly improved according to this embodiment.

FIG. 5 is a block diagram showing a basic configuration of a third embodiment of the present invention. In FIG. 5, reference numeral 501 denotes a solid-state imaging device. Reference numeral 502 denotes an actuator for displacing the solid-state imaging device 501, and reference numeral 503 denotes an actuator driving circuit, which constitute a position changing unit that can change a relative position between the optical image and the solid-state imaging device 501. Reference numeral 504 denotes a motion detection circuit that detects displacement of the imaging apparatus main body due to camera shake, 505 denotes a motion correction circuit that performs motion correction using the motion amount data detected by the motion detection circuit 504, and 506 denotes an output of the solid-state imaging device 501. A memory circuit, which is a storage means for storing a signal, 507 is a memory control circuit that controls the memory circuit 506. 508 is an adder for adding the video signal of one field as a unit, 509 is a memory circuit for storing the output signal of the adder 508, 510 is a memory control circuit for controlling the memory circuit 509, and these constitute a still image output circuit doing. Reference numeral 511 denotes a signal processing circuit that is a moving image output circuit, and 512 denotes a system control circuit that comprehensively controls the above-described circuits.

The operation of the imaging apparatus of the present embodiment thus configured will be described with reference to FIG. P ₁₁ to P ₃₅ in FIG. 6 is a pixel of the solid-state imaging device 501. The actuator driving circuit 503 shown in FIG. 5 drives the actuator 502 at a certain period (field period) under the control of the system control circuit 512, whereby the solid-state imaging device 501 is driven every one field period. It moves bidirectionally along the horizontal direction by a distance of about one pixel. Therefore, that pixel P ₁₁ to P ₃₅ is the odd field period in the NTSCTV signal is in a position indicated by the solid line, the even field period, the solid-state imaging device
Pixels P ₁₁ to P ₃₅ of the 501 is in the position indicated by reference numeral P _'11 ~P' ₃₅ was moved a distance H respectively in the horizontal direction. As a result, a signal whose apparent number of pixels is doubled and whose horizontal resolution is doubled is obtained.

As described above, the output signal of the vibrating solid-state imaging device 501 is stored in the memory circuit 506. Further, the motion detection circuit 504 obtains motion amount data corresponding to the displacement of the imaging apparatus main body due to camera shake or the like, and the motion correction circuit 505 creates motion correction data using this data. Then, based on the motion correction data, the memory control circuit 507
From 06, the corrected video signal whose motion has been corrected is read. The corrected video signal read here is added to the adder 508 and the memory circuit 509.
By the operation of the memory control circuit 510, the image is reorganized into a high-resolution frame still image using a two-field video signal or a high-resolution, high-sensitivity frame still image using a video signal of four or more fields. You.

Further, the corrected video signal read from the memory circuit 506 passes through the signal processing circuit 511 to become a television signal.

In this manner, according to this embodiment, by improving the resolution by the horizontal vibration of the solid-state imaging device 501, the motion is corrected to remove only the signal change due to the displacement of the imaging device main body. Done. A signal processing circuit 511 for outputting a television signal, an adding circuit 508 for outputting a still image, a memory circuit 509, and a memory control circuit 510 are also provided.
With this configuration, it is possible to perform a moving image mode process and a still image mode process.For example, in the moving image mode, a moving image output with normal camera shake correction is performed. Thus, a high-resolution still image can be obtained by combining a plurality of field images.

FIG. 7 shows an example of the configuration of a motion detection circuit 504 for removing only a change in the signal due to the displacement of the imaging apparatus body as described above. In FIG. 7, parts corresponding to the respective parts shown in FIG. 5 are denoted by the same reference numerals. In the figure, reference numerals 701 and 702 denote representative point memories, reference numeral 703 denotes a representative point memory control circuit for controlling the representative point memories 701 and 702, reference numeral 704 denotes an arithmetic circuit for the output signal of the representative point memory 701 or 702 and the input signal _VIN, and reference numeral 705 denotes an arithmetic circuit This is a motion amount generation circuit that obtains motion amount data from the output signal of 704.

The representative point memories 701 and 702 store the values of the representative points in the video signal every other field by the representative point memory control circuit 703. For example, when the value of the representative point of the Nth field is stored in the representative point memory 701 and the value of the representative point of the (N + 1) th field is stored in the representative point memory 702, the arithmetic circuit 704 outputs the next field, the (N + 2) th field signal and 2 The value of the representative point of the Nth field before the field is calculated. In this way, the motion amount generation circuit 705 obtains the motion amount data due to the blur between the frames by calculating the value of each representative point between the frames, that is, between the frames, and the motion correction circuit 505 uses the motion amount data. Obtain motion correction data. Further, the memory control circuit 50 uses the motion correction data.
7 obtains the signal of the (N + 2) th field whose motion has been corrected from the memory circuit 506. With the above operation, the motion detection circuit 504 detects the displacement of the imaging device main body corresponding to the same position state (for example, the first field and the third field) in the actuator 502 and the actuator drive circuit 503 as the position changing means. Will do.

In this manner, motion compensation is performed by detecting the amount of motion between frames, so that despite the change in the position of the solid-state imaging device 501 between the odd field and the even field, the signal due to the displacement of the imaging device main body. The motion compensation can be accurately performed by removing only the change in

FIG. 8 is a block diagram showing a basic configuration of a fourth embodiment of the present invention. Since this embodiment is similar to the third embodiment shown in FIG. 5, the corresponding parts are denoted by the same reference numerals. In the figure, reference numeral 801 denotes a field selection circuit that passes only one of the odd-numbered and even-numbered field signals, 802 denotes a representative point memory that stores a representative point value in the passed field signal, and 803 denotes an output signal of the representative point memory. An operation circuit with the output signal of the field selection circuit 801, a movement amount creation circuit 804 that obtains movement amount data from the output signal of the operation circuit 803, and 805, another movement amount data using the output signal of the movement amount creation circuit 804. The obtained motion amount synthesizing circuit, 806 is a data selection circuit for selecting output signals of the motion amount generating circuit 804 and the motion amount synthesizing circuit 805, and 808 is a field memory. The field selection circuit 801, the representative point memory 802, the arithmetic circuit 803, the motion amount creation circuit 804, the motion amount synthesis circuit 805, and the data selection circuit 80
The motion detection circuit 800 is configured to include 6.

Although the field selection circuit 801 passes only one of the odd-numbered and even-numbered field video signals, a case where only the odd-numbered field video signal is passed will be described here. The representative point memory 802 stores the representative point value of the odd field. The arithmetic circuit 803 calculates a representative point value in a certain odd field and a current odd field signal two fields later. Using the calculation result, the motion amount creation circuit 804 obtains motion amount data. The motion amount data obtained here is inter-frame motion amount data in the odd field signal. Next, the motion amount synthesizing circuit 805 uses two inter-frame motion amount data in the odd field signal described above to create motion amount data in the even field signal therebetween. That is, using the first and third field signals, the motion amount data Do ₃ for correcting the motion of the third field is obtained, and then the motion of the fifth field is similarly corrected using the third and fifth field signals. The motion amount data Do ₅ is obtained. At this time, using the motion amount data Do ₃ and Do ₅ , the motion amount data Do ₄ in the fourth field is obtained by so-called interpolation.

The data selection circuit 806 selects the motion amount data of the odd field and the even field and supplies it to the motion correction circuit 505. The field memory 808 outputs the input video signal with a delay of one field period, and outputs the video signal in the memory circuit 506. And the timing with the motion correction data.

As described above, according to the present embodiment, motion between frames is corrected using one of the field signals (in this case, the odd field signal), and the motion of the even field signal is calculated using the motion correction data between the odd fields. Obtained motion compensation data. Therefore, even when the position of the solid-state imaging device 501 changes in the odd and even fields, an accurate amount of motion can be obtained using the signal when the positions of the solid-state imaging device 501 are equal, and the motion compensation data between frames can be obtained. , The motion correction data between fields is synthesized, so that the motion component between fields can also be corrected.

In the above-described first and second embodiments, a non-recursive configuration using N field memories has been described as a method of adding the motion-compensated corrected video signal. With this configuration, the memory circuit scale can be reduced.

In the third embodiment, the solid-state imaging device is
Although the case of a single-chip camera is shown, two and three cameras, that is, two- and three-chip cameras may be used, and the two-chip and three-chip cameras share a picture element shifting method to further increase the resolution. It is good also as a structure made to improve.

Further, with respect to the amplitude H of the vibration of the solid-state imaging device 501, the case where the vibration width is shorter than the cell pitch length is shown. However, the same applies to the case where the vibration width longer than the cell pitch length is set due to the relationship of a color filter or the like. Also, various configurations such as a stacked type can be considered.

In the third embodiment, a case is described in which a two-dimensional sensor is used as the solid-state imaging device 501. However, a one-dimensional sensor may be used, and the vibration direction is set to the vertical direction to improve the vertical resolution. Alternatively, vibration in two directions, horizontal and vertical, or vibration in an oblique direction may be applied to the solid-state imaging device 501. Further, instead of vibrating the solid-state imaging device, the optical axis may be vibrated,
The relative position between the light image and the image sensor changes similarly.

Further, in the above-described embodiment, the case where the correlation of the video signal between the fields or between the frames is used as the motion detection method has been described. However, a configuration using a motion detection sensor may be used, or a configuration using both of them may be used.

Further, in the fourth embodiment, the motion amount synthesizing circuit 805 is obtained from the motion amount data which is the output signal of the motion amount generating circuit 804.
Has shown the configuration for obtaining the motion amount data of the other fields, but it is also possible to obtain the motion correction data from the motion amount data and obtain the motion correction data of the other field using the motion correction data.

Further, in the third and fourth embodiments, as a method of removing only a signal change due to a displacement of the imaging device main body even when the position of the solid-state imaging device changes, a video signal in which the solid-state imaging device is in the same position state is used. The motion detection is performed using the solid-state imaging device. However, the solid-state imaging device detects the amount of motion from the video signals at different positions, and corrects the position change of the solid-state imaging device from the detected amount of motion to thereby determine the same position. It is also possible to perform a motion detection corresponding to the amount of motion between them.

〔The invention's effect〕

As described above, according to the imaging apparatus of the present invention, it is possible to reconstruct an image of a plurality of fields into a video signal of one field or one frame without causing a mutual shift. Even when the imaging position slightly changes due to a displacement due to camera shake or the like, the image can be properly reconstructed. Thereby, in the imaging apparatus, for example, a configuration having a configuration for improving resolution by superimposing images of a plurality of fields, a configuration for improving resolution, and further improving a sensitivity and a signal-to-noise ratio at low illuminance. For example, each function can be favorably operated.

In addition, by providing a moving image output circuit and a still image output circuit, moving image mode processing and still image mode processing can be performed. It is possible to obtain a high-resolution still image by combining a plurality of field images by combining position movement and camera shake correction.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a basic configuration of a first embodiment of the present invention, FIGS. 2 and 3 are diagrams for explaining the operation of the first embodiment, and FIG. FIG. 5 is a block diagram showing the basic configuration of the second embodiment, FIG. 5 is a block diagram showing the basic configuration of the third embodiment of the present invention, and FIG. 6 is for explaining the operation of the third embodiment. FIG. 7, FIG. 7 is a block diagram showing a configuration example of a motion detection circuit in the third embodiment, FIG. 8 is a block diagram showing a basic configuration of a fourth embodiment of the present invention, and FIG. FIG. 10 is a block diagram showing a basic configuration of an imaging apparatus of the related art, FIG. 10 is a diagram for explaining an operation of the conventional imaging apparatus, and FIG. 11 is a block diagram showing a basic configuration of another conventional example. . 101,506 ... Memory circuit (storage means), 102,507 ... Memory control circuit, 106,504,800 ... Motion detection circuit, 107,505 ...
… Motion compensation circuit

Claims (2)

(57) [Claims] An image pickup apparatus configured to reconstruct an input video signal of a plurality of fields into a video signal of one field or one frame, comprising: a position changing means for changing a relative position between an optical image and an image pickup device; Storage means for storing the image sensor output signal of at least one field; a motion detection circuit for detecting a displacement of the imaging apparatus main body corresponding to the same position state in the position change means; A motion correction circuit for generating correction data for correcting a change in the video signal due to the displacement of the apparatus main body; and correcting the input video signal stored in the storage means based on the correction data from the motion correction circuit. A memory control circuit for outputting, from the storage unit, a corrected video signal obtained by correcting a displacement of the video signal due to the displacement of the imaging device body. Imaging device, characterized in that the. 2. A position change means for changing a relative position between a light image and an image pickup device, a storage means for storing at least one field of the image pickup device output signal, and a motion detection circuit for detecting a displacement of an image pickup device main body. A motion correction circuit that generates correction data for correcting a change in a video signal due to a displacement of the imaging apparatus main body from the motion detection circuit output; and the storage unit based on the correction data from the motion correction circuit. A memory control circuit that corrects the input video signal stored in the storage device and outputs a corrected video signal obtained by correcting the displacement of the video signal due to the displacement of the imaging device main body from the storage unit; and outputs the storage unit output as a moving image mode. A moving image output circuit, and the storage unit output, which is a video signal of a plurality of fields having different spatial positions due to the position changing unit, is set to a still image mode. Imaging apparatus characterized by comprising a still image output circuit for outputting Te.