JPH07203318A - Image pickup device - Google Patents

Abstract

PURPOSE:To Provide an image pickup device for a dynamic image exceeding a resolution limit determined by driving frequency. CONSTITUTION:A CCD 11 is an moving area image pickup element, its frame frequency is 30Hz and the maximum number of picture elements is 2,000,000 picture elements. A CCD 12 is a still area image pickup element, its frame frequency is 7.5Hz and the maximum number of picture elements is 8,000,000. A highly accurate image can be obtained by synthesizing outputs from both the CCDs 11, 12.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving image input technique,
More specifically, it relates to a high-definition moving image pickup device using a solid-state image pickup device.

[0002]

2. Description of the Related Art Since an image pickup apparatus using a solid-state image pickup element has many features such as small size, low power consumption, and no afterimage, it has recently been widely used as a home video camera or a handy type broadcast camera. It is used. However, the resolution is still lower than that of an image pickup apparatus using an image pickup tube, and there are expectations for higher resolution in the future.

Since the resolution of an image pickup device using a solid-state image pickup device is determined by the number of pixels of the solid-state image pickup device used, conventionally, the resolution has been increased by increasing the number of pixels of the image pickup device.

[0004]

As described above, in the conventional image pickup apparatus, high resolution has been realized by the method of increasing the number of pixels of the solid-state image pickup element (hereinafter abbreviated as image pickup element). However, if the number of pixels of the image sensor is increased, the drive frequency of the image sensor must be increased. That is, in the moving image pickup device, at least about 25 or more images must be captured per second in order to continuously reproduce the motion (hereinafter, the number of images captured per second is called a frame frequency. ), The driving frequency is proportional to the product of the number of pixels and the frame frequency, so that increasing the number of pixels requires raising the driving frequency.

When the driving frequency of the image pickup device is increased, the transfer efficiency at the time of transferring the signal charge generated in each pixel to the output end is lowered, and it becomes impossible to realize a high resolution proportional to the number of pixels. In addition, when converting to a digital signal, the conversion amount per unit time increases and A / D conversion becomes difficult. Further, since the amount of heat generated in the image pickup device increases, there is a problem that noise is generated as the temperature of the device rises.

The increase in the number of pixels of the image pickup device has the above-mentioned problems. Therefore, the number of pixels of the moving image pickup device having a frame frequency of about 30 Hz is currently about 2 million. Although there have been reports of image pickup devices having a prime number greater than this, all of these have a frame frequency of several Hz and are used exclusively for still images.

SUMMARY OF THE INVENTION It is an object of the present invention to solve the above-mentioned conventional problems in a moving image pickup apparatus and to provide a moving image pickup apparatus that exceeds a resolution limit that can be achieved by a driving frequency.

[0008]

SUMMARY OF THE INVENTION An image pickup apparatus according to the present invention comprises at least one or a set of moving area image pickup devices whose main purpose is to pick up an image of a moving area, and a moving area image pickup device having a number of pixels. The present invention is provided with at least one or a set of still-region image pickup devices whose main purpose is to image a still region whose number of pixels exceeds the frame frequency of the moving-region image pickup device.

That is, the two image pickup devices for the moving region and the image pickup device for the stationary region are A and B, and the minimum frame frequency required for moving image pickup is fm, or the maximum number of pixels that can be realized when the frame frequency is fm is P _MAX . Then, the moving area image sensor A sets the frame frequency to fm or higher and the number of pixels to P _MAX or lower. On the other hand, the still image pickup device B has a frame frequency lower than fm and a pixel number higher than P _MAX .

[0010]

In the present invention, the two image pickup devices A,
The same region is imaged by B, the moving region is extracted from the image captured by the moving region image sensor A, while the still region is extracted from the image captured by the stationary region image sensor B, and the extracted 2 By combining two images, it is possible to improve the definition of the input image.

That is, since only the still region is extracted from the still region image sensor B, there is no restriction on the frame frequency in the image capturing. Therefore, it is possible to capture an image at a frame frequency smaller than the minimum frequency fm required for capturing a moving image. On the other hand, since the moving region is imaged by the image sensor having a frame frequency of fm or higher, continuous natural motion can be reproduced.

In the above-described image pickup apparatus of the present invention, the resolution of the still area can be increased, but the resolution of the moving area is the same as that of the conventional method. However, it is known that the visual characteristics of the human eye are inferior to the stationary area in the visual characteristics of the moving area, and the resolution required for the moving area is lower than that of the stationary area. Therefore, as in the present invention, the definition of the input image can be visually enhanced by increasing only the resolution of the still region.

[0013]

【Example】

[First Embodiment] A schematic configuration of a first embodiment of the present invention is shown in FIG.
Shown in. In FIG. 1, 11 to 12 are CCD area sensors (hereinafter, simply referred to as CCD), 2 are lenses, 3 are half mirrors, 4 are frame frequency converters, 5 are interpolation circuits,
Reference numeral 6 is a moving area detector, and 7 is an image synthesizer.

In this embodiment, two CCDs are used as the image pickup device.
The same subject image is formed on the image pickup surfaces of the CCDs 11 and 12 by the half mirror 3 arranged behind the lens 2 using 11 and 12. Here, the CCD 11 is a moving-region image pickup element for the purpose of picking up an image of the moving region, and therefore, the frame frequency is set to 30 Hz which enables moving image pickup. Further, the number of pixels of the CCD 11 is 30 when the frame frequency is
The maximum number of pixels that can be realized at present is 200
It is assumed to be 10,000 pixels.

On the other hand, the CCD 12 is an image sensor for a stationary area for the purpose of capturing an image of the stationary area, and the frame frequency is 7.5 Hz which is ¼ of that of the CCD 11. By lowering the frame frequency, the number of pixels can be increased, and the number of pixels of the CDD 12 is set to 8 million pixels, which is four times that of the CDD 11.

The frame frequency of the output image signal V of the image pickup apparatus of this embodiment is set to 30 Hz at which moving images can be reproduced, while
The number of pixels is set to 8 million pixels, which is the same as the number of pixels for the static area image sensor. Hereinafter, a method of generating the output image signal V having such a frame frequency and the number of pixels will be described.

First, the analog output signals of the CCD 11 and CCD 12 are A / D converted and converted into digital signals (the A / D converter is omitted in FIG. 1). Next, the CCD 11
Signal is an image signal of 2 million pixels, the interpolation circuit 5
Interpolation processing is performed to convert into a moving image signal Vm of 8 million pixels. On the other hand, since the frame frequency of the signal of the CCD 12 is 7.5 Hz, this is converted to 30 Hz by the frame frequency converter 4 to generate the still image signal Vs.

FIG. 2 shows a schematic configuration of the frame frequency converter 4.
Shown in. In FIG. 2, 41 and 42 are frame memories. The signal from the CCD 12 is a frame memory 41.
Written in. Since this writing is performed in synchronism with the reading of the signal from the CCD 12, it takes time for four frames in the output image signal V. Next, the image signal written in the frame memory 41 is transferred to the frame memory 42.
Transfer to. This transfer is the last short period of the writing period to the frame memory 41 (0.
It is executed in about 5 frame periods). Frame memory 4
The image signal written in 2 is read out every frame from the next output frame to 4 frames. That is, the same image is output from the frame memory 42 over four frames. The frame frequency of this output is 30H
z.

FIG. 3 shows the output image signal V, CCD 11 and CC.
The temporal relationship of frames in the output signal of D12, the writing period of the frame memories 41 and 42, and the output signals of the CCD11 and CCD12 and the still image signal V
m, the relationship with the video signal Vs. In FIG.
, N-1, n, n + 1, n + 2, ... Show the frame numbers of the output image signal V and the output signal of the CCD 11.
Also, ..., m-1, m, m + 1, m + 2, ... is CC
The frame number of D12 is shown. Furthermore, the still image signal V
m, [] in the video signal Vs indicates the frame number of the output signal of the CCD 11 and CCD 12 corresponding to the frame image. As shown in FIG. 3, although the signal from the CCD 11 is interpolated, it becomes the moving image signal Vm as it is in the same frame. On the other hand, the still image signal V
s is a unit of 4 frame periods, and during 4 frame periods, a signal from the CCD 12 during the immediately preceding 4 frame periods is output.

The moving picture signal Vm and the still picture signal Vs generated by the above method are combined in the image combiner 7 according to the equation (1) to obtain the output image signal V.

[0021]

## EQU1 ## V = kVm + (1-k) Vs (1) where k is a moving area detection signal, which is generated by the moving area detector 6 using the signal from the CCD 11.

Next, a method of generating the moving area detection signal k will be described. Now, it is assumed that the moving area detection signal of the pixel (i, j) is generated in the nth frame of the output image signal V. The still image signal Vs output in this frame is shown in FIG.
As shown in FIG. 5, since it is the signal of the m-1th frame of the output signal of the CCD 12, that is, the signal of the n-4th to n-1th frame of the output image signal V, it is the signal of the n-4th to the nth frame. Pixels that have moved during a period must be treated as a moving area. Therefore, the pixel (i,
The moving area detection signal k _n (i, j) of j) is obtained from the equation (2).

[0023]

[Formula 2] k _n (i, j) = | V _n-3 (i, j) -V _n-4 (i, j) | + | V _n-2 (i, j) -V _n-3 ( i, j) | + | V _n-1 (i, j) -V _n-2 (i, j) | + | V _n (i, j) -V _n-1 (i, j) | …… ( 2) Here, V _n (i, j) represents the signal value of the pixel (i, j) in the nth frame. The still image signal Vs output in the next n + 1th frame is the m-1th frame signal of the output signal of the CCD 12, that is, the n-4th to n-1th frame signal of the output image signal V. Therefore, in this frame, the pixel in which the motion has occurred in the period from the (n−4) th frame to the (n + 1) th frame must be treated as a moving region. Therefore, the moving area detection signal k _{n + 1} (i, j) of the pixel (i, j) in this frame is obtained by the equation (3).

[0024]

[Equation 3] k _{n + 1} (i, j) = k _n (i, j) + | V _{n + 1} (i, j) -V _n (i, j) | …… (3) In the subsequent frames Similarly, the moving area detection signal is obtained.

FIG. 4 shows a schematic structure of the moving area detector 6.
In FIG. 4, FM1 to FM7 are frame memories and AD
D and ADDT are adders, ABS is a signal converter for converting a signal into an absolute value, and 61 is a signal interpolator. In the moving area detector 6 of this embodiment, the image signals sum up the inter-frame differences over the past seven frames at maximum, and generate the moving area detection signal K. Therefore, in order to obtain the difference between frames, the image signals of the past seven frames are stored in the frame memory memories FM1 to FM1.
The signal is stored in FM7 and the signal difference between adjacent frame memories is obtained by the adder ADD. The absolute value of this interframe difference is taken by the signal converter ABS and input to the adder ADDT. Here, the number of inter-frame differences required to generate the moving area detection signal K differs depending on the frame. That is, for example, in the nth frame, four frame differences from the nth frame to the n-4th frame are taken, but in the (n + 3) th frame, seven frame differences from the n + 3th frame to the n-4th frame are taken. Therefore, the input coefficient of the adder ADDT is switched for each frame. The input coefficients C1 to C7 for each frame are shown in FIG. Here, the frame is identified by CCD 11 and CC.
This is performed using the vertical synchronizing signals VD11 and VD12 of D12.

Using the moving area detection signal K calculated by the method described above, the still image signal Vs and the moving image signal Vm are combined according to the equation (1) to obtain the output image signal V. In the output image signal V, the moving area has HDTV resolution, but the still area can have four times the resolution of HDTV. Considering that the resolution of the human eye with respect to the moving area is lower than the resolution with respect to the still area, it is possible to visually capture a moving image equivalent to four times the resolution of HDTV by the image pickup apparatus of this embodiment. [Second Embodiment] A schematic configuration of a second embodiment of the present invention is shown in FIG.
Shown in. In FIG. 6, 11R, 11G, 11B, 12
R, 12G, 12B are CCDs, 4R, 4G, 4B are frame frequency converters, 5R, 5G, 5B are interpolation circuits, 6
R, 6G, 6B are moving area detectors, 7R, 7G, 7B are image synthesizers, 8 is an optical low-pass filter, DP1, DP
2 is a dichroic prism.

In this embodiment, a half mirror 3 is arranged behind the lens 2 and, further, dichroic prisms DP1 and DP2 are provided on the respective optical paths branched by the half mirror 3.
To place. With this arrangement, six subject images separated into RGB are captured by six CCDs. here,
The CCDs 11R and 12R capture a red (R) component subject image, the CCDs 11G and 12G capture a green (G) component subject image, and the CCDs 11B and 12B capture a blue (B) subject image.
A subject image of the component is captured. Also, 11R, 11G, 1
The CCD of 1B is intended for imaging a moving region, and the frame frequency is 30 Hz with 2 million pixels. On the other hand, 12
The CCD of the set of R, 12G, and 12B is for the purpose of capturing an image of a static region, and has the number of pixels of 8 million and the frame frequency of 7.5 Hz as in the first embodiment.

In this embodiment, the first embodiment is performed for each color of RGB.
Therefore, it is possible to capture a high-definition color moving image. [Third Embodiment] FIG. 7 shows a schematic configuration of a third embodiment of the present invention.
Shown in. In FIG. 7, 9G is an image inverter and DP is a dichroic prism.

Also in this embodiment, the dichroic prism is used to capture a high-definition color moving image. However, in this embodiment, the present invention is applied only to the green component image of the three color components to achieve high definition. This has high responsiveness up to the frequency range where the spatial frequency characteristics of the human eye are high only for the luminance component,
Therefore, it is based on the fact that the color image can be visually made fine by making only the green component close to the luminance component fine.

In order to apply the present invention to the green component image, in this embodiment, the dichroic prism DP having the structure shown in FIG. 7 is used.
To use. The dichroic prism DP splits the incident light into three color components of red, green and blue, and further splits the green component light into two. Therefore, it has four emission end faces, and the CCDs 11R, 11G, 11B are attached to these four end faces.
And 12G are attached. Here, CCD11R, 1
1G and 11B aim to image a moving region, and the frame frequency is set to 30 Hz with 2 million pixels. On the other hand, CCD12
G is for the purpose of imaging a still region, and has the number of pixels of 8 million and the frame frequency of 7.5 Hz as in the first embodiment.

Since the incident surface of the dichroic prism DP to which the CCD 11G is attached reflects the incident light an odd number of times and then exits, the image formed on the CCD is vertically inverted as compared with the images of the other end surfaces. ing. Therefore, the image is electrically inverted and corrected by the image inverter 9G, and the signal from the image inverter 9G and the signal from the CCD 12G are passed through a circuit similar to that of the first embodiment to enhance the green component image. Refine.

On the other hand, the signals of the red and blue component images from the CCDs 11R and 11B are output by the usual method.
However, in order to match the number of output pixels with the green component image, interpolation processing is performed by the interpolation circuits 5R and 5B, respectively.
Quadruple the number of pixels. [Fourth Embodiment] FIG. 8 shows a schematic configuration of a fourth embodiment of the present invention.
Shown in. In FIG. 8, CM1 and CM2 are CCD cameras. In this embodiment, the still area image and the moving area image are captured by different cameras. Two CCD cameras C
The half mirror 3 is arranged as shown in FIG. 8 so that M1 and CM2 can image the same area.

Using the output signals of the two CCD cameras CM1 and CM2, the same processing as in the above embodiment is performed to obtain a high definition image. Here, the CCD cameras CM1 and C used
When M2 is a three-plate type and outputs RGB, the circuit configuration is the second embodiment.
In the case of a single-plate type camera, the same as in the first embodiment.

It is needless to say that various modifications can be made without departing from the spirit of the present invention, merely showing the four embodiments of the present invention. For example, although the CCD is used as the image pickup device in the above-described embodiment, the invention can be carried out regardless of the type of the image pickup device.

Although the CCD camera is used in the fourth embodiment, it is obvious that the present invention can be implemented regardless of the type of the camera. Further, in the fourth embodiment, two CCDs are used.
The half mirror 3 is used so that the cameras CM1 and CM2 can image the same area. However, if a function of specifying the image pickup area is provided, a method of arranging two cameras in parallel without using the half mirror 3 is also possible. .

[0036]

As described above, according to the present invention, at least one or a set of moving area image pickup devices whose main purpose is to image a moving area, and the number of pixels of the moving area image pickup device is the number of pixels. And a set of at least one set of image sensors for a still region whose main purpose is to image a still region whose frame frequency is lower than the frame frequency of the image sensor for a moving region. Since it is divided into two and imaged by different image pickup devices, respectively, it is possible to capture an image with characteristics suitable for each region. That is, with respect to a still region that does not change with time, the frame frequency can be lowered to increase the number of pixels and a sufficiently high-definition image can be captured. In addition, a moving area can be captured at the same speed as conventional moving image capturing, and a natural moving image can be captured. As described above, the image pickup apparatus of the present invention can take a high-definition moving image without increasing the drive frequency of the image pickup element. Therefore, a high-definition moving image pickup device such as an HDTV can be realized at low cost. Furthermore, if the present invention is carried out using a high-resolution image pickup device for HDTV or the like as in the above-described embodiment, it is possible to pick up an ultra-high-definition moving image that has never been seen before.

[Brief description of drawings]

FIG. 1 is a schematic configuration diagram of a first embodiment of the present invention.

FIG. 2 is a schematic configuration diagram of a frame frequency converter of the first embodiment.

FIG. 3 is a diagram showing a temporal relationship between frames of each image sensor in the first embodiment.

FIG. 4 is a schematic configuration diagram of a moving area detector in the first embodiment.

FIG. 5 is a diagram showing coefficients by which an input signal of an adder used in the moving area detector of the first embodiment is multiplied.

FIG. 6 is a schematic configuration diagram of a second embodiment of the present invention.

FIG. 7 is a schematic configuration diagram of a third embodiment of the present invention.

FIG. 8 is a schematic configuration diagram of a fourth embodiment of the present invention.

[Explanation of symbols]

 2 lens 3 half mirror 4 frame frequency converter 4R frame frequency converter 4G frame frequency converter 4B frame frequency converter 5 interpolation circuit 6 moving area detector 7 image synthesizer 8 optical low pass filter 9G image inverter 11 CCD area sensor 12 CCD area sensor 41 frame memory 42 frame memory 61 signal interpolator ABS signal converter for converting signal to absolute value ADD adder ADDT adder FM frame memory CM1 CCD camera CM2 CCD camera

Claims (1)

[Claims] 1. At least one or a set of moving-region image pickup devices whose main purpose is to image a moving region, the number of pixels exceeds the number of pixels of the moving-region image pickup device, and the frame frequency is An image pickup apparatus comprising: at least one or a set of image pickup devices for a stationary area, the main purpose of which is to image a still region below the frame frequency of the image pickup device for a region.