JP2003023569A - Imaging device and method therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging device and an imaging method, capable of video-recording images usable in common between video media, having different frame frequencies and capable of easily outputting one more numbers of video images, having different frame frequencies. SOLUTION: The device is provided with an imaging section 10 for converting an optical image of an object into an electrical signal to output a video signal of a first frame frequency according to the electrical signal, and a signal processing section 20 for converting the video signal of the first frame frequency into a signal of a second frame frequency and a signal of a third frame frequency to output the converted signals. The first frame frequency is a least common multiple of the second and the third frame frequencies.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup apparatus and an image pickup method, and more particularly to an image pickup apparatus and an image pickup method for outputting images having different frame frequencies.

[0002]

2. Description of the Related Art In recent years, various video media such as movies and televisions have been developed with the improvement in performance of video equipment. For example, the video of a certain video medium is recorded in a unique format corresponding to the video medium.

By the way, in movies as an example of video media, it has long been said that high quality images cannot be maintained unless they are films. However, with the progress of the electronic equipment of movie equipment, the film camera, which has been mainly used as a film camera, is developed with a 24 frame (frame / second) movie CCD camera based on high-definition technology. .

Therefore, it is expected that the barrier between television and movies will be lowered and the sharing of materials and contents will become more active due to the improvement in image quality of television and the electronic equipment of movie equipment. In addition to televisions and movies, it is expected that materials and contents will be shared among various video media as the performance of video equipment is improved.

[0005]

However, in order to share the images distributed in the unique format corresponding to the image media, the difference in frame frequency between them becomes a major obstacle. For example, when video is shared between a television and a movie, the frame frequency of the television is 3
0 (frames / second) or 29.97 (frames /
Second) and the frame frequency of the movie is 24 (frame /
Seconds).

Therefore, in order to share images between image media having different frame frequencies, equipment such as a format converter for converting the frame frequency is required.

Such a format converter is composed of, for example, several to several tens of frame memories and arithmetic units, and performs subsampling processing or interpolation processing in the time domain. The sub-sampling process converts the frame frequency of the video from, for example, 30 (frames / second) to 24 (frames / second). Further, the interpolation processing is performed by changing the frame frequency of the video from 24 (frame / second) to 3
It is converted to 0 (frame / second).

As described above, there is a problem that a large-scale equipment or device for converting the frame frequency is required in order to share the image between the image media having different frame frequencies.

Further, there is a problem that it takes time to convert the frame frequency, and it is very difficult to obtain one or more images having different frame frequencies in real time.

The present invention has been made in view of the above points, and it is possible to shoot an image that can be shared by image media having different frame frequencies, and to obtain images having different frame frequencies.
An object is to provide an imaging device and an imaging method capable of easily outputting one or more videos.

[0011]

In order to solve the above problems, the present invention provides an image pickup apparatus which outputs a video signal according to a subject, in which an optical image of the subject is converted into an electric signal and converted into the electric signal. An image pickup unit that outputs a video signal of a first frame frequency corresponding thereto; and a signal processing unit that converts the video signal of the first frame frequency into a video signal of a second frame frequency and a video signal of a third frame frequency and outputs the video signal. And the first frame frequency is the least common multiple of the second frame frequency and the third frame frequency.

In such an image pickup apparatus, since the first frame frequency is the least common multiple of the second frame frequency and the third frame frequency, it is possible to shoot an image that can be shared by image media having different frame frequencies. It is possible to easily and in real time output one or more images having different frequencies.

Further, the present invention is characterized in that the signal processing section performs a filtering process in the time direction on the video signal of the first frame frequency.

In such an image pickup apparatus, by performing filtering processing in the time direction, it is possible to change the motion resolution and the smoothness of motion of an image having a low frame frequency, and to reproduce an optimal image according to the frame frequency. can do.

Further, according to the present invention, the signal processing unit configures a predetermined number of video signals having a first frame frequency in accordance with the relationship between the first frame frequency and the second frame frequency or the third frame frequency. It is characterized in that one frame constituting a video signal of the second frame frequency or the third frame frequency is generated from the frame.

In such an image pickup device, since the first frame frequency is the least common multiple of the second frame frequency and the third frame frequency, it is sufficient to process the video signal in frame units, and the device structure can be simplified. For example, the first frame frequency is 120 Hz and the second frame frequency is 30H
z, by setting the third frame frequency to 24 Hz,
It is possible to output a television image and a movie image at the same time.

Further, according to the present invention, in an image pickup device for outputting a video signal according to a subject, photoelectric conversion signals for each unit pixel are accumulated, and the accumulated photoelectric change signals are used as one or more video signals having different accumulation times. An image pickup device for outputting and a signal processing unit for generating and outputting one or more video signals having different frame frequencies from one or more video signals having different storage times are provided.

In such an image pickup device, one or more video signals having different frame frequencies are generated and output from one or more video signals having different storage times. Therefore, it is possible to shoot a video that can be shared by video media having different frame frequencies, and it is possible to easily output one or more videos having different frame frequencies.

[0019]

BEST MODE FOR CARRYING OUT THE INVENTION Next, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram of the first embodiment of the image pickup apparatus of the present invention. The imaging device 1 of FIG. 1 is configured to include an imaging unit 10 and a signal processing unit 20. The imaging unit 10 converts the optical image of the subject into an electric signal and outputs the converted electric signal as a video signal. Further, the signal processing unit 20 converts the video signal supplied from the imaging unit 10 into a plurality of video signals having different frame frequencies and outputs the video signal. For example, the signal processing unit 20 in FIG. 1 outputs an A (frame / second) video signal 1 and a B (frame / second) video signal 2.

The image pickup section 10 outputs a video signal having a frame frequency which is the least common multiple of the frame frequencies of the video signals 1 and 2 output from the signal processing section 20. For example, when the frame frequency of the video signal output from the image pickup unit 10 is L (frames / second), the frame frequency L of the video signal output from the image pickup unit 10 (frames / second) is output from the signal processing unit 20. The frame frequency A (frame / second) of the video signal 1 and the frame frequency B (frame / second) of the video signal 2 have the relationships of the following expressions (1) to (3).

L = A ′ × B ′ × G (1) A = A ′ × G (2) B = B ′ × G (3) G is the greatest common divisor of A and B. If there is a relationship of the above equations (1) to (3), the signal processing unit 2
For 0, the video signal may be processed in frame units as shown in FIG. 2, and the device configuration can be simplified.

FIG. 2 is a diagram showing an example for explaining the processing in the signal processing section. Note that in FIG. 2, A = 3, B = 2, and G =
The case of 1 is shown. From the signal processing unit 20 to A
(Frame / sec) video signal 1 and B (frame / sec)
2 is output, the image pickup unit 10 outputs a video signal of 6 (frames / second) as shown in FIG. 2A based on the equations (1) to (3).

The signal processing unit 20 adds the video signals as shown in FIG. 2A in units of two frames and generates one frame from the result of addition, so that the three as shown in FIG.
Output the output video 1 of (frame / second). Further, the signal processing unit 20 integrates the video signal as shown in FIG. 2A in units of 3 frames and generates 1 frame from the integration result, so that 2 (frames / second) as shown in FIG. ) Output image 2 is output.

FIG. 3 is a block diagram of an embodiment of the signal processing section. The signal processing unit 20 of FIG.
23, adders 24 and 25, and switches 26 and 27.

The video signal output from the image pickup section 10 is supplied to the frame delay units 21 and 22 and the adders 24 and 25. The frame delay unit 21 delays the video signal output from the imaging unit 10 for one frame period and outputs the delayed video signal to the adder 24. In addition, the frame delay unit 22 delays the video signal output from the image pickup unit 10 for one frame period and outputs the delayed video signal to the adder 25 and the frame delay unit 23. The frame delay unit 23 delays the video signal output from the frame delay unit 22 for one frame period and outputs the delayed video signal to the adder 25.

The adder 24 adds the video signal output from the image pickup section 10 and the video signal output from the frame delay unit 21 and outputs the added video signal as the output video signal 1 via the switch 26. To do. The switch 26 performs sub-sampling processing. For example, in the case of FIG. 2, the output video signal 1 for every two frames can be output by selecting and outputting an even frame.

Further, the adder 25 adds the video signal output from the image pickup unit 10, the video signal output from the frame delay unit 22 and the video signal output from the frame delay unit 23, and adds them. The video signal is output as the output video signal 2 via the switch 27. The switch 27 performs sub-sampling processing. For example, in the case of FIG. 2, the output video signal 2 for every three frames can be output by selecting and outputting a frame that is a multiple of three.

FIG. 4 is a diagram for explaining an example of filtering processing in the signal processing section. For example, when the signal processing unit 20 has a filtering function in the time direction,
This can be realized by installing a plurality of frame delay units 21 to 23 similar to those in FIG.

FIG. 4 shows the case of 7 taps. Further, k11 to k37 in FIG. 4 represent filtering coefficients. For example, filter coefficients k11 to k17 when adding an image as shown in FIG.
The time axis filter characteristic can be changed by changing the.

By thus providing the signal processing unit 20 with the time-axis filter characteristic and varying the time-axis filter characteristic, it is possible to vary the motion resolution and the smoothness of motion of an image having a low frame frequency. It is possible to reproduce the optimum video according to the frame frequency.

FIG. 5 shows a block diagram of the second embodiment of the image pickup apparatus of the present invention. The imaging device 2 of FIG. 5 is configured to include an imaging element 30 and a signal processing unit 40. Image sensor 30
Converts the optical image of the subject into an electric signal, and outputs the converted electric signal as a video signal from the output terminals 31 and 32.

The signal processor 40 converts the video signals (signal 1 and signal 2) supplied from the output terminals 31 and 32 of the image pickup device 30 into a plurality of video signals (output video signal 1 and output video signal 2) having different frame frequencies. Converted to and output. For example, the signal processing unit 40 includes the subtractor 41 and the frame delay unit 4
2, 43, a switch 44, an adder 45, and a frame memory 46.

The video signal (signal 1) supplied from the output terminal 31 of the image pickup device 30 is output as the output video signal 1 and is also supplied to the subtractor 41. Also, a video signal (signal 2) supplied from the output terminal 32 of the image sensor 30.
Is supplied to the subtractor 41 and the adder 45.

The subtractor 41 calculates the difference between the video signals supplied from the output terminals 31 and 32, and outputs the calculated difference signal to the frame delay unit 42. The frame delay unit 42 delays the difference signal supplied from the subtractor 41 for one frame period and outputs it to the frame delay unit 43 and the adder 45.

The frame delay unit 43 delays the differential signal (signal 3) supplied from the frame delay unit 42 for one frame period and outputs the differential signal delayed for one frame period to the switch 44. The switch 44 is the control signal SW1
Open / close control is performed in accordance with the above, and for example, the ground (GND) level is selected when the control signal SW1 is at the H level, and the output of the frame delay unit 43 is selected when the control signal SW1 is at the L level. The switch 44 converts the differential signal supplied from the frame delay unit 43 into a signal (signal 4) corresponding to the control signal SW1 and outputs the signal.

The adder 45 receives the video signal (signal 2) output from the output terminal 32, the difference signal (signal 3) output from the frame delay device 42, and the signal (signal 4) output from the switch 44. Are added, and the addition signal is output to the frame memory 46. The writing timing of the frame memory 46 is controlled according to the control signal W, and the addition signal output from the adder 45 is written according to the control signal W. The addition signal written in the frame memory 46 is output as the output video signal 2.

FIG. 6 is a block diagram of an embodiment of the image pickup device. In the image sensor 30, the unit pixels 35 are arranged in a grid pattern. Each unit pixel 35 has a read pulse line 36.
a and 36b and the reset pulse line 37, the signal reading pulse line 3 is connected to the vertical scanning circuit 33.
It is connected to the horizontal scanning circuits 34a and 34b via 8a and 38b. Each unit pixel 35 has an XY address structure capable of nondestructive read.

Since the unit pixel 35 has the horizontal scanning circuits 34a and 34b, two different signals 1 and 2 can be read from the unit pixel 35. Two different signals 1 and 2 read out from the unit pixel 35 are one signal 1
Is output from the output terminal 31 via the signal read line 38a and the amplifier 39a, and the other signal 2 is read out from the output line 38.
It is output from the output terminal 32 via b and the amplifier 39b.

FIG. 7 is a block diagram showing an example of a unit pixel.
The unit pixel 35 includes a transistor Tr. 1-Tr. 7 and
A photodiode PD, a capacitor C for holding a nondestructive read signal, read pulse lines 36a and 36b,
Reset pulse line 37 and signal readout lines 38a, 38
and b.

Transistor Tr. 1 amplifies the photoelectric conversion signal accumulated in the photodiode PD and outputs it to the signal read line 38a. Transistor Tr. The gate terminal of the transistor Tr.2 is connected to the read pulse line 36b, and according to the read pulse 2 supplied from the read pulse line 36b, the transistor Tr. The operation of No. 1 is controlled on / off.

Transistor Tr. The gate terminal 3 is connected to the reset pulse line 37 and resets the electric charge accumulated in the photodiode PD. Transistor T
r. Reference numeral 4 reads the photoelectric conversion signal stored in the photodiode PD, and stores the read photoelectric conversion signal in the capacitor C. The transistor Tr. Reference numeral 4 is non-destructive reading from the photodiode PD.

Transistor Tr. 5, the gate terminal is connected to the read pulse line 36a, and the transistor Tr.5 corresponds to the read pulse 1 supplied from the read pulse line 36a. The operation of No. 4 is turned on / off. Transistor Tr. 6 amplifies the signal accumulated in the capacitor C and outputs it to the signal read line 38b. Transistor Tr. 7
Has a gate terminal connected to the read pulse line 36b, and the transistor Tr. The signal read timing of No. 6 is controlled by the read pulse 2 supplied from the read pulse line 36b.

FIG. 8 shows a timing chart of an example of the image pickup device according to the present invention. The frame frequencies of the output video signal 1 and the output video signal 2 output from the imaging device 2 are Fv1 and F
If v2 (Fv1> Fv2), the accumulation time is Tv1,
It is necessary to obtain an output video signal that satisfies Tv2 (Tv1 <Tv2).

The period of the reset pulse for resetting the photodiode PD of each unit pixel 35 in order to obtain the output video signal 1 of the accumulation time Tv1 as shown in FIG. 8D is shown in FIG.
Tv1 as in (E). Then, output video signal 2
The storage time Tv2 required for the calculation is as shown in FIG.
A period longer than the reset pulse of (E) (for example, +
).

The photoelectric conversion signal of the period is output from the output terminal 1 according to FIG. Therefore, in order to obtain the photoelectric conversion signal in the period, the read pulse 1 in FIG. 8F is set to the H level after the period has elapsed after the reset pulse in FIG. 8E is output.

The read pulse 1 in FIG. 8 (F) is set to the H level after a period has elapsed from the output of the reset pulse in FIG. 8 (E), whereby the photoelectric conversion signal in the period is accumulated in the capacitor C. . The signals accumulated in the photodiode PD and the capacitor C are read in a cycle of Tv1 according to the read pulse 2 as shown in FIG. 8G, and are output from the output terminals 31 and 32 of the image sensor 30.

The following frames can be similarly considered. The read pulse 1 as shown in FIG.
By setting to the H level in the cycle of v2, an intermediate (for example, period, etc.) photoelectric conversion signal necessary for obtaining the signal of the accumulation time Tv2 from the photodiode PD operating in the cycle Tv1 of the reset pulse is output terminal. It is possible to obtain from 32.

FIG. 9 shows a timing diagram of an example of the signal processing unit according to the present invention. Signal 1 in FIG. 9A is a signal output from the output terminal 31. Signal 2 in FIG. 9 (D)
Is a signal output from the output terminal 32. Figure 9
The signal 3 in (F) is a signal obtained by delaying the difference signal between the signal 1 and the signal 2 by one frame period (Tv1). Figure 9
In the signal 4 of (H), the signal 3 is in one frame period (Tv1).
The signal delayed and delayed by one frame period (Tv1) is a signal output via the switch 44 controlled by the control signal SW1.

The output video signal 1 is the same as the signal 1 of FIG. 9A. On the other hand, output video signal 2
Adds the signal 2, the signal 3, and the signal 4, writes the added signal to the frame memory 46 according to the control signal W,
The written signal is output in the frame period Tv2.

As shown in FIG. 9 (J), in the 1'th frame of the output video signal 2, the signal 2 of the accumulation period output from the output terminal 32 in the period of the second frame and the frame delay in the period of the second frame. It is generated by adding the storage time signal 3 output from the device 42, writing the added storage time (+) signal to the frame memory 46, and converting the written signal by time-axis conversion and reading.

In the second 'field of the output video signal 2, the accumulation time signal 2, the accumulation time signal 3 and the accumulation time signal 4 are added, and the added accumulation time (++
) Is written in the frame memory 46, and the written signal is read out after the 1'th frame.

As described above, the signal processing section 40 converts the video signals (signal 1 and signal 2) supplied from the output terminals 31 and 32 of the image pickup device 30 into a plurality of video signals (output video signal 1, It can be converted into an output video signal 2) and output.

In this embodiment, the frame period Tv1
Although the example of <Tv2 <(Tv1 × 2) has been described, the output terminals 31 and 32 of the image sensor 30 are based on a similar idea.
It is possible to convert the video signal (signal 1, signal 2) supplied from the device into the output video signal 2 of various frame periods and output.

[0055]

As described above, according to the present invention, it is possible to shoot an image that can be shared by image media having different frame frequencies, and to easily and in real time output one or more images having different frame frequencies. It is possible.

Further, according to the present invention, it is possible to change the motion resolution and the smoothness of motion of an image having a low frame frequency, and it is possible to reproduce an optimal image according to the frame frequency.

Further, according to the present invention, the video signal only needs to be processed in frame units, and the device configuration can be simplified. For example, by setting the first frame frequency to 120 Hz, the second frame frequency to 30, and the third frame frequency to 24, it is possible to output a television image and a movie image at the same time.

[0058]

[Brief description of drawings]

FIG. 1 is a configuration diagram of a first embodiment of an image pickup apparatus of the present invention.

FIG. 2 is a diagram illustrating an example of processing performed by a signal processing unit.

FIG. 3 is a configuration diagram of an embodiment of a signal processing unit.

FIG. 4 is a diagram illustrating an example of filtering processing in a signal processing unit.

FIG. 5 is a configuration diagram of a second embodiment of the image pickup apparatus of the present invention.

FIG. 6 is a configuration diagram of an embodiment of an image sensor.

FIG. 7 is a configuration diagram of an example of a unit pixel.

FIG. 8 is a timing diagram of an example of an image sensor according to the present invention.

FIG. 9 is a timing diagram of an example of a signal processing unit according to the present invention.

[Explanation of symbols]

1 Imaging device 10 Imaging unit 20, 40 Signal processor 21,22,23,42,43 Frame delay device 24,25,45 adder 26, 27, 44 switch 30 image sensor 31, 32 output terminals 33 Vertical scanning circuit 34a, 34b Horizontal scanning circuit 35 unit pixels 41 Subtractor 46 frame memory

   ─────────────────────────────────────────────────── ─── Continued front page    (72) Inventor Fumio Okano             1-10-11 Kinuta, Setagaya-ku, Tokyo, Japan             Broadcasting Association Broadcast Technology Institute F-term (reference) 5C022 AB17 AB31 AC42 AC69                 5C024 BX05 CY06 GZ44 HX02 HX30                       HX50 JX35

Claims (8)

[Claims] 1. An imaging device that outputs a video signal according to a subject, the imaging unit converting an optical image of the subject into an electrical signal and outputting a video signal of a first frame frequency according to the electrical signal, A signal processing unit for converting and outputting the video signal of the first frame frequency into a video signal of the second frame frequency and a video signal of the third frame frequency, and outputting the first frame frequency to the second frame frequency and the third frame frequency.
An imaging device having a least common multiple of a frame frequency. 2. The image pickup apparatus according to claim 1, wherein the signal processing unit performs temporal filtering processing on the video signal of the first frame frequency. 3. The signal processing unit selects from a predetermined number of frames constituting a video signal of a first frame frequency to a second frame in accordance with the relationship between the first frame frequency and the second frame frequency or the third frame frequency. The image pickup apparatus according to claim 1 or 2, wherein one frame constituting a video signal having a frame frequency or a third frame frequency is generated. 4. An image pickup device for outputting a video signal according to a subject, wherein the photoelectric conversion signal for each unit pixel is accumulated, and the accumulated photoelectric change signal is output as one or more video signals having different accumulation times. And a signal processing unit that generates and outputs one or more video signals having different frame frequencies from one or more video signals having different storage times. 5. The image pickup device includes one or more unit pixel circuits, and the unit pixel circuit includes a photoelectric conversion unit, a first readout unit for reading out a photoelectric conversion signal from the photoelectric conversion unit, and the photoelectric conversion unit. Reset means for resetting, a storage means for storing photoelectric conversion signals read from the photoelectric conversion means, and a second reading means for reading the photoelectric conversion signals stored in the storage means, the first reading means and the second reading means. The image pickup apparatus according to claim 4, wherein the reading means is controlled to output one or more video signals having different storage times. 6. The first reading means reads the photoelectric conversion signal at a cycle of resetting the photoelectric conversion means, and the second reading means makes a photoelectric conversion at a cycle longer than a cycle of resetting the photoelectric conversion means. The image pickup apparatus according to claim 5, wherein a signal is read out. 7. An image pickup method for outputting a video signal according to a subject, the method comprising: converting an optical image of the subject into an electric signal and outputting a video signal of a first frame frequency according to the electric signal; Converting a video signal of one frame frequency into a video signal of a second frame frequency and a video signal of a third frame frequency and outputting the video signal, wherein the first frame frequency is the second frame frequency and the third frame frequency.
An imaging method, which is a least common multiple of a frame frequency. 8. An image pickup method for outputting a video signal according to a subject, wherein photoelectric conversion signals for each unit pixel are accumulated, and the accumulated photoelectric change signals are output as one or more video signals having different accumulation times. And, generating and outputting one or more video signals having different frame frequencies from one or more video signals having different accumulation times.