JP4430840B2 - Imaging apparatus and imaging method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an imaging apparatus and an imaging method, and more particularly to an imaging apparatus and an imaging method that output videos having different frame frequencies.
[0002]
[Prior art]
In recent years, various video media such as movies and televisions have been developed along with improvement in performance of video equipment and the like. For example, a video of a certain video media is recorded in a unique format corresponding to the video media.
[0003]
By the way, in a movie as an example of video media, it has been said that high image quality cannot be maintained unless it is a film. However, with the progress of electronic equipment for movie equipment, film cameras with 24 frames (frames / seconds) based on high-definition technology are being developed for movie cameras, where film cameras were the mainstream. .
[0004]
Therefore, it is expected that the barrier between the television and the movie will be lowered and the sharing of materials and contents will become more and more active as the image quality of the television and the electronic equipment of the movie equipment are reduced. Moreover, it is expected that sharing of materials and contents will be advanced among various video media as video equipment and the like become more sophisticated, not limited to television and movies.
[0005]
[Problems to be solved by the invention]
However, in order to share the video distributed in a unique format corresponding to the video media, the difference in the frame frequency of each other is a major obstacle. For example, when video is shared between a television and a movie, the television frame frequency is 30 (frames / second) or 29.97 (frames / second), and the movie frame frequency is 24 (frames / second). It is.
[0006]
Therefore, in order to share video between video media having different frame frequencies, equipment such as a format converter for converting the frame frequency is required.
[0007]
Such a format converter is composed of, for example, several to several tens of frame memories and arithmetic devices, and performs sub-sampling processing or interpolation processing in the time axis region. In the sub-sampling process, the frame frequency of the video is converted from, for example, 30 (frame / second) to 24 (frame / second). Further, the interpolation processing is to convert the frame frequency of the video from, for example, 24 (frame / second) to 30 (frame / second).
[0008]
As described above, in order to share a video between video media having different frame frequencies, there is a problem that a large-scale equipment or device for converting the frame frequency is required.
[0009]
In addition, it takes time to convert the frame frequency, and it is very difficult to obtain one or more videos having different frame frequencies in real time.
[0010]
The present invention has been made in view of the above points, and can capture an image that can be shared by video media having different frame frequencies, and can easily output one or more videos having different frame frequencies. An object is to provide an apparatus and an imaging method.
[0011]
[Means for Solving the Problems]
Therefore, in order to solve the above-described problem, the present invention provides an image pickup apparatus that outputs a video signal corresponding to a subject, accumulates photoelectric conversion signals for each unit pixel, and stores one or more accumulated photoelectric change signals with different accumulation times. And a signal processing unit that generates and outputs one or more video signals having different frame frequencies from one or more video signals having different accumulation times, and includes one image sensor. The unit pixel circuit includes the above-described unit pixel circuit, and the unit pixel circuit reads from the photoelectric conversion unit, the first reading unit that reads out the photoelectric conversion signal from the photoelectric conversion unit, the reset unit that resets the photoelectric conversion unit, and the photoelectric conversion unit. Storage means for storing the photoelectric conversion signal, and second reading means for reading the photoelectric conversion signal stored in the storage means, wherein the first reading means The first reading unit and the second reading unit read the photoelectric conversion signal at a cycle longer than the cycle of resetting the photoelectric conversion unit, and read the photoelectric conversion signal at a cycle of resetting the photoelectric conversion unit. second reading means controlled by accumulating time and wherein also be output from the different one or more video signals.
[0018]
In such an imaging apparatus, one or more video signals having different frame frequencies are generated and output from one or more video signals having different accumulation times. Therefore, it is possible to capture a video that can be shared by video media having different frame frequencies, and it is possible to easily output one or more videos having different frame frequencies.
[0019]
DETAILED DESCRIPTION OF THE INVENTION
Next, embodiments of the present invention will be described with reference to the drawings.
[0020]
FIG. 1 is a configuration diagram of a first embodiment of an imaging apparatus according to the present invention. The imaging apparatus 1 in FIG. 1 is configured to include an imaging unit 10 and a signal processing unit 20. The imaging unit 10 converts the optical image of the subject into an electrical signal, and outputs the converted electrical signal as a video signal. The signal processing unit 20 converts the video signal supplied from the imaging unit 10 into a plurality of video signals having different frame frequencies and outputs the video signals. For example, the signal processing unit 20 in FIG. 1 outputs an A (frame / second) video signal 1 and a B (frame / second) video signal 2.
[0021]
The imaging unit 10 outputs a video signal having a frame frequency that is the least common multiple of the frame frequencies of the video signal 1 and the video signal 2 output from the signal processing unit 20. For example, if the frame frequency of the video signal output from the imaging unit 10 is L (frame / second), the frame frequency L (frame / second) of the video signal output from the imaging unit 10 is output from the signal processing unit 20. The frame frequency A (frame / second) of the video signal 1 and the frame frequency B (frame / second) of the video signal 2 have the relationship of the following formulas (1) to (3).
[0022]
L = A ′ × B ′ × G (1)
A = A 'x G (2)
B = B 'x G (3)
G is the greatest common divisor of A and B. If the relations of the above formulas (1) to (3) are satisfied, the signal processing unit 20 may process the video signal in units of frames as shown in FIG. 2, and the apparatus configuration can be simplified.
[0023]
FIG. 2 is a diagram illustrating an example of processing performed by the signal processing unit. FIG. 2 shows the case where A = 3, B = 2, and G = 1. When the video signal 1 of A (frame / second) and the video signal 2 of B (frame / second) are output from the signal processing unit 20, the imaging unit 10 performs the processing shown in FIG. 2 based on the equations (1) to (3). A video signal of 6 (frame / second) as shown in (A) is output.
[0024]
The signal processing unit 20 adds the video signals as shown in FIG. 2A in units of two frames, and generates one frame from the addition result, thereby generating 3 frames / second as shown in FIG. Output video 1 is output. Further, the signal processing unit 20 integrates the video signal as shown in FIG. 2A in units of 3 frames, and generates one frame from the result of the integration, thereby generating 2 (frame / second) as shown in FIG. ) Output video 2 is output.
[0025]
FIG. 3 shows a configuration diagram of an embodiment of the signal processing unit. The signal processing unit 20 of FIG. 3 is configured to include frame delay devices 21 to 23, adders 24 and 25, and switches 26 and 27.
[0026]
The video signal output from the imaging unit 10 is supplied to the frame delay units 21 and 22 and the adders 24 and 25. The frame delay unit 21 delays the video signal output from the imaging unit 10 by one frame period and outputs it to the adder 24. The frame delay unit 22 delays the video signal output from the imaging unit 10 by one frame period and outputs the delayed video signal to the adder 25 and the frame delay unit 23. The frame delay unit 23 delays the video signal output from the frame delay unit 22 by one frame period and outputs it to the adder 25.
[0027]
The adder 24 adds the video signal output from the imaging unit 10 and the video signal output from the frame delay unit 21, and outputs the added video signal as the output video signal 1 via the switch 26. The switch 26 performs a sub-sampling process. For example, in the case of FIG. 2, the output video signal 1 can be output every two frames by selecting and outputting even frames.
[0028]
The adder 25 adds the video signal output from the imaging unit 10, the video signal output from the frame delay unit 22, and the video signal output from the frame delay unit 23, and adds the added video signal. The output video signal 2 is output via the switch 27. The switch 27 performs sub-sampling processing. For example, in the case of FIG. 2, the output video signal 2 can be output every three frames by selecting and outputting a frame that is a multiple of three.
[0029]
FIG. 4 is a diagram illustrating an example of filtering processing in the signal processing unit. For example, when the signal processing unit 20 is provided with a filtering function in the time direction, it can be realized by installing a plurality of frame delay devices 21 to 23 similar to those in FIG.
[0030]
FIG. 4 shows the case of 7 taps. Also, k11 to k37 in FIG. 4 represent filtering coefficients. For example, the time axis filter characteristics can be varied by changing the filter coefficients k11 to k17 when the video as shown in FIG. 4A is added in units of 7 frames.
[0031]
As described above, the signal processing unit 20 is provided with a time axis filter characteristic, and by changing the time axis filter characteristic, it is possible to change the motion resolution and the smoothness of the motion of the video having a low frame frequency. The optimum video can be reproduced accordingly.
[0032]
FIG. 5 shows a block diagram of a second embodiment of the imaging apparatus of the present invention. The imaging device 2 in FIG. 5 is configured to include an imaging element 30 and a signal processing unit 40. The image sensor 30 converts an optical image of a subject into an electric signal, and outputs the converted electric signal from the output terminals 31 and 32 as a video signal.
[0033]
The signal processing unit 40 converts the video signals (signal 1 and signal 2) supplied from the output terminals 31 and 32 of the image sensor 30 into a plurality of video signals (output video signal 1 and output video signal 2) having different frame frequencies. Output. For example, the signal processing unit 40 is configured to include a subtracter 41, frame delay units 42 and 43, a switch 44, an adder 45, and a frame memory 46.
[0034]
The video signal (signal 1) supplied from the output terminal 31 of the image sensor 30 is output as the output video signal 1 and also supplied to the subtractor 41. The video signal (signal 2) supplied from the output terminal 32 of the image sensor 30 is supplied to the subtracter 41 and the adder 45.
[0035]
The subtractor 41 calculates the difference between the video signals supplied from the output terminals 31 and 32 and outputs the calculated difference signal to the frame delay unit 42. The frame delay unit 42 delays the difference signal supplied from the subtracter 41 by one frame period and outputs the delayed signal to the frame delay unit 43 and the adder 45.
[0036]
The frame delay unit 43 delays the differential signal (signal 3) supplied from the frame delay unit 42 by one frame period and outputs the differential signal delayed by one frame period to the switch 44. The switch 44 is open / close controlled in accordance with the control signal SW1, for example, selects the ground (GND) level when the control signal SW1 is at H level, and selects the output of the frame delay device 43 when the control signal SW1 is at L level. . The switch 44 converts the differential signal supplied from the frame delay unit 43 into a signal (signal 4) corresponding to the control signal SW1 and outputs the signal.
[0037]
The adder 45 adds the video signal (signal 2) output from the output terminal 32, the difference signal (signal 3) output from the frame delay device 42, and the signal (signal 4) output from the switch 44. The addition signal is output to the frame memory 46. The frame memory 46 has a write timing controlled according to the control signal W, and writes the addition signal output from the adder 45 according to the control signal W. The addition signal written in the frame memory 46 is output as the output video signal 2.
[0038]
FIG. 6 shows a configuration diagram of an embodiment of the image sensor. In the image sensor 30, unit pixels 35 are arranged in a grid pattern. Each unit pixel 35 is connected to the vertical scanning circuit 33 through readout pulse lines 36a and 36b and a reset pulse line 37, and is connected to horizontal scanning circuits 34a and 34b through signal readout pulse lines 38a and 38b. ing. Each unit pixel 35 has an XY address structure capable of nondestructive reading.
[0039]
Since the unit pixel 35 includes the horizontal scanning circuits 34 a and 34 b, two different signals 1 and 2 can be read from the unit pixel 35. Of the two different signals 1 and 2 read out from the unit pixel 35, one signal 1 is outputted from the output terminal 31 via the signal readout line 38a and the amplifier 39a, and the other signal 2 is outputted from the readout line 38b and the amplifier 39b. From the output terminal 32.
[0040]
FIG. 7 shows a configuration diagram of an example of a unit pixel. The unit pixel 35 includes a transistor Tr. 1-Tr. 7, a photodiode PD, a capacitor C that holds a nondestructive read signal, read pulse lines 36 a and 36 b, a reset pulse line 37, and signal read lines 38 a and 38 b.
[0041]
Transistor Tr. 1 amplifies the photoelectric conversion signal accumulated in the photodiode PD and outputs it to the signal readout line 38a. Transistor Tr. 2 has a gate terminal connected to the readout pulse line 36b, and in accordance with the readout pulse 2 supplied from the readout pulse line 36b, the transistor Tr. The operation of 1 is turned on / off.
[0042]
Transistor Tr. 3 has a gate terminal connected to the reset pulse line 37 and resets the charge accumulated in the photodiode PD. Transistor Tr. 4 reads the photoelectric conversion signal accumulated in the photodiode PD, and accumulates the read photoelectric conversion signal in the capacitor C. Note that the transistor Tr. 4 performs nondestructive reading from the photodiode PD.
[0043]
Transistor Tr. 5 has a gate terminal connected to the readout pulse line 36a, and in accordance with the readout pulse 1 supplied from the readout pulse line 36a, the transistor Tr. 4 is turned on / off. Transistor Tr. 6 amplifies the signal accumulated in the capacitor C and outputs it to the signal readout line 38b. Transistor Tr. 7 has a gate terminal connected to the read pulse line 36b and a transistor Tr. 6 is controlled by the read pulse 2 supplied from the read pulse line 36b.
[0044]
FIG. 8 shows a timing diagram of an example of an image sensor according to the present invention. When the frame frequency of the output video signal 1 and output video signal 2 output from the imaging device 2 is Fv1, Fv2 (Fv1> Fv2), it is necessary to obtain an output video signal whose accumulation time is Tv1, Tv2 (Tv1 <Tv2). There is.
[0045]
In order to obtain the output video signal 1 having the accumulation time Tv1 as shown in FIG. 8D, the period of the reset pulse for resetting the photodiode PD of each unit pixel 35 is Tv1 as shown in FIG. Then, the accumulation time Tv2 required for the output video signal 2 becomes a period (for example, (1) + (2)) longer than the reset pulse of FIG. 8E as shown in FIG. 8C.
[0046]
The photoelectric conversion signal in the period (1) is output from the output terminal 1 in accordance with FIG. Therefore, in order to obtain the photoelectric conversion signal of period (2), the read pulse 1 of FIG. 8 (F) is set to the H level after the period (2) has elapsed after the reset pulse of FIG. 8 (E) is output. .
[0047]
After the period {circle around (2)} has passed since the reset pulse of FIG. 8 (E) is output, the readout pulse 1 of FIG. 8 (F) is set to the H level, so that the photoelectric conversion signal of the period {circle around (2)} is transferred to the capacitor C. Accumulated. The signals accumulated in the photodiode PD and the capacitor C are read at a cycle of Tv1 in accordance with the read pulse 2 as shown in FIG. 8G and are output from the output terminals 31 and 32 of the image sensor 30.
[0048]
The same can be considered for the subsequent frames. By setting the readout pulse 1 as shown in FIG. 8F to the H level in the cycle of Tv2, an intermediate (eg, for example) necessary to obtain a signal of the accumulation time Tv2 from the photodiode PD operating in the cycle Tv1 of the reset pulse (for example, It is possible to obtain from the output terminal 32 photoelectric conversion signals of periods (2), (5), (7), etc.
[0049]
FIG. 9 shows a timing diagram of an example of a signal processing unit according to the present invention. Signal 1 in FIG. 9A is a signal output from the output terminal 31. Signal 2 in FIG. 9D is a signal output from the output terminal 32. The signal 3 in FIG. 9F is a signal obtained by delaying the difference signal between the signal 1 and the signal 2 by one frame period (Tv1). The signal 4 in FIG. 9H is a signal output via the switch 44 in which the signal 3 is delayed by one frame period (Tv1) and the signal delayed by one frame period (Tv1) is controlled by the control signal SW1. is there.
[0050]
The output video signal 1 is obtained by outputting the signal 1 in FIG. 9A as it is. On the other hand, the output video signal 2 is obtained by adding the signal 2, the signal 3 and the signal 4, writing the added signal in the frame memory 46 according to the control signal W, and outputting the written signal in the frame period Tv2. .
[0051]
As shown in FIG. 9J, the first frame of the output video signal 2 has a frame delay in the period of the second frame and the signal 2 of the accumulation period {circle around (2)} output from the output terminal 32 in the period of the second frame. The signal 3 of the storage time {circle over (1)} outputted from the device 42 is added, the signal of the added storage time ({circle around (1) + {2})) is written to the frame memory 46, and the written signal is time-axis converted. Generated by reading.
[0052]
The second 2 'field of the output video signal 2 is obtained by adding the signal 2 of the accumulation time (5), the signal 3 of the accumulation time (4), and the signal 4 of the accumulation time (3), and adding the accumulated accumulation time (▲ 3) + (4) + (5)) is written in the frame memory 46, and the written signal is read out following the first 'frame.
[0053]
Thus, the signal processing unit 40 converts the video signals (signal 1 and signal 2) supplied from the output terminals 31 and 32 of the image sensor 30 into a plurality of video signals (output video signal 1 and output video signal having different frame frequencies). It can be converted into 2) and output.
[0054]
In the present embodiment, the example of the frame period Tv1 <Tv2 <(Tv1 × 2) has been described. However, the video signals (signals 1 and 2) supplied from the output terminals 31 and 32 of the image sensor 30 are similarly considered. ) Can be converted into an output video signal 2 of various frame periods and output.
[0055]
【The invention's effect】
As described above, according to the present invention, a video that can be shared by video media having different frame frequencies can be taken, and one or more videos having different frame frequencies can be output easily and in real time.
[0056]
Further, according to the present invention, the motion resolution and smoothness of motion of a video with a low frame frequency can be varied, and an optimal video can be reproduced according to the frame frequency.
[0057]
In addition, according to the present invention, the video signal may be processed in units of frames, and the apparatus configuration can be simplified. For example, by setting the first frame frequency to 120 Hz, the second frame frequency to 30, and the third frame frequency to 24, it is possible to output a television image and a movie image simultaneously.
[0058]
[Brief description of the drawings]
FIG. 1 is a configuration diagram of a first embodiment of an imaging apparatus according to the present invention.
FIG. 2 is a diagram illustrating an example of processing in a signal processing unit.
FIG. 3 is a configuration diagram of an embodiment of a signal processing unit.
FIG. 4 is a diagram for explaining an example of filtering processing in a signal processing unit.
FIG. 5 is a configuration diagram of a second embodiment of the imaging apparatus according to the present invention.
FIG. 6 is a configuration diagram of an embodiment of an image sensor.
FIG. 7 is a configuration diagram of an example of a unit pixel.
FIG. 8 is a timing chart of an example of an image sensor according to the present invention.
FIG. 9 is a timing diagram of an example of a signal processing unit according to the present invention.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 Image pick-up device 10 Image pick-up part 20, 40 Signal processing part 21, 22, 23, 42, 43 Frame delay device 24, 25, 45 Adder 26, 27, 44 Switch 30 Image pick-up element 31, 32 Output terminal 33 Vertical scanning circuit 34a , 34b Horizontal scanning circuit 35 Unit pixel 41 Subtractor 46 Frame memory

Claims (2)

In an imaging device that outputs a video signal corresponding to a subject,
An image sensor that accumulates photoelectric conversion signals for each unit pixel and outputs the accumulated photoelectric change signals as one or more video signals having different accumulation times;
A signal processing unit that generates and outputs one or more video signals having different frame frequencies from one or more video signals having different accumulation times;
The image sensor is composed of one or more unit pixel circuits,
The unit pixel circuit is a photoelectric conversion means, a first reading means for reading a photoelectric conversion signal from the photoelectric conversion means, a reset means for resetting the photoelectric conversion means, and a storage means for accumulating the photoelectric conversion signals read from the photoelectric conversion means. , Comprising a second readout means for reading out the photoelectric conversion signal accumulated in the accumulation means,
The first reading unit reads the photoelectric conversion signal at a cycle for resetting the photoelectric conversion unit, and the second reading unit reads the photoelectric conversion signal at a cycle longer than the cycle for resetting the photoelectric conversion unit. the imaging apparatus characterized by said first reading means and storage time and controls the second reading means outputs a different one or more video signals. In an imaging method for outputting a video signal corresponding to a subject,
A first stage in which the image sensor accumulates a photoelectric conversion signal for each unit pixel, and outputs the accumulated photoelectric change signal as one or more video signals having different accumulation times;
A second stage in which the signal processing unit generates and outputs one or more video signals having different frame frequencies from one or more video signals having different accumulation times;
The first stage includes a photoelectric conversion means provided in one or more unit pixel circuits constituting the image sensor, a first reading means for reading a photoelectric conversion signal from the photoelectric conversion means, and a reset for resetting the photoelectric conversion means. Among the first reading means and the second reading means, the second reading means for reading the photoelectric conversion signals accumulated in the accumulating means, and the second reading means for reading the photoelectric conversion signals accumulated in the accumulating means . The first reading unit reads the photoelectric conversion signal at a cycle for resetting the photoelectric conversion unit, and the second reading unit reads the photoelectric conversion signal at a cycle longer than the cycle for resetting the photoelectric conversion unit. An imaging method comprising: outputting one or more video signals which are controlled to have different accumulation times.

Images