JP5331573B2 - Imaging apparatus and imaging method - Google Patents

Description

  The present invention relates to an imaging apparatus and an imaging method for imaging a subject by multiple exposure.

  When imaging the trajectory of a moving subject intermittently, or when imaging the form of a rotating subject at a specific phase, a conventional imaging device does not capture the subject with the shutter open. The subject's trajectory and the form at the time of rotation were imaged by projecting flash light with a period of λ and performing multiple exposure of reflected light on an image sensor (hereinafter referred to as “light receiving element”) (for example, Patent Document 1). reference).

JP 2006-180111 A

  However, since conventional imaging devices capture images with the shutter open, in order to expose only intermittent reflected light from the subject, the subject needs to be placed in a dark room. It was. For this reason, for example, in the outdoors where there is ambient light such as the sun, the locus of the moving subject is all exposed, so that the conventional imaging apparatus cannot expose only the reflected light of the flash. In addition, when imaging a rotating subject, if there is jitter in the rotation period, even if the imaging device performs multiple exposure with a fixed period equal to the rotation period, the rotation of the object image does not appear stationary and blurs. There is also a problem that a subject image is captured.

  The present invention has been made in view of the above-described points, and intermittently images the locus of a moving subject by multiple exposure even in the presence of ambient light, or at a specific phase of a rotating subject. An object of the present invention is to provide an imaging device capable of imaging a form.

The present invention has been made in order to solve the above-described problem, and includes a photoelectric conversion unit (for example, the photoelectric conversion unit 124 of the embodiment) that outputs charges according to an exposure amount, and a charge storage that stores the charges. Unit (for example, the charge storage unit 118 of the embodiment), the drain electrode for discarding the charge of the photoelectric conversion unit (for example, the drain electrode 114 of the embodiment), the potential of the photoelectric conversion unit and the charge storage unit A light-receiving element (for example, the light-receiving element 110 of the embodiment) including a plurality of pixels (for example, the pixel 111 of the embodiment) having a reset electrode (for example, the reset electrode 122 of the embodiment) to be initialized; , every time the synchronizing signal in a period shorter than the image frame period is output, after reading the first charge accumulating the charges in the charge storage part, the said drain electrode Discards the charge of the photoelectric conversion unit, when the image frame period has elapsed, the charge reading section for reading the charges accumulated in the charge storage portion as the second charge (e.g., charge reading unit embodiment 130 ) And a control unit (for example, the control unit 140 of the embodiment) that outputs the synchronization signal having a predetermined cycle to the charge reading unit, and the control unit has the first charge equal to or higher than a predetermined threshold value. In some cases, the charge reading unit is controlled to reduce the charge accumulation time according to the synchronization signal, and when the first charge is equal to or less than a predetermined threshold, the charge reading unit is configured to increase the charge accumulation time according to the synchronization signal. an imaging apparatus characterized that you control (e.g., the imaging apparatus 100 of the embodiment).

The present invention, the control unit is an imaging apparatus characterized by generating the synchronization signals to detect the movement of the Utsushitai, synchronized with the movement of the subject based on the detection result.

  Further, the present invention is the imaging apparatus, wherein the control unit generates the synchronization signal having a predetermined period that is not synchronized with an image frame period inside the imaging apparatus.

  Further, the present invention is the imaging apparatus, wherein the control unit outputs the synchronization signal to the charge reading unit so as to reduce a charge accumulation time when the charge is equal to or greater than a predetermined threshold.

  Further, the present invention is the imaging apparatus, wherein the control unit outputs the synchronization signal to the charge reading unit so as to increase a charge accumulation time when the charge is equal to or less than a predetermined threshold value.

The present invention is also an imaging method in the imaging apparatus (for example, the imaging apparatus 100 of the embodiment), in which the photoelectric conversion unit (for example, the photoelectric conversion unit 124 of the embodiment) outputs a charge corresponding to the exposure amount. A charge storage unit (for example, the charge storage unit 118 of the embodiment) stores the charge, and a drain electrode (for example, the drain electrode 114 of the embodiment) discards the charge of the photoelectric conversion unit. A step, a reset electrode (for example, the reset electrode 122 of the embodiment) initializes potentials of the photoelectric conversion unit and the charge storage unit, and a charge reading unit (for example, the charge reading unit 130 of the embodiment). after the initialization, each time the synchronization signal in a shorter period than the image frame period is output, after reading the first charge accumulating the charges in the charge storage section It discards the charge of the photoelectric conversion unit to the drain electrode, when the image frame period has elapsed, a step of reading the charge stored in the charge storage portion as the second charge, the control unit (e.g., The control unit 140) of the embodiment outputs the synchronization signal having a cycle shorter than the image frame cycle to the charge reading unit, and when the first charge is equal to or greater than a predetermined threshold, the charge accumulation time is set by the synchronization signal. Controlling the charge reading unit to decrease, and controlling the charge reading unit to increase a charge accumulation time according to the synchronization signal when the first charge is equal to or less than a predetermined threshold value. This is an imaging method.

According to the first to fourth aspects of the present invention, since the imaging apparatus performs exposure with a predetermined cycle a plurality of times per image frame, one image of a subject in which a rotating subject appears to be stationary is displayed. Can get into the frame. For this reason, the imaging device can capture the form of the rotating subject in a specific phase even outdoors with ambient light without increasing the number of image frames more than necessary.

In addition, according to the first to fourth aspects of the present invention, the imaging apparatus performs exposure of a predetermined cycle a plurality of times per image frame, so that a plurality of subject images representing the locus of the subject are obtained as one image. Can get into the frame. For this reason, the imaging apparatus can capture an intermittent trajectory of a moving subject even outdoors where there is ambient light without increasing the number of image frames more than necessary.

According to the first to fourth aspects of the present invention, the imaging apparatus increases or decreases the charge accumulation time per image frame according to the amount of charge output from the light receiving element. The subject can be imaged by adjusting the exposure amount in a short time while sufficiently acquiring the charge amount (accumulated potential) output from the element.

It is a block diagram of the imaging device in one embodiment of the present invention. It is a block diagram of a pixel in one embodiment of the present invention. 3 is an operation timing chart of the imaging apparatus according to the embodiment of the present invention. It is a figure which shows the to-be-photographed image image | photographed by the image frame in one Embodiment of this invention. It is a figure which shows the example of a timing of the image frame period and electric charge accumulation period in one Embodiment of this invention.

  An embodiment for carrying out the present invention will be described. FIG. 1 is a block diagram of an imaging apparatus according to an embodiment of the present invention. The imaging apparatus 100 includes a light receiving element 110, a charge reading unit 130, and a control unit 140.

  The subject 400 is a subject imaged by the imaging apparatus 100 and includes a rotation shaft 405 and blades 401 to 404 attached to the rotation shaft 405. It is assumed that the blades 401 to 404 rotate at a predetermined cycle about the rotation shaft 405. This configuration of the subject 400 is an example. For example, the subject 400 may be a moving sphere as will be described later with reference to FIG. The detector 500 is, for example, a photointerrupter. When any of the blades passes through the detector 500, the detector 500 outputs a “pass signal” indicating that the blade has passed to the control unit 140.

  The optical system 200 passes a light beam incident from the subject 400 and projects a subject image on the light receiving element (image sensor) 110. The optical system 200 includes a lens (not shown), and moves the lens (not shown) according to a focus adjustment instruction input from the control unit 140 of the imaging apparatus 100. The optical system 200 includes a diaphragm (not shown), and adjusts the diaphragm according to a diaphragm adjustment instruction input from the control unit 140.

  FIG. 2 is a block diagram of a pixel in an embodiment of the present invention. A plurality of pixels 111 are arranged in a matrix on the light receiving element 110 in FIG. The pixel 111 includes a photoelectric conversion unit 124, a drain gate 112, a drain gate control terminal 113, a drain electrode 114, a drain electrode output terminal 115, a charge storage gate 116, a charge storage gate control terminal 117, and a charge storage. Unit 118, charge storage unit output terminal 119, reset gate 120, reset gate control terminal 121, reset electrode 122, and reset electrode output terminal 123.

  The photoelectric conversion unit 124 outputs a charge corresponding to the exposure amount to the drain gate 112 or the charge accumulation gate 116. The drain gate control terminal 113 inputs the drain gate instruction output from the charge reading unit 130 of FIG. Similarly, the drain gate control terminal 113 receives a reset instruction for initializing the potential from the charge reading unit 130 and outputs the reset instruction to the drain gate 112.

  When either the drain gate instruction or the reset instruction is input, the drain gate 112 outputs the charge of the photoelectric conversion unit 124 to the drain electrode 114. When neither the drain gate instruction nor the reset instruction is input, the drain gate 112 does not output the charge of the photoelectric conversion unit 124 to the drain electrode 114. The drain electrode 114 outputs the charge input from the drain gate 112 to the drain electrode output terminal 115. The charges output to the drain electrode output terminal 115 are discarded.

  The charge storage gate control terminal 117 outputs the charge storage gate instruction input from the charge reading unit 130 of FIG. Similarly, the charge storage gate control terminal 117 outputs a reset instruction input from the charge reading unit 130 to the charge storage gate 116. When either the charge accumulation gate instruction or the reset instruction is input, the charge accumulation gate 116 outputs the charge of the photoelectric conversion unit 124 to the charge accumulation unit 118. When neither the charge accumulation gate instruction nor the reset instruction is input, the charge accumulation gate 116 does not output the charge of the photoelectric conversion unit 124 to the charge accumulation unit 118.

  The charge storage unit 118 stores the charge input from the charge storage gate 116. When the charge storage unit output terminal 119 is opened by the charge reading unit 130 of FIG. 1, the charge storage unit 118 has a potential corresponding to the amount of charge stored in the charge storage unit 118 (hereinafter referred to as “storage potential”). Is output to the charge readout unit 130 via the charge storage unit output terminal 119. In particular, a potential output from the charge storage unit 118 in a state initialized (reset) to a predetermined potential by the reset electrode 122 is hereinafter referred to as a “reset potential”. 1 outputs the reset potential and the accumulated potential due to the charge accumulated in the charge accumulation unit 118 to the charge reading unit 130.

  The reset gate control terminal 121 inputs a reset instruction output from the charge reading unit 130 of FIG. When the reset instruction is input, the reset gate 120 outputs the charge of the charge storage unit 118 to the reset electrode 122. When the reset instruction is not input, the reset gate 120 does not output the charge stored in the charge storage unit 118 to the reset electrode 122.

  The reset electrode 122 is at a power supply potential, for example, and outputs the charge input from the reset gate 120 to the reset electrode output terminal 123. The charges output to the reset electrode output terminal 123 are discarded.

  A passing signal indicating that the blade has passed the detector 500 is input to the control unit 140. The control unit 140 counts the number of times of passing signal input. Since the subject 400 includes four blades, the control unit 140 changes the synchronization signal from a low level to a high level (up edge) and outputs it to the charge reading unit 130 every time the passage signal is input four times.

  Further, the accumulated potential (charge amount) output from the light receiving element 110 is input from the charge reading unit 130 to the control unit 140. When the accumulated potential is equal to or greater than a predetermined threshold, the control unit 140 determines that the exposure amount is sufficient, reduces the predetermined high level time (charge accumulation time) of the synchronization signal, and outputs the same to the charge reading unit 130. To do. Further, when the accumulated potential is equal to or less than a predetermined threshold, the control unit 140 determines that the exposure amount is not sufficient, increases the predetermined high level time (charge accumulation time) of the synchronization signal, and outputs the same to the charge reading unit 130. To do.

  When the high level time of the synchronization signal ends, the control unit 140 changes the synchronization signal from the high level to the low level (down edge), and outputs it to the charge reading unit 130. Therefore, for example, the synchronization signal becomes an up edge immediately after the blade 403 passes the detector 500, and when the high level time of the synchronization signal ends, a down edge is output to the charge reading unit 130. The control unit 140 outputs a pixel signal output instruction to the charge reading unit 130 at a predetermined one frame period (for example, 16.67 [ms]). The image processing apparatus 300 performs predetermined image processing based on the pixel signal input from the charge reading unit 130 and generates an image frame.

  The accumulated potential is input from the light receiving element 110 to the charge reading unit 130. In addition, a synchronization signal and a pixel signal output instruction are input from the control unit 140 to the charge reading unit 130. When the pixel signal output instruction is input, the charge reading unit 130 opens the charge storage unit output terminal 119 and acquires the storage potential. The charge reading unit 130 performs correlated double sampling using the accumulated potential and the reset potential. The charge reading unit 130 performs analog-digital (AD) conversion on the amount of charge acquired by correlated double sampling, and outputs the result to the image processing apparatus 300 as a pixel signal. One image frame is generated by the image processing apparatus 300 based on the data output from the light receiving element 110 every predetermined one frame period.

  Further, when the output of the pixel signal is completed, the charge reading unit 130 outputs a reset instruction to the charge accumulation gate control terminal 117, the drain gate control terminal 113, and the reset gate control terminal 121. As a result, the reset potential is input from the light receiving element 110 to the charge reading unit 130.

  FIG. 3 is an operation timing chart of the imaging apparatus 100 according to the embodiment of the present invention. The charge reading unit 130 operates as follows based on the synchronization signal so that the photoelectric conversion unit 124 of FIG. 2 accumulates charges. Here, step S1 to step S5 in FIG. 3 is one frame cycle, and the imaging apparatus 100 repeats this frame cycle.

  First, the optical system 200 passes a light beam incident from the subject 400 and projects a subject image on the light receiving element 110. Thereby, the photoelectric conversion unit 124 is exposed. When the blade passes through the detector 500, the control unit 140 acquires a passage signal. As a result, the synchronization signal is input from the control unit 140 to the charge reading unit 130.

  The charge reading unit 130 outputs a reset instruction to the charge accumulation gate control terminal 117, the drain gate control terminal 113, and the reset gate control terminal 121. As a result, the charges accumulated in the photoelectric conversion unit 124 are discarded from the drain electrode 114 and the reset electrode 122, and the potential of the photoelectric conversion unit 124 is initialized (reset). The charges accumulated in the charge accumulation unit 118 are discarded from the reset electrode 122, and the potential of the charge accumulation unit 118 is initialized (reset) (step S1).

  Next, the charge reading unit 130 inputs the drain gate instruction to the drain gate 112 and further stops the output of the reset instruction. Since the output of the reset instruction is stopped, the potential of the reset gate 120 and the potential of the charge accumulation gate 116 are increased, and no charge is output from the charge accumulation unit 118 to the reset gate 120 or the charge accumulation gate 116. Further, the charge reading unit 130 opens the charge storage unit output terminal 119. As a result, the charge storage unit 118 outputs a reset potential to the charge reading unit 130 via the charge storage unit output terminal 119 (step S2).

  Next, it is assumed that the synchronization signal input from the control unit 140 becomes a high level (up edge) because the blade has passed the detector 500 four times. The charge reading unit 130 inputs a charge storage gate instruction to the charge storage gate control terminal 117 while the synchronization signal is at a high level. The photoelectric conversion unit 124 outputs a charge corresponding to the exposure amount to the charge accumulation gate 116. As a result, the charge accumulation gate 116 outputs the charge of the photoelectric conversion unit 124 to the charge accumulation unit 118. The charge storage unit 118 stores the charge input from the charge storage gate 116. Next, it is assumed that the synchronization signal input from the control unit 140 becomes a low level (down edge) because the charge accumulation time has elapsed. As a result, the charge reading unit 130 stops outputting the charge storage gate instruction (step S3).

  Further, charge reading unit 130 inputs a drain gate instruction to drain gate 112. Accordingly, the drain gate 112 outputs the charge of the photoelectric conversion unit 124 to the drain electrode 114. The drain electrode 114 outputs the charge input from the drain gate 112 to the drain electrode output terminal 115. The charges output to the drain electrode output terminal 115 are discarded.

  When the blade again passes the detector 500 four times, the control unit 140 sets the synchronization signal to a high level (up edge). Therefore, charge reading unit 130 stops outputting the drain gate instruction. Further, the control unit 140 determines whether or not a predetermined one frame period has been reached since the reset in step S1. If the cycle is not one frame, the control unit 140 and the charge reading unit 130 return to step S3. When the period of one frame is reached, the control unit 140 and the charge reading unit 130 proceed to step S5 (step S4).

  The control unit 140 measures a predetermined one frame period, and it is assumed that one frame period has elapsed since the previous pixel signal output instruction. For this reason, the control unit 140 outputs a pixel signal output instruction to the charge reading unit 130. The charge readout unit 130 opens the charge storage unit output terminal 119 because the pixel signal output instruction is input. The charge storage unit 118 outputs the stored potential to the charge reading unit 130 via the charge storage unit output terminal 119. In this way, the light receiving element 110 in FIG. 1 outputs the accumulated potential due to the charges accumulated in the charge accumulating unit 118 to the charge reading unit 130. The charge reading unit 130 performs correlated double sampling using the accumulated potential and the reset potential acquired in step S2. The charge reading unit 130 AD-converts the amount of charge acquired by correlated double sampling and outputs it to the image processing apparatus 300 as a pixel signal. The image processing device 300 performs predetermined image processing based on the pixel signal input from the charge reading unit 130, and generates an image frame (step S5).

  FIG. 4 is a diagram showing a subject image photographed in an image frame in one embodiment of the present invention. When the rotating subject 400 shown in FIG. 1 is imaged, a stationary subject image 400 is output as a pixel signal from the light receiving element 110 by being exposed at a timing synchronized with the rotation as shown in FIG. . The image processing apparatus 300 performs predetermined image processing based on the pixel signal input from the charge reading unit 130 and generates an image frame 406.

  In this way, the imaging apparatus 100 outputs from the light receiving element 110 a subject image that has been exposed multiple times per image frame in synchronization with the movement of the subject 400. Therefore, the image processing apparatus 300 can obtain the subject image in one image frame 406. The imaging apparatus 100 can capture the form of a rotating subject in a specific phase even outdoors where there is ambient light, without increasing the number of image frames more than necessary.

  In addition, since the imaging apparatus 100 increases or decreases the charge accumulation time per image frame according to the amount of charge output from the light receiving element 110, the amount of charge (accumulated potential) output from the light receiving element 110 is increased. The subject 400 can be imaged by adjusting the exposure amount in a short time while sufficiently acquiring.

  In addition, the control unit 140 may generate a synchronization signal having a predetermined period inside the imaging apparatus 100 without using the passing signal. Here, the synchronization signal may not be synchronized with the image frame period (asynchronous). FIG. 5 is a diagram showing an example of the timing of the image frame period and the charge accumulation period in one embodiment of the present invention. For example, as shown in FIG. 5, when a sphere moving at a predetermined speed (subject images 413a to 413h) is exposed (charge accumulation) in the charge accumulation periods 414a to 414h, the control unit 140 performs the image frame period. The subject images 413a to 413h that are exposed in synchronization with the synchronization signal generated inside the imaging apparatus 100 asynchronously with the image sensor 100 are output from the light receiving element 110 as pixel signals. The image processing apparatus 300 performs predetermined image processing based on these pixel signals input from the charge reading unit 130 and generates image frames 410 to 412.

  As a result, since the imaging apparatus 100 performs exposure synchronized with the movement of the subject 413 a plurality of times per image frame, a plurality of subject images representing the trajectory of the subject 413, for example, the subject image 413a to the subject image 413c are respectively associated The charge accumulation periods 414a to 414c can be obtained in one image frame 410. For this reason, the imaging apparatus 100 can capture an intermittent trajectory of a moving subject even outdoors with ambient light without increasing the number of image frames more than necessary.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

  The imaging device according to the present invention corresponds to the imaging device 100, the photoelectric conversion unit corresponds to the photoelectric conversion unit 124, the charge storage unit corresponds to the charge storage unit 118, and the drain electrode corresponds to the drain electrode. 114, the reset electrode corresponds to the reset electrode 122, the pixel corresponds to the pixel 111, the light receiving element corresponds to the light receiving element 110, the charge reading unit corresponds to the charge reading unit 130, and is controlled. The unit corresponds to the control unit 140.

  3 is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into the computer system and executed, thereby executing an execution process. May be. Here, the “computer system” may include hardware such as an OS (Operating System) and peripheral devices.

  Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used. The “computer-readable recording medium” means a flexible disk, a magneto-optical disk, a ROM, a writable nonvolatile memory such as a flash memory, a portable medium such as a CD-ROM, a hard disk built in a computer system, etc. This is a storage device.

Further, the “computer-readable recording medium” refers to a volatile memory (for example, DRAM (Dynamic) in a computer system serving as a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. Random Access Memory)) that holds a program for a certain period of time is also included.
The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.
The program may be for realizing a part of the functions described above.
Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.

  The present invention is suitable for an imaging apparatus that images a subject by multiple exposure.

DESCRIPTION OF SYMBOLS 100 ... Imaging device (imaging device) 110 ... Light receiving element (light receiving element) 111 ... Pixel 112 ... Drain gate 113 ... Drain gate control terminal 114 ... Drain electrode (drain electrode) 115 ... Drain electrode output terminal 116 ... Charge storage gate 117 ... Charge storage gate control terminal 118 ... Charge storage unit (charge storage unit) 119 ... Charge storage unit output terminal 120 ... Reset gate 121 ... Reset control terminal 122 ... Reset electrode (reset electrode) 123 ... Reset electrode output terminal 124 ... Photoelectric conversion unit (Photoelectric conversion unit) 130 ... Charge reading unit (charge reading unit) 140 ... Control unit (control unit) 200 ... Optical system 300 ... Image processing device 400 ... Subject 401 to 404 ... Blade 405 ... Rotating shaft 406 ... From image frame 410 412 ... Image frame 4 414h from the 413 h ... subject image 414a 3a ... charge accumulation period 500 ... detector

Claims (4)

A photoelectric conversion unit that outputs a charge corresponding to the exposure amount, a charge accumulating section for accumulating the charge, a drain electrode discarding the charge of the photoelectric conversion unit, the potential of the photoelectric conversion part and the charge storage part A light receiving element including a plurality of pixels having a reset electrode to be initialized;
After the initialization, each time the synchronization signal in a shorter period than the image frame period is output, after reading the first charge accumulating the charges in the charge storage part, the photoelectric conversion unit to said drain electrode A charge reading unit for discarding the charge and reading the charge accumulated in the charge accumulation unit as a second charge when an image frame period elapses ;
A control unit that outputs the synchronization signal having a cycle shorter than an image frame cycle to the charge reading unit;
Equipped with a,
When the first charge is equal to or greater than a predetermined threshold, the control unit controls the charge read unit to reduce a charge accumulation time according to the synchronization signal, and when the first charge is equal to or less than a predetermined threshold, imaging apparatus characterized that you control the charge reading section so as to increase the charge storage time by said synchronizing signal. Wherein the control unit detects the movement of the Utsushitai imaging apparatus according to claim 1, characterized in that for generating the synchronizing signal synchronized with the movement of the subject based on the detection result.   The imaging apparatus according to claim 1, wherein the control unit generates the synchronization signal having a predetermined period that is not synchronized with an image frame period in the imaging apparatus. An imaging method in an imaging apparatus,
A step in which the photoelectric conversion unit outputs an electric charge according to the exposure amount;
A charge storage unit for storing the charge;
A drain electrode discarding the charge of the photoelectric conversion unit ;
A reset electrode initializing potentials of the photoelectric conversion unit and the charge storage unit;
Charge reading section, after the initialization, each time the synchronization signal is output in a shorter period than the image frame period, after reading the first charge accumulating the charges in the charge storage section, said drain electrode When the charge of the photoelectric conversion unit is discarded and an image frame period elapses, the charge stored in the charge storage unit is read as a second charge ;
The control unit outputs the synchronization signal having a period shorter than the image frame period to the charge reading unit, and when the first charge is equal to or greater than a predetermined threshold, the charge is reduced so as to reduce a charge accumulation time by the synchronization signal. Controlling the reading unit to control the charge reading unit to increase a charge accumulation time according to the synchronization signal when the first charge is equal to or lower than a predetermined threshold ;
An imaging method characterized by comprising:

Images