JP2014207641A - High-speed imaging device having trigger signal generation function - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a high-speed imaging device having a function to generate a trigger signal for determining imaging timing.SOLUTION: A part of pixels of an ISIS-CCD is replaced with a luminance comparison sensor pixel. In the sensor pixel, a photodiode charge is recorded into a CCD memory and a capacitor, so that the magnitude (luminance) thereof is compared with the magnitude of a charge after a predetermined time (predetermined frame). As a result, when there is a certain degree of difference, it is determined that a change occurs in a subject and a change detection signal is output accordingly. The output is then extracted from the image sensor and input to a drive circuit as a trigger signal so as to control the operation of the imaging device. This enables ultra high-speed imaging without losing imaging timing.

Description

  The present invention relates to an image sensor used for ultra high-speed imaging, and more particularly to a high-speed image sensor having a function of capturing a change in a subject at high speed and generating a trigger signal for determining imaging timing.

  In response to the increasing demand for ultra-high-speed shooting in recent years, a high-speed image sensor called ISIS (In-situ Storage Image Sensor) having a shooting speed of several million to several tens of million fps (frames per second) has been developed. ,It's being used. The ISIS includes an image recording unit that records image information (video signal) adjacent to each photodiode in the light receiving surface of the imaging element, and is also called a pixel peripheral recording type imaging element. During shooting, the image information is not read from the image sensor, but continuously stored in the image recording unit, and the image information is sequentially output after shooting. The shooting speed per image can be increased from the limit of the transfer time to the adjacent image recording unit.

  Among the ISIS, an image recording unit having a CCD (Charge Coupled Device) structure adjacent to a photodiode is called an ISIS-CCD, and an ultra-high speed camera using the ISIS-CCD has been put into practical use. . (Patent Document 1)

  The structure and operation of an image pickup device composed of an ISIS-CCD will be briefly described.

  FIG. 6 is an example of a planar structure of the ISIS-CCD. The ISIS-CCD is a photodiode 1 as a photoelectric conversion unit arranged in an array in the horizontal and vertical directions in the image area shown in the figure, and one end thereof is adjacent to each photodiode 1 in the direction of photodiode arrangement. On the other hand, an image recording unit 2 arranged in an oblique direction and a vertical transfer unit 3 arranged in a vertical direction are provided.

The image recording unit 2 includes n recording elements 2 ₁ to 2 _n . Specifically, it has a CCD structure with n stages of transfer units, and always records signal charges (images) for n frames while moving the signal charges. The number n of the recording elements 2 ₁ to 2 _n constituting the image recording unit 2 can be designed as necessary. _If the number n of recording elements is increased, the number of images (number of frames) that can be recorded can be increased, and high speed Although shooting of the phenomenon can be facilitated, about tens to about 100 is generally selected because of problems such as an increase in cost and a limitation on the mounting area. Since the ISIS-CCD includes the image recording unit 2 inside the image area, the unit pixel region (the vertical direction is the length of the repeating unit of the photodiode, and the horizontal direction is the repeating unit of the vertical transfer unit 3). The width area is relatively large, and is designed to be, for example, about 50 μm × 50 μm.

Next, the operation of the ISIS-CCD will be described. First, at the imaging stage, the photodiode 1 and the image recording unit 2 (2 ₁ to 2 _n ) are driven by the clock, and the signal charges generated by the photodiode 1 are transferred to the image recording unit 2 (2 ₁ to 2 _n ). Sequentially transferred and stored. At the imaging stage, the accumulated signal charge is discarded from the discharge unit (not shown) provided at the end of the image recording unit 2 in the oldest order, so that the image recording unit 2 always has the latest video (image information). ) Is accumulated.

In the signal charge reading stage after the imaging operation is completed, the signal charges stored in the image recording units 2 (2 ₁ to 2 _n ) are sequentially transferred to the vertical transfer unit 3 by the clock, and then the vertical transfer unit 3 The signal charge is transferred in the vertical direction. The charges transferred in the vertical direction by the vertical transfer unit 3 are transferred in the horizontal direction by a horizontal transfer unit (not shown) and are subjected to signal processing in the output circuit.

  By the way, especially in ultra-high-speed shooting that exceeds several hundred thousand fps, the occurrence of a phenomenon is notified due to restrictions on the number of shots, etc., and an external trigger is used to start / stop the camera shooting operation at an appropriate timing. It is essential to input a signal. For example, a method is used in which a trigger signal synchronized with a phenomenon desired to be photographed is used, photographing is performed for a preset number of frames from the trigger signal, and then the photographing operation is terminated. If the occurrence of the phenomenon can be controlled from the outside, the trigger signal may be generated and input at a necessary timing. If not, the occurrence of the phenomenon should be detected and the trigger signal generated without delay.

  Conventionally, it has been attempted to generate a trigger signal by detecting a phenomenon in which a subject changes momentarily at a high speed. For example, there is an apparatus that captures a change in luminance of a subject and generates a trigger signal based on the change (Patent Document 2). The trigger signal generator includes a light irradiating unit that irradiates a subject with reference light having a predetermined wavelength, a reflected light receiving unit that receives reflected light reflected from the subject, and a luminance change detecting unit that detects a luminance change of the subject. And trigger signal generating means for generating a trigger signal based on the luminance change, and can detect the change of the subject and generate the trigger signal at high speed.

  An apparatus that generates a trigger signal by capturing a phenomenon (subject change) using image processing based on a video signal of a subject captured by an imaging device is also being studied (Patent Document 3). In this apparatus, a standard image consisting of M × N pixel data and the density of each pixel being s (i, j) is prepared in advance, and the density of each pixel is f (i, j) from the image sensor. When the correlation value of s (i, j) and f (i, j) is calculated by a predetermined calculation formula while sequentially receiving M × N image data as j), and the correlation value exceeds the threshold value A trigger signal is being output.

JP 2000-165750 A JP 2007-184703 A JP 2011-171806 A

  The apparatus described in Patent Document 2 requires light irradiation means and reflected light receiving means in addition to the imaging apparatus, and the system for imaging tends to increase in size. In addition, when these means cannot be properly installed, a trigger signal cannot be obtained.

  The method of using a part of the light (image) from the subject for generating the trigger signal is not limited to the above-mentioned device, but as an imaging device, for example, photographing a microscope or observing a phenomenon in a special chamber from a viewing window It cannot be used when it is difficult to separate the light from the subject into devices other than the camera, or when the disadvantage that the image of the imaging device becomes dark by dividing the light cannot be allowed. Even if the trigger signal generator is installed in the optical system of the camera using any method, there is a drawback in that the image of the camera becomes dark because the light is not changed.

  In addition, since the apparatus described in Patent Document 3 calculates a correlation value of a large amount of pixel data, it can be considered that only a shooting speed of about several thousand fps can be handled in consideration of the time required for image processing. Further, if the area of M × N pixel data to be processed is reduced in order to increase the processing speed, only the small section of the captured image is used, and trigger signal generation is not possible. May be accurate.

  Further, even if the means for sequentially calculating and processing the video signal obtained from the image sensor is applied to, for example, the ISIS-CCD, the ISIS-CCD does not output the video signal to the outside during super-high-speed shooting. Judgment based on was impossible.

  An object of the present invention is to solve such drawbacks of the prior art, and a trigger signal for determining the shooting timing in super-high-speed shooting such as several hundred thousand fps to ten million fps. An object is to detect an optical change of a subject and generate a trigger signal by the image sensor itself even in a situation that cannot be obtained from the outside.

  In order to solve the above problems, a high-speed imaging device according to the present invention is a high-speed imaging device having an image area in which a plurality of pixels are arranged in an array, and includes at least one sensor pixel in the image area, A trigger signal is generated based on a signal obtained from a sensor pixel indicating that a subject has changed, and the trigger signal is used as a timing signal for starting or stopping shooting in a high-speed shooting operation. .

  It is desirable that the sensor pixel compares the brightness of images received before and after a predetermined time, and determines that the subject has changed and outputs a signal when the difference is greater than a set threshold value.

  The sensor pixel is recorded in a photoelectric conversion unit, a CCD memory or capacitor that records charges generated in the photoelectric conversion unit, and a current amount of charge generated in the photoelectric conversion unit and in the CCD memory or capacitor. It is desirable to provide a comparison circuit that compares the amount of the charged charges.

  The sensor pixel includes a photoelectric conversion unit, an A / D converter that performs A / D conversion on the amount of charge generated by the photoelectric conversion unit, a delay circuit that inputs an output signal of the A / D converter, It is desirable to provide a comparison circuit that compares the output signal of the A / D converter and the output signal of the delay circuit.

  The high-speed imaging device is preferably an ISIS (In-situ Storage Image Sensor) -CCD having an image recording unit adjacent to the photoelectric conversion unit.

  According to the present invention, since the trigger signal is generated based on the signal from the luminance comparison type sensor pixel provided in the image area of the image sensor, the change of the subject image received by the image sensor is captured even during the super-high-speed shooting operation. Thus, it is possible to notify the drive circuit of the timing of the start of shooting or the stop of shooting. In addition, since the image sensor and the sensor are integrated, it is not necessary to separately divide the light from the subject for the sensor, and a bright video signal can be acquired.

It is a top view which shows the example of arrangement | positioning of the sensor pixel of the image pick-up element of this invention. It is a block diagram of the sensor pixel of this invention. It is a top view which shows the structure around the sensor pixel of this invention. It is a structural example of the sensor pixel of this invention. It is a flowchart of operation | movement of the sensor pixel of this invention. It is a top view which shows an example of the pixel structure of ISIS-CCD.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(Embodiment)
FIG. 1 is a plan view showing an arrangement example of sensor pixels of an image sensor according to the present invention. The present invention can be applied to any image sensor, but in the embodiment, an ISIS-CCD, which is a high-speed image sensor that can effectively use the present invention, will be described as an example.

  The image sensor 10 includes an image area 20, and a large number of pixels (not shown individually) are arranged in an array in the image area 20. In the image area 20, sensor pixels 30 are arranged by replacing pixels of the image sensor every several tens to several hundreds of pixels vertically and horizontally. As will be described in detail later, the sensor pixel 30 is a pixel having a function of detecting a change in the subject by capturing a change in luminance of the received image. FIG. 1 conceptually shows the arrangement of sensor pixels, and the size, shape, and the like of the sensor pixels 30 are not accurately drawn.

  The arrangement of the sensor pixels 30 in FIG. 1 is an example, and the number of sensor pixels may be further increased or decreased, and at least one sensor pixel 30 may be provided in the image area at a minimum. Further, the arrangement is not limited to the matrix arrangement as shown in FIG. The sensor pixel 30 can be appropriately arranged in consideration of the relationship with the detection sensitivity of the subject change.

  In addition, in the place where the sensor pixel 30 is arranged, the video signal cannot be used as the data of the captured image. However, the video data of the pixel replaced with the sensor pixel 30 is complemented by the arithmetic processing from the peripheral pixels, so It is possible to prevent defects from occurring.

  FIG. 2 is a block diagram of the sensor pixel of the present invention. The sensor pixel 30 includes a photoelectric conversion unit 31 including a photodiode, a signal charge or a luminance signal accumulation unit 32 obtained by converting the signal charge into a corresponding voltage value, and a comparison / change detection signal output unit 33. .

  The light beam conversion unit 31 can be composed of an arbitrary light receiving element. However, if the light conversion unit 31 has the same structure as the photodiode 1 of other pixels, it is advantageous because the manufacturing process can be shared. The charge that is the image information of the subject generated by the photoelectric conversion unit 31 moves to the storage unit 32 as a signal charge or a luminance signal based on the amount of the signal charge.

  The accumulation unit 32 has a function of accumulating a signal charge output from the light beam conversion unit 31 or a luminance signal based on the amount of the signal charge for a predetermined time (that is, for a predetermined number of frames). Various means are conceivable for realizing the function of the storage unit 32. Typically, there are two types, a charge storage method using a CCD cell or the like and a signal delay method using a delay circuit or the like. In the charge accumulation method, signal charges for one or a plurality of frames are accumulated in the accumulation unit 32, and a signal charge for one or a plurality of frames before or a luminance signal based on the amount of the signal charges is sequentially output. When the image sensor is an ISIS-CCD, a part of the image recording unit 2 having a CCD structure adjacent to the photodiode can be used as the storage unit 32. In the signal delay method, the image information (luminance signal) of the subject generated by the photoelectric conversion unit 31 is output after being delayed for a predetermined time by the delay circuit in the storage unit 32. The output of the storage unit 32 is input to the comparison / change detection signal output unit 33.

  The comparison / change detection signal output unit 33 compares the luminance signal of the current frame output from the photoelectric conversion unit 31 with the luminance signal of one or more frames before output from the storage unit 32, and compares the difference between them (that is, , Whether or not the subject has changed based on whether or not the difference in brightness exceeds a predetermined threshold value, and outputs a change detection signal.

  Thus, each sensor pixel 30 can detect a change in luminance of the subject from the luminance signal and output it as a change detection signal.

  Outputs from the sensor pixels 30 are aggregated over the entire image sensor, and when a change detection signal is output from one or a predetermined number of sensor pixels 30, processing is performed so that a trigger signal for the image sensor is generated. For example, when the sensor pixels 30 in the image area 20 are sequentially scanned and a change detection signal is output, the signal is taken out and input to the drive circuit as a trigger signal to control the operation of the image sensor. . Thereby, it becomes possible to perform super-high-speed shooting without missing the shooting timing. The scanning is not limited to the method of scanning all sensor pixels uniformly and sequentially, and a change detection signal may be acquired by intensively scanning a part of the image area 20 where the movement of the subject is easily detected. .

  In order to be able to respond as a timing signal for starting or stopping shooting, it is desirable that the signal from the sensor pixel 30 can be read regardless of whether or not the shooting operation is being performed. For this reason, this change detection signal is read by means different from the image signal of the normal pixel of the image sensor.

  FIG. 3 shows a structure around the sensor pixel of the ISIS-CCD. The same parts as those in FIG. 6 are denoted by the same reference numerals, and description thereof is omitted. As shown in FIG. 6, the normal portion of the pixel is composed of a photodiode 1 which is a photoelectric conversion unit and an image recording unit 2 having a CCD structure, but the sensor pixel 30 performs signal processing on the photoelectric conversion unit (photodiode) 31. The circuit is added.

  FIG. 3A shows a case in which a charge storage type storage unit 32 is used. The photodiode 31 and the comparison / change detection signal output unit 33 including a storage unit 32 and a comparison circuit made of a CCD memory A sensor pixel 30 is configured. The means for storing the charges in the storage unit 32 is not limited to the CCD memory, and other charge storage means such as a capacitor may be used. For example, one of the CCD structures used in the image recording unit 2 of a normal pixel may be used. Use of a part is advantageous because the manufacturing process can be made common.

  FIG. 3B shows a case where a signal delay type storage unit 32 ′ is used, and a photodiode 31, a storage unit 32 ′ including a delay circuit, and a comparison / change detection signal output unit 33 including a comparison circuit. Thus, the sensor pixel 30 is configured.

  The output from the sensor pixel 30 is immediately output from the image sensor without being sent to the adjacent image recording unit 2. For example, the output of the sensor pixel 30 is connected to a wiring (not shown) made of a metal or a transparent conductive material that extends in the vertical or horizontal direction adjacent to the photodiode 1 and can be immediately output to the outside of the image sensor. it can. A plurality of sensor pixels 30 may be connected by a common wiring so that change detection signals may be acquired collectively, and a change detection signal is acquired by sequentially scanning the output of the sensor pixels 30 using a scanning circuit. You may do it. When there is a sensor pixel 30 that outputs a change detection signal, a trigger signal is generated.

  In the sensor pixel 30, the CCD memory (the hatched CCD structure portion in FIG. 3) as the image recording unit 2 originally formed corresponding to the photodiode 31 is not used. It is desirable to provide a drain region 4 for discharging the charge at the entrance of the transfer path of the CCD memory that is not used in order to prevent charges from entering the CCD memory that is not used accidentally and being transferred to cause noise.

FIG. 4 is a configuration example of the sensor pixel of the present invention, and is a schematic diagram of a unit pixel region including the sensor pixel 30. FIG. 4A shows a sensor pixel 30 having a CCD memory 34. The sensor pixel 30 includes a photodiode 31, CCD memories 34 ₁ and 34 ₂ , a comparison circuit 35, and a selection switch 36. As can be seen from FIG. 3, in the unit pixel area, image recording units (CCD memory) 2 of other pixels around the sensor pixel are arranged so as to pass through a part thereof.

In FIG. 4A, two CCD memories 34 are shown, but the number of CCD memories may be one or three or more. The number of CCD memories corresponding to the period (number of frames) for accumulating signal charges is provided. If CCD memory 34 is one, according to the arrow drawn by the solid line, the one frame before the signal charges accumulated in the CCD memory 34 ₁ volume and, the current generated by the photodiode 31 frames of the signal charge The amount is compared by the comparison circuit 35. In comparison, since image information is stored as charges (signal charges) in the photodiode 31 and the CCD memory 34, the charge − is input to the respective outputs of the photodiode 31 and the CCD memory 34 or to the entrance of the comparison circuit 35. It is desirable to provide a voltage converter (for example, a capacitor) and convert it to a luminance signal (voltage signal) corresponding to the amount of signal charge, and compare this luminance signal with the comparison circuit 35.

If CCD memory 34 is two, according to the arrow drawn in broken lines, the amount of two frames before the signal charges accumulated in the CCD memory 34 ₂ (or luminance signal), the amount of the signal charges of the current frame ( (Or luminance signal) is compared by the comparison circuit 35. When n CCD memories 34 are provided, the comparison circuit 35 can compare the amount of signal charges (or luminance signal) n frames before and the amount of signal charges (or luminance signal) of the current frame. . For example, the comparison circuit can be designed to output an output signal when these comparison results exceed a predetermined threshold.

Further, the present signal charge generated by the photodiode 31 amounts and, in one frame before the signal charges accumulated in the CCD memory 34 ₁ volume and, stored in the CCD memory 34 ₂ 2 frames before the signal charges A change in the subject may be detected by examining three correlations with the quantity. The calculation for detecting the change can be selected as appropriate.

  The control unit of the image sensor sequentially scans the sensor pixels using, for example, the selection switch 36, determines that the subject has changed when there is a change detection signal, and generates a trigger signal. Can do.

  FIG. 4B is an example of the sensor pixel 30 having a delay circuit instead of the CCD memory. The sensor pixel 30 includes a photodiode 31, a capacitor (not shown) and an analog / digital (A / D) converter 37, a delay circuit 38, a comparison circuit 35, and a selection switch 36. In the unit pixel area, an image recording unit (CCD memory) 2 of other pixels around the sensor pixel is arranged so as to pass therethrough.

  In FIG. 4B, a signal charge that is a video signal of a subject generated by the photodiode 31 is converted into a voltage signal by a capacitor (not shown) or the like, and further an analog / digital (A / D) converter 37. Is converted into a digital signal. The digitized signal (image luminance information) is output to the comparison circuit 35 and the delay circuit 38, respectively. In the delay circuit 38, the signal is delayed for a time corresponding to the generation of an image of one or a plurality of frames and output to the comparison circuit 35. The comparison circuit 35 generates a digitized signal based on the signal charge amount of the image of the current frame and a digitized signal based on the signal charge amount of the image one or more frames before delayed by the delay circuit 38. For example, when these comparison results exceed a predetermined threshold value, the comparison circuit 35 can be designed to output an output signal.

  The control unit of the image sensor sequentially scans the sensor pixels using, for example, the selection switch 36, determines that the subject has changed when there is a change detection signal, and generates a trigger signal. Can do.

  Next, the operation of the sensor pixel of the present invention will be described with reference to the flowchart shown in FIG. It will be described with reference to the configuration of the sensor pixel 30 in FIG.

  In the first step (S0: step 0), all the signals are discharged from the storage unit 32. This eliminates unnecessary signals that cause noise and misrecognition.

  In the next step (S1: step 1), the video signal generated by the photodiode 31 (corresponding to the video signal of the previous frame of the subject) is moved from the photodiode 31 to the storage unit 32. Note that the video signal may be a signal charge generated by a photodiode or a luminance signal based on the amount of the signal charge. The movement of the signal is controlled by a clock and is performed for each frame.

In the next step (S2: Step 2), photoelectric conversion is performed in the photodiode 31, and the generated charge as a video signal of the subject is accumulated in the photodiode 31. During this time, the signal of the previous frame is accumulated in the accumulation unit 32. When there are a plurality of CCD memories 34 in the storage unit 32, or when delay processing for a plurality of frames is performed in the delay circuit 38 of the storage unit 32, the above steps 1 and 2 may be repeated a plurality of times. In this case, the storage unit 32 utilizes a CCD transfer function, the signal charges move from the CCD memory 34 ₁ in the CCD memory 34 _2.

  In the next step (S3: step 3), a signal corresponding to the signal charge amount accumulated in the photodiode 31 is output from the photodiode 31, and a signal corresponding to the signal charge amount accumulated in the accumulation unit 32 is output. Or a signal subjected to delay processing, and both are input to the comparison circuit 35 to compare the signals of the photodiode 31 and the storage unit 32.

  Next (S4: Step 4), it is determined whether or not the result of comparing the signals of the photodiode 31 and the storage unit 32 has exceeded a predetermined threshold.

  If there is no change exceeding the predetermined threshold (S5: Step 5), it is determined that there is no change in the subject, and the change detection signal is turned off.

  If there is a change exceeding the predetermined threshold (S6: Step 6), it is determined that the subject has changed, and the change detection signal is turned ON. That is, the change detection signal is output from the sensor pixel when the difference between the current luminance of the image of the subject and the past luminance is equal to or greater than a predetermined threshold.

  It is desirable that the predetermined threshold value can be set to an arbitrary value from the outside of the image sensor (sensor pixel).

  After steps 5 and 6, the process returns to the first step.

  As described above, the present invention can generate a trigger signal that determines the shooting timing of the high-speed imaging device based on a change detection signal obtained from a part of pixels in the image area as a sensor pixel.

  Although the present invention has been described based on the drawings and examples, it should be noted that those skilled in the art can easily make various modifications and corrections based on the present disclosure. Therefore, it should be noted that these variations and modifications are included in the scope of the present invention. For example, the functions included in each means, each step, etc. can be rearranged so that there is no logical contradiction, and a plurality of means, steps, etc. can be combined or divided into one. .

1: Photoelectric converter (photodiode)
2: Image recording unit 3: Vertical transfer unit 4: Drain region 10: Image sensor 20: Image area 30: Sensor pixel 31: Photoelectric conversion unit (photodiode)
32: Accumulator 33: Comparison / change detection signal output unit 34: CCD memory 35: Comparison circuit 36: Selection switch 37: A / D converter 38: Delay circuit

Claims (5)

In a high-speed imaging device having an image area in which a plurality of pixels are arranged in an array,
Comprising at least one sensor pixel in the image area;
Based on a signal indicating that the subject obtained from the sensor pixel has changed, a trigger signal is generated,
A high-speed imaging device, wherein the trigger signal is used as a timing signal for starting or stopping shooting in a high-speed shooting operation.   The sensor pixel compares the luminance of images received before and after a predetermined time, and determines that the subject has changed when the difference is greater than a set threshold, and outputs a signal, The high-speed image sensor according to claim 1.   The sensor pixel is recorded in a photoelectric conversion unit, a CCD memory or capacitor that records charges generated in the photoelectric conversion unit, and a current amount of charge generated in the photoelectric conversion unit and in the CCD memory or capacitor. The high-speed imaging device according to claim 2, further comprising a comparison circuit that compares the amount of the charged charges.   The sensor pixel includes a photoelectric conversion unit, an A / D converter that performs A / D conversion on the amount of charge generated by the photoelectric conversion unit, a delay circuit that inputs an output signal of the A / D converter, The high-speed imaging device according to claim 2, further comprising a comparison circuit that compares an output signal of the A / D converter and an output signal of the delay circuit. 5. The high-speed imaging device is an in-situ storage image sensor (ISIS) -CCD having an image recording unit adjacent to a photoelectric conversion unit. 6. High-speed image sensor.

Images