JP2006324834A - Device and method for imaging - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging device and method that can take a picture so that a part outside a cut-out range is not in a dead angle as much as possible while following up change of a subject through a cutout read of an imaging element. <P>SOLUTION: The imaging element is configured to access respective pixels at random and read them, and also provided with two or more signal output terminals to read two or more places simultaneously. The whole is normally photographed at a slow frame rate. Once change of an image is detected from the whole image, the vicinity of the change is cut out and a preferential signal read is made through a second signal output terminal to increase the image update rate of the part where the change is present. A signal reading of the other part where there is no change is made at the normal slow frame rate as it is, those parts are individually signal-processed, and then two images are composed. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

Related to imaging device and imaging method

  As background art in this technical field, for example, there is JP-A-2004-23747 (Patent Document 1). This publication describes as a problem “Providing an electronic camera capable of speeding up autofocus control and optimizing preview display during autofocus control”. The CCD image pickup device 10 in the camera corresponds to a draft mode for reading an image signal corresponding to a predetermined number of pixels among all the pixels in the image pickup device as a read mode of the image signal, and a smaller number of pixels than the draft mode. An autofocus mode that reads out a plurality of images at high speed by reading out the image signal to be automatically focused when the shutter button is pressed halfway down, based on the image signal read out in the autofocus mode. Control is performed, and the display unit (back LCD, etc.) reads in draft mode immediately before shifting to autofocus mode. Has been described as. "The composite image of the image Gh based on the image signal read by the by the image Gb based on the image signal had an automatic focusing mode is displayed.

Japanese Patent Laid-Open No. 2004-23747

  An image pickup device using photoelectric conversion is supported by a miniaturization technique of a manufacturing process, and the number of pixels is increasing. While the resolution of the imaging device has increased due to the increase in the number of pixels of the imaging device, the digital still camera is forced to reduce the frame rate of moving image processing due to the limitation of the signal readout speed from the imaging device. Has occurred. Some consumer video cameras capture a frame rate required for moving image processing by reading out signals after effectively reducing the number of pixels by performing pixel mixing in the image sensor.

  Also, in a scene where a high image update rate is required in a digital still camera, for example, as described in Japanese Patent Application Laid-Open No. 2004-23747, a part of the screen is cut out and read when performing a focusing operation. A technique for performing control for shooting by increasing an image update rate is described. Further, according to the present invention, it is possible to confirm the angle of view taken by saving the captured image immediately before the focus operation and displaying the image on the viewfinder or the like using the viewfinder or the like.

  When installing surveillance cameras, I don't want to make blind spots as much as possible. By using a wide-angle lens for the surveillance camera, it is possible to reduce the blind spot if a wide area is photographed at once. However, when photographing a wide range with a wide-angle lens, a subject far away is photographed small, making it impossible to confirm details of the subject far away. In order to solve such a problem, it is conceivable to use an image pickup device having a large number of pixels in the background art, but the image update rate is sacrificed, and thus the speed of following the change of the subject is sacrificed. In addition, there is a technique (described about focus control) that increases the screen update rate by cutting out a part of the screen to track the subject, as in the technique described in Japanese Patent Application Laid-Open No. 2004-23747. The outside is actually a blind spot, which is contrary to the idea of not wanting to make a blind spot as much as possible.

  Therefore, usability can be improved if it is possible to perform shooting so that a portion outside the cutout range does not become a blind spot as much as possible while following the change of the subject by cutout readout of the image sensor.

  The problem to be solved by the present invention is to improve the usability of the imaging apparatus.

  The above problems are solved by the invention described in the claims.

  According to the present invention, usability of the imaging apparatus is improved.

  Hereinafter, examples will be described with reference to the drawings.

  FIG. 1 shows an example of the configuration of an imaging apparatus according to the present invention. In the figure, 101 is an imaging lens for imaging light coming from a subject to be imaged on the imaging surface of the image sensor 102, and 102 is light converted to electrical energy and accumulated as electric charge. An imaging device, 103 and 104 are AD conversion means for converting an analog imaging signal read from the imaging device into a digital imaging signal, and 105 and 106 are signals that generate an image signal composed of a luminance signal and a color signal from the digital imaging signal. The processing means 1 and the signal processing means 2, 107 and 108 are generated by the signal processing means 1 and the signal processing means 2, respectively. Image synthesizing processing means for synthesizing the two obtained image signals, 110 a controller for controlling the operation of the imaging apparatus, and 111 for the operation of the imaging apparatus A timing generating means for generating a timing reference. Reference numeral 112 denotes a signal processing unit in which the signal processing unit 1, the signal processing unit 2, the recognition processing unit 1, the recognition processing unit 2 and the image composition processing unit are formed, and is referred to as a digital signal processing unit in the present invention.

  The operation of the imaging apparatus will be described as follows.

  In the imaging device, pixels that perform photoelectric conversion are arranged in a grid pattern on the imaging surface, and two signal readout routes A and B are provided for each pixel (FIG. 3). The readout control means (not shown) provided in the imaging device outputs the imaging signal via the wiring A or the wiring B by opening and closing the switch A and the switch B provided in each pixel. This is done by sending it to terminal A or output terminal B.

  The range of pixels read from each output terminal of the output terminal A and the output terminal B is determined by an area having a start line and an end position set by the controller in the read control unit as diagonal lines, and is read from the output terminal A. When areas read from the output terminal B overlap, priority is given to one of the areas set by the controller so that each area becomes exclusive.

  It is assumed that the signal reading order is performed sequentially for each line in the horizontal direction or the vertical direction from the start position to the end position. FIG. 10 illustrates the reading order when reading is performed for each line in the horizontal direction with respect to the N × M pixel region. As shown in the figure, when reading out the area of N × M pixels for each horizontal line, when the sequential reading of the consecutive N pixels in the horizontal direction is completed with the start pixel as the head, a total of M lines in the form of the next line is completed. Reading is performed for the pixels. However, when the readout stop period of the exclusive area is reached, the switch opening / closing operation is stopped so that signal readout is not performed during that period. Also, when all pixels are read at high speed, all pixels are read by simultaneously opening and closing some of the readout switches for the pixels exposed with light dispersed by the same color filter and adding signals as necessary. Period. For example, when signals of four pixels are added, the signals are read out in the order shown in FIG. 12 if the image sensor has a color filter array as shown in FIG.

  The image pickup signal read from the image pickup device is converted from an AD analog image pickup signal into a digital image pickup signal by the AD conversion unit 1 and the AD conversion unit 2 and supplied to the digital signal processing unit.

  The digital signal processing means first processes two image signals read from two output terminals provided in the image sensor using the signal processing means 1 and the signal processing means 2 to obtain two image signals. Generate. At this time, if the signal readout range from one imaging device is set to the entire imaging surface of the imaging device and the signal readout range from the other imaging device is set to a part of the imaging surface of the imaging device, the angle of view at which the camera can shoot Two image signals of the entire captured image (hereinafter referred to as “background image”) and a captured image obtained by cutting out a part of the captured image (hereinafter referred to as “target image”) are generated. The generated background image and the target image are respectively compared with the newly captured image supplied to the recognition processing unit 1 and the recognition processing unit 2 and the captured image captured before that to determine the movement of the captured object. It detects and extracts information (hereinafter referred to as motion vector information) indicating the amount of movement at each position in the screen (x: amount of movement in the horizontal direction, y: amount of movement in the vertical direction) and parameterizes it into a database. In addition, the background image and the target image are combined by the image combining processing means to be a monitor output. As a combining method, the position information of the target image is acquired from the controller and the target image is inserted into the background image. . At this time, since the readout range of the target image is narrower than the readout range of the background image, when the readout time per unit pixel is the same, a plurality of target images are generated while one background image is generated. I can do it. If the generation rate of the attention image is three times the generation rate of the background image, three attention images are generated for one background image. By utilizing this, it becomes possible to generate three composite images by combining the background image with each of the three target images, and as a result, three high-definition whole images are formed. (The state of image composition is shown in FIG. 6). FIG. 7 shows a cycle of image processing when continuously capturing images at this time. In addition, the three images configured here are equivalent to images obtained by updating the screen by paying attention only to a portion having motion, and are images that have high follow-up performance with respect to the motion within the screen.

  Based on the motion vector information of the captured image obtained by the digital signal processing means, the controller determines an attention area indicating a range in which the attention image is cut out from the monitoring area that forms the background image of the captured image. The initial state of the imaging apparatus starts from a state where there is no area of interest, but the area of interest is determined as follows. FIG. 2 shows a motion vector distribution in the case where a moving object (in the broken line portion in FIG. 2A) exists in the monitoring area as shown in FIG. 2A. The attention area is set based on the distribution of the motion vector, and is set so as to surround the moving part as shown in FIG. 5 and to increase in the moving direction of the object in consideration of the movement of the object. To do. For the region once set as the attention area, the attention area in the next processing cycle is predicted to update the attention area using the motion vector information in the attention area. FIG. 8 illustrates a state in which the attention area is updated once every time the background image is generated. The attention image before the attention image c generated at the end of the generation period of the background image A in FIG. The attention area of the background image B readout period is set from the motion vector information at. Further, when a new motion is detected outside the attention area, for example, when a new motion is detected by analyzing the motion vector information of the background image A in FIG. 7, attention is paid to the generation timing of the background image B. An area is added to generate an attention image of two areas, a prediction area and an attention area. In addition, when there are motion vector groups having different directions in the same area in the area prediction, the area is divided in the next area prediction (FIG. 13). When the prediction results of a plurality of areas overlap, the areas are also combined (FIG. 14). Furthermore, when it is desired to improve the tracking performance over the entire monitoring area, the resolution as a still image is reduced, but the update rate of the entire monitoring area is set by performing pixel mixing with the image sensor set to the pixel mixing mode. The imaging apparatus is controlled so as to speed up the process.

  The timing generation unit generates a signal that serves as a reference for an image update cycle, and synchronizes the operation sequences of the image sensor, the digital signal processing unit, and the controller. For example, by generating a binary pulse that fluctuates in the background image update cycle as a reference signal, the image sensor, the digital signal processing means, and the controller perform internal operations based on the change point of the binary pulse. Control the timing and synchronize the whole system.

  According to the above, it is possible to take a photographed image of a wide area at a time with high definition and high followability to the movement of the photographed object.

  The following is an example when an imaging device is configured using an imaging device that can be randomly accessed and has only one readout output as the imaging device to be used. If an image sensor that can be randomly accessed is used as the image sensor, by devising the arrangement of the filters, the roles of the pixels are grouped for the background and the image of interest, and the pixels of the background image and the image of interest Can be read out in a time-sharing manner. FIG. 15 shows an example of the configuration of the imaging apparatus at that time. Here, for example, the signal supplied to the signal processing unit is obtained by offsetting the AD conversion cycle by 180 degrees between the AD converter 1 and the AD converter 2. Can be separated. FIG. 4 shows an example in which grouping is performed for each column, and FIG. 16 shows an arrangement example of pixel filters at that time. Moreover, the state of the offset of AD conversion is shown in FIG. The image reading range in the image sensor of FIG. 15 is based on the angle of view with respect to the background, and the designation of the area of interest depends on the designation of the controller as in the first embodiment. Based on the above, it is possible to realize an imaging apparatus capable of performing imaging with high definition and good tracking performance as in the first embodiment. Further, in the second embodiment, since the pixels that form the image of the attention area and the pixels that make the background are independent, the prediction of the attention area can be performed every time the image of the attention area is generated. can get. In the explanatory diagram of this embodiment (FIG. 15), there are two AD converters, but digital data may be distributed after AD conversion with one AD.

  The present invention is applied to a configuration of an imaging apparatus and an imaging method thereof.

Example of configuration of imaging device The figure which shows the relation between the movement of the object in the surveillance area and the motion vector Configuration example of imaging apparatus of first embodiment An example of a method of grouping pixels as a source of generation of a background image and a target image in the second embodiment Explanatory drawing of setting criteria for attention area Image composition in image composition means Time chart showing generation timing relationship between background image and attention image Predictive image of attention area Additional image of attention area The figure which shows the signal reading order from an image sensor Example of color filter pattern of image sensor The figure which shows the reading order in the case of performing multiple pixel (4 pixel) simultaneous reading Image of attention area division Image of attention area combination Second embodiment of the present invention different from the first embodiment Example of image sensor color pattern in the second embodiment The figure which shows the mode of the offset of AD conversion in 2nd Example

Explanation of symbols

101 imaging lens 102 imaging device 103 AD conversion means 1
104 AD conversion means 2
105 Signal processing means 1
106 Signal processing means 2
107 Recognition processing means 2
108 Recognition processing means 1
109 Image composition processing means 110 Controller 111 Timing generation means 1502 Imaging device

Claims (10)

An image sensor having a plurality of signal readout routes for each pixel and outputting signals in parallel from each signal readout route;
Signal processing means for processing a plurality of signals output in parallel from each signal readout route of the image sensor;
Combining means for combining a plurality of signals processed by the signal processing means to generate one signal;
An imaging apparatus comprising: In claim 1,
The plurality of signals output in parallel from the image sensor have different angles of view and / or frame rates,
An imaging apparatus characterized by the above. In claim 1 or 2,
Of the plurality of signals output in parallel from the image sensor, the first signal has an angle of view substantially equal to the maximum angle of view that can be imaged by the image sensor, and the second signal is smaller than the angle of view. The angle of view,
An imaging apparatus characterized by the above. 4. The imaging apparatus according to claim 3, wherein the frame rate of the first signal is smaller than the frame rate of the second signal. In claim 3 or 4,
Recognition processing means for detecting the movement of the imaging object from the output signal of the signal processing means;
With
The second signal has a predetermined angle of view including a portion in which movement is detected by the recognition processing means;
An imaging apparatus characterized by the above.   An image sensor having at least two or more signal readout routes, and each of the signal readout routes enables readout for all pixels, and AD conversion is individually performed for each output of the image sensor. A plurality of AD means for performing, and a plurality of digital imaging signals digitized by the plurality of AD conversion means, respectively, individually performing signal processing to generate digital image signals composed of luminance signals and color signals Image signal processing means, and a plurality of recognition processing means for individually detecting and quantifying the amount of movement of an object in the image for a plurality of digital image signals generated by the plurality of image signal processing means, An image composition processing means for combining the plurality of digital image signals to generate another digital image signal, and an apparatus based on the object movement information in the image digitized by the plurality of recognition processing means. An image pickup apparatus including a controller that controls the image pickup device, which reads out image signals having different angles of view and frame rates in parallel from two or more readout routes of the image sensor, and is generated from the two or more image pickup signals. An image pickup apparatus that generates another digital image signal by combining the digital image signals.   7. The imaging apparatus according to claim 6, wherein one of the two or more imaging signals read from the imaging element is an imaging signal for generating a captured image of a maximum range that can be captured by the imaging apparatus, and the remaining The angle of view is set so that the imaging signal becomes an imaging signal for generating a captured image by cutting out a portion including the periphery of the portion where the imaging object moves within the range that can be imaged by the imaging device. Shooting by raising the frame rate of the part, generating a composite image that combines all of the captured images obtained by cutting out the moving part and the periphery of the captured image in the maximum range, and using that, a moving image is generated An imaging method to be generated.   An image sensor capable of reading out all pixels, a plurality of AD means for performing AD conversion on the output of the image sensor in a time-sharing manner, and a plurality of digitalized signals by the plurality of AD converter means A plurality of image signal processing means for individually generating a digital image signal composed of a luminance signal and a color signal, and a plurality of image signal processing means generated by the plurality of image signal processing means. A plurality of recognition processing means for individually detecting and digitizing the amount of movement of an object in the image with respect to the digital image signal, and image synthesis for generating another digital image signal by combining the plurality of digital image signals An imaging device comprising a processing means and a controller for controlling the device based on object movement information in the image digitized by the plurality of recognition processing means, wherein two or more readings of the imaging element are read. Angle from the root out, reads in parallel different imaging signal frame rates, they more than one imaging device to generate another digital image signal a digital image signal combined and generated from the image signal.   An image sensor capable of reading out all pixels, an AD means for AD-converting the output signal of the image sensor, and a digital image signal digitized by the AD converter means in a time division manner and individually A plurality of image signal processing means for generating a digital image signal composed of a luminance signal and a color signal by performing signal processing, and a plurality of digital image signals generated by the plurality of image signal processing means, respectively, A plurality of recognition processing means for detecting and digitizing the amount of movement of an object in the image, an image composition processing means for synthesizing the plurality of digital image signals to generate another digital image signal, and a numerical value by the plurality of recognition processing means An image pickup apparatus including a controller that controls the apparatus based on object movement information in the converted image, and includes an angle of view and a frame rate from two or more readout routes of the image pickup element. Different imaging signal is read out in parallel with Mureto, they more than one imaging device to generate another digital image signal a digital image signal combined and generated from the image signal. 10. The imaging device according to claim 8, wherein one of the two or more imaging signals divided by time in a dot-sequential manner generates a captured image in a maximum range that can be captured by the imaging device. The remaining image signal is an image signal that is used to generate a captured image that is obtained by cutting out the area where the object is moving and the area surrounding the area where the object is moving within the range that can be captured by the image capturing apparatus. Set the corner, shoot with the frame rate of the moving part raised, and generate a composite image that combines the moving part and all the captured images cut out of the periphery of the captured image of the maximum range An imaging method for generating a moving image using the same.

Images