JP4527027B2 - Imaging apparatus and imaging method - Google Patents

Description

  The present invention relates to an imaging apparatus and an imaging method. In particular, the present invention relates to an image pickup apparatus and an image pickup method for picking up a moving image with an image pickup element in which a plurality of pixels each storing received light as charges are arranged in a matrix.

  An imaging apparatus such as a digital camera or a digital video camera captures an image of a subject by an imaging element in which pixels that store received light as charges are arranged in a matrix. And the electric charge accumulate | stored in each pixel is acquired from an image pick-up element, and the gradation value of each pixel is obtained based on the amount of electric charges.

  As a configuration for acquiring electric charge from an image sensor, a CCD system in which a vertical CCD (VCCD) is provided corresponding to each column of pixels and a horizontal CCD (HCCD) connected to the ends of a plurality of vertical CCDs is often used. (See Patent Documents 1 and 2). As described in Patent Documents 1 and 2, in a general form of still image capturing, the charges of all pixels are transferred to the VCCD at the time of image capturing, and the charges of each pixel are transferred one line from the end of all the VCCDs. Are sequentially transferred to the HCCD. Then, every time the charge of each pixel for one row is transferred from the entire VCCD to the HCCD, the charge of all the pixels for one row is sequentially acquired from the end of the HCCD.

  On the other hand, when a moving image is output by the interlace method, the charge of each pixel is first transferred to the VCCD every other row, and the charge of each pixel every other row is sequentially transferred to the HCCD from every end of all VCCDs. . Then, every time the charge of each pixel for each row is transferred from the entire VCCD to the HCCD, the charge for all the pixels every other row is sequentially acquired from the edge of the HCCD for each row (Patent Document 1 and Patent Document 1). 2).

Conventionally, since the resolution of a moving image is lower than that of a still image, a low-resolution image for moving images is generated by thinning out or mixing charges for a plurality of pixels in the vertical direction when capturing a moving image. A technique has been disclosed (see Patent Document 3).
JP-A-5-83636 Japanese Patent Laid-Open No. 5-130525 JP 2004-48799 A

  In recent years, due to the commercialization of high-definition TV (HDTV), there is an increasing expectation for an imaging device that captures a moving image with a resolution as high as a still image. As the definition becomes higher, the number of pixels to be transferred per frame increases, so the transfer speed of HCCD becomes a problem.

  Accordingly, an object of the present invention is to provide an imaging apparatus and an imaging method that can solve the above-described problems. This object is achieved by a combination of features described in the independent claims. The dependent claims define further advantageous specific examples of the present invention.

  According to the first aspect of the present invention, there is provided an imaging apparatus that captures a moving image by an imaging element in which a plurality of pixels each storing received light as an electric charge are arranged in a matrix, and a frame image immediately before the moving image is obtained. An image recording unit for recording, and a charge acquisition unit that sequentially extracts charges from a group of pixels that are discontinuous in the row direction and the column direction among all the pixels in each of a plurality of phases, thereby extracting charges from the plurality of pixels And an image updating unit that updates gradation data of each pixel in the frame image recorded in the image recording unit in accordance with each charge amount extracted from each pixel in each phase, and the image updating An image pickup apparatus comprising: a moving image output unit that sequentially outputs the frame images updated by the unit.

  The charge acquisition unit may extract charge from the group of pixels arranged at predetermined intervals in the row direction and the column direction in each of the plurality of phases.

  In the first phase, the charge acquisition unit extracts charges from the group of pixels arranged in every other row and column directions, and in the second phase following the first phase, in the first phase. The charge may be taken out from each pixel located in the next row and next column of each pixel from which the charge has been taken out.

  In the third phase subsequent to the second phase, the charge acquisition unit extracts the charge from each pixel located in the same row and the previous column of each pixel from which the charge has been extracted in the second phase. In the fourth phase following the phase, the charge may be extracted from each pixel located in the next row and previous column of each pixel from which charge has been extracted in the third phase.

  In the third phase following the second phase, the charge acquisition unit extracts charge from each pixel located in the same row and previous column as each pixel from which charge has been extracted in the second phase, and the third phase In the fourth phase after the phase, the charge may be extracted from each pixel located in the previous row and the next column of each pixel from which the charge has been extracted in the third phase.

  Each of the charge acquisition units is provided corresponding to each column of the image sensor, and is arranged at predetermined intervals in the row direction among the pixels arranged in the corresponding column in each phase. A plurality of vertical CCDs that output the charges acquired from the pixels in series, and two or more vertical CCDs arranged at predetermined intervals in the column direction among the plurality of vertical CCDs in each phase. You may have the horizontal CCD which acquires the charge for 1 pixel per said vertical CCD output synchronously in parallel, and outputs it in series.

  In the first phase, the horizontal CCD obtains electric charge from the vertical CCDs in the odd-numbered columns using the region connected to the odd-numbered vertical CCDs as a potential well and the region connected to the even-numbered vertical CCDs as a potential barrier. In the second phase, charges are supplied from the vertical CCDs in the even columns using the regions connected to the vertical CCDs in the odd columns as potential barriers and the regions connected to the vertical CCDs in the even columns as potential wells. You may acquire and output.

  In any one of the phases, the image processing unit further includes a detection unit that detects whether flash light is irradiated on the subject during imaging, and the image update unit is in the phase where the flash light irradiation is not detected. The gradation data of each pixel in the frame image recorded in the image recording unit may be updated in accordance with the amount of charge extracted from each pixel.

  According to the second aspect of the present invention, there is provided an imaging method for imaging a moving image by an imaging apparatus including an imaging element in which a plurality of pixels each storing received light as a charge are arranged in a matrix, and immediately before the moving image In the image recording stage for recording the frame image, and by sequentially taking out charges from a group of pixels discontinuous in the row direction and the column direction among all the pixels in each of the plurality of phases, the charges are extracted from the plurality of pixels. A charge acquisition stage to be taken out, and an image update stage to update the gradation data of each pixel in the frame image recorded in the frame recording stage in accordance with the amount of charge taken out from each pixel in each phase. And a moving image output step for sequentially outputting the frame images updated in the image update step. The law provides.

  The above summary of the invention does not enumerate all the necessary features of the present invention, and sub-combinations of these feature groups can also be the invention.

  ADVANTAGE OF THE INVENTION According to this invention, the imaging device which acquires efficiently the electric charge of each pixel from an image pick-up element, and outputs a high resolution moving image can be provided.

  Hereinafter, the present invention will be described through embodiments of the invention. However, the following embodiments do not limit the invention according to the scope of claims, and all combinations of features described in the embodiments are included. It is not necessarily essential for the solution of the invention.

  FIG. 1 shows a configuration of an imaging apparatus 10 according to the present embodiment. The imaging device 10 aims to efficiently acquire the charge of each pixel from the imaging element in order to capture a high-definition moving image. The imaging device 10 includes a lens unit 100, an image sensor 110, a charge acquisition unit 120, an image update unit 130, an image recording unit 140, a moving image output unit 150, a moving image recording unit 160, a detection unit 170, And a flash unit 180.

  The lens unit 100 includes a lens, a diaphragm, a mechanical shutter, and the like, captures an image of a subject, and forms an image on the image sensor 110. The image sensor 110 has a configuration in which a plurality of pixels that accumulates light received as charges are arranged in a matrix. The charge acquisition unit 120 sequentially extracts the charge from each pixel of the image sensor 110 and obtains the gradation value of the pixel corresponding to the charge amount. The image updating unit 130 updates the gradation data of each pixel in the frame image recorded in the image recording unit 140 with the gradation value corresponding to each charge amount extracted from each pixel.

  The image recording unit 140 records the immediately preceding frame image in the moving image. Then, in response to an update instruction from the image updating unit 130, the gradation data of each pixel in the next frame image is overwritten as the gradation data of each pixel of the immediately preceding frame image. The moving image output unit 150 sequentially outputs the frame images updated by the image update unit 130. Here, when outputting an interlaced moving image, the moving image output unit 150 reads out the gradation data of the pixels in each row at a predetermined line interval such as every other row, and sequentially outputs it to the moving image recording unit 160. On the other hand, when a progressive format moving image is output, the moving image output unit 150 reads out the gradation data of the pixels in each row in order, and outputs them sequentially to the moving image recording unit 160. Here, the moving image output unit 150 may directly output the moving image to an external display device or the like instead of outputting the moving image to the moving image recording unit 160. The moving image recording unit 160 records the moving image output by the moving image output unit 150.

  The flash unit 180 irradiates the subject with flash light when the brightness of the subject is insufficient or when the photographer instructs the subject. The detection unit 170 detects whether or not flash light has been emitted to the subject during moving image capturing. Then, the detection unit 170 instructs the image update unit 130 not to write the gradation data based on the charge accumulated by the image sensor 110 when the flash light is irradiated to the image recording unit 140.

  Here, the detection unit 170 detects that the flash light has been irradiated on the condition that the flash light is irradiated for the still image imaging when the photographer of the imaging apparatus 10 instructs to capture the still image. Good. The detection unit 170 may include an optical sensor that detects external brightness, and may determine that flash light has been irradiated when brightness that is equal to or higher than a preset reference value is detected. In addition, the detection unit 170 determines whether the gradation value of the pixel acquired by the charge acquisition unit 120 in the phase or the average value of the gradation value of at least two pixels is a predetermined reference in comparison with the value in the other phase. When the value is larger than the value, it may be detected that the flash light has been irradiated. Further, the detection unit 170 may determine that the flash light has been irradiated on the condition that a flash emission notification transmitted by wireless or the like is received when another imaging device has irradiated the flash light.

  FIG. 2 shows a configuration of the image sensor 110 and the charge acquisition unit 120 according to the present embodiment. The image sensor 110 includes pixels 200 arranged in a two-dimensional matrix in the row direction (row 1, row 2,..., Row Ny) and in the column direction (column 1, column 2,..., Column Nx). In addition, the charge acquisition unit 120 is connected to a plurality of VCCDs 210 provided corresponding to the respective columns of the image sensor 110 and ends of the VCCDs 210, and charges for one row output from the VCCDs 210 are paralleled. HCCD 220 acquired in series and output in series, and a charge detection unit 230 that detects the charge amount of each pixel output in series by HCCD 220 and obtains the gradation value of the pixel based on the charge amount.

  The charge acquisition unit 120 according to the present embodiment is discontinuous in the row direction and the column direction among all the pixels in each of a plurality of phases obtained by dividing one frame period instead of acquiring the charges of all the pixels at once. By sequentially taking out charges from a group of pixels, the charges are taken out from a plurality of pixels of the image sensor 110. That is, for example, 120 takes out charge from a group of pixels arranged at predetermined intervals in the row direction and column direction, such as every other row and every other column, for example, in each of a plurality of phases.

  In order to realize this, each VCCD 210 according to the present embodiment has the charge acquired from the pixels arranged at predetermined intervals in the row direction among the pixels arranged in the corresponding column in each phase. Are output in series. That is, for example, each VCCD 210 acquires the charge of each pixel arranged in the corresponding column every other row and outputs the charge to the HCCD 220 in order from the charge of the pixel having the largest row number.

  Next, in each phase, the HCCD 220 according to the present embodiment has one pixel per VCCD 210 that is synchronously output from two or more VCCDs 210 arranged at predetermined intervals in the column direction among the plurality of VCCDs 210. Are obtained in parallel and output in series. That is, for example, the HCCD 220 acquires the charge of each pixel in every other column in parallel among the charge of each pixel for one row output in parallel from each VCCD 210 and outputs it in series.

  More specifically, in the first phase, the HCCD 220 acquires charges from the odd-numbered VCCD 210 in the first phase using the region connected to the odd-numbered VCCD 210 as a potential well and the region connected to the even-numbered VCCD 210 as a potential barrier. Output. Further, in the second phase, a region connected to the odd-numbered VCCD 210 is used as a potential barrier, and a region connected to the even-numbered VCCD 210 is used as a potential well to acquire and output charges from the even-numbered VCCD 210.

The image update unit 130 updates the gradation data of each pixel in the frame image recorded in the image recording unit 140 with the gradation value corresponding to each charge amount extracted from each pixel in each phase.

  When the charge of pixels every other row and every other column is acquired for each phase, processing for four phases is required to update the frame image in the image recording unit 140. Instead, the number of pixels from which charges are to be acquired in each phase is 1/4. Here, when the charge of every other row and every other column of pixels is acquired, the charge of the pixels is uniformly acquired from the entire imaging region for each phase. Therefore, the charge is acquired row by row from the top as in the progressive method. When compared with the case where the charge is acquired every other row as in the case of the interlace method, the flicker of the output moving image becomes smaller. For this reason, according to the imaging device 10 according to the present embodiment, flickering of moving images can be suppressed even if the frame interval is slightly increased. As a result, the operating speed of the HCCD 220 can be reduced.

  Further, the gradation data of each pixel is temporarily stored in the image recording unit 140 and then output, so that the gradation data is acquired from the pixels arranged at predetermined intervals in the row direction and the column direction, and the progressive format and Video data compatible with both interlace formats can be output.

  FIG. 3 shows an example of the update order of each pixel in the image recording unit 140 according to this embodiment. In the first phase, the charge acquisition unit 120 extracts charges from a group of pixels arranged every other row and column. That is, for example, the charge acquisition unit 120 takes out the charge of each pixel located in the odd-numbered row and the odd-numbered column as indicated by (1) in the drawing. Next, in the second phase subsequent to the first phase, the charge acquisition unit 120 extracts charge from each pixel located in the next row and next column of each pixel from which charge has been extracted in the first phase. That is, for example, as shown in (2) in the figure, the charge acquisition unit 120 takes out the charge of each pixel located in an even-numbered row and an even-numbered column.

  Furthermore, in the third phase subsequent to the second phase, the charge acquisition unit 120 extracts charges from each pixel located in the same row and column before each pixel from which charges have been extracted in the second phase. That is, for example, the charge acquisition unit 120 extracts the charge of each pixel located in the odd-numbered row and even-numbered column as indicated by (3) in the figure. Then, in the fourth phase after the third phase, the charge acquisition unit 120 extracts the charge from each pixel located in the next row and the previous column of each pixel from which the charge has been extracted in the third phase. That is, for example, the charge acquisition unit 120 takes out the charge of each pixel located in the even-numbered row and the odd-numbered column as indicated by (4) in the drawing.

  Note that the charge acquisition unit 120 may start acquiring the charge according to the new frame image from any of the above phases. That is, for example, for a new frame image, charges may be acquired from each pixel in the order of (1), (2), (3), and (4) in the figure, and instead of this, (3), (4), and (1) in the figure. ) Charge may be acquired from each pixel in the order of (2).

  In this manner, by updating the gradation data of each pixel sparsely in the row direction and the column direction, the imaging apparatus 10 can suppress flickering of the moving image.

  FIG. 4 shows another example of the update order of each pixel in the image recording unit 140 according to this embodiment. In the first phase and the second phase, the charge acquisition unit 120 extracts charges from each pixel in the same order as in FIG. Next, in the third phase following the second phase, the charge acquisition unit 120 extracts charge from each pixel located in the same row and previous column as each pixel from which charge has been extracted in the second phase. That is, for example, the charge acquisition unit 120 extracts the charge of each pixel located in the even-numbered row and the odd-numbered column as indicated by (3) in the drawing. Then, in the fourth phase after the third phase, the charge acquisition unit 120 extracts the charge from each pixel located in the previous row and the next column of each pixel from which the charge has been extracted in the third phase. That is, for example, the charge acquisition unit 120 extracts the charge of each pixel located in the odd-numbered row and even-numbered column as indicated by (4) in the figure.

  Note that the charge acquisition unit 120 may start acquiring the charge according to the new frame image from any of the above phases, as in FIG.

  According to the imaging device 10 described above, a moving image with less flicker can be obtained by acquiring the charge of the pixels at predetermined intervals in the row direction and the column direction in each phase. Further, each VCCD 210 only needs to acquire the charges of the pixels 200 in separate rows and transfer them to the HCCD 220. Therefore, compared with the case where all the rows are acquired in parallel and transferred to the HCCD 220, the number of potential wells and potential barriers of the CCD is reduced. Can be reduced. As a result, the circuit scale of the VCCD 210 can be reduced. Similarly, the HCCD 220 only needs to transfer the charges received from the VCCD 210 to the charge detection unit 230 in a row, so that the potential of the CCD is compared with the case where the charges in all the columns are acquired in parallel and transferred to the charge detection unit 230. The number of wells and potential barriers can be reduced.

  When this method is implemented in the update order shown in FIG. 3 or FIG. 4, the pixel indicated by (1) in the figure receives light during the first phase to the fourth phase and charges at the beginning of the next first phase. Will be output. Similarly, the pixels indicated by (2), (3) and (4) in the figure are respectively from the second phase to the first phase, from the third phase to the second phase, from the fourth phase to the third phase. Light is received until the phase.

  Alternatively, charges may be emitted from each pixel before light reception. For example, in the update sequence of FIG. 3, while the odd-numbered VCCD 210 outputs the charge of the pixel indicated by (1) in the figure to the HCCD 220, the even-numbered VCCD 210 indicates that the pixel indicated by (2) in the figure is the same. The charge is acquired and released to the ground or the like without being transferred to the HCCD 220. This operation can be realized by propagating electric charges to the HCCD 220 side in synchronization with the odd-numbered VCCDs 210 and discharging them from the end of the VCCD 210 to the ground or the like. Similarly, while the even-numbered VCCD 210 outputs the charges of the pixels indicated by (2) in the figure to the HCCD 220, the odd-numbered VCCD 210 acquires the charges of the pixels indicated by (3) in the figure and supplies them to the HCCD 220. It is discharged to the ground etc. without being transferred. In particular, in the update sequence of FIG. 3, while the odd-numbered VCCD 210 outputs the charge to the HCCD 220, the even-numbered VCCD 210 releases the charge and the even-numbered VCCD 210 outputs the charge to the HCCD 220. It is sufficient that the odd-numbered VCCDs 210 release the charges.

  According to this embodiment, the light receiving periods of the pixels do not overlap. Therefore, in any phase, when the detection unit 170 detects whether flash light has been applied to the subject during imaging, the charge acquired from the pixels received in the phase is released. You can also. In this case, the image update unit 130 determines the gradation of each pixel in the frame image recorded in the image recording unit 140 according to the amount of charge extracted from each pixel in the phase where the flash light irradiation is not detected. Update the data. As a result, the image update unit 130 can generate a moving image that is less affected by the flash light.

  FIG. 5 shows configurations of the image sensor 110 and the charge acquisition unit 120 according to a modification of the present embodiment. Since the charge acquisition unit 120 only needs to acquire charge from each pixel in a row for each phase, a configuration may be adopted in which one VCCD 210 is provided for a plurality of columns of pixels 200 that do not transfer charges simultaneously in the same phase. it can.

  In the figure, one VCCD 210 is provided for every two columns of the odd-numbered columns of pixels 200 and the even-numbered columns of pixels 200 next to the odd-numbered columns. When the charge is acquired from the pixel of (1) in FIG. 3, the VCCD 210 acquires the charge from every other row of pixels 200 and transfers it to the HCCD 220. When acquiring the charge from the pixel (2) in FIG. 3, the VCCD 210 acquires the charge from the even-numbered pixels 200 every other row and transfers it to the HCCD 220. In order to realize this, a gate that permits or prohibits the movement of charges may be provided between each pixel 200 and the VCCD 210.

  According to the imaging apparatus 10 described above, the flicker of the moving image that is output by acquiring the charge of the pixels uniformly from the entire imaging region for each phase is reduced, and the flickering of the moving image is suppressed even if the frame interval is slightly increased. It becomes possible.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiment. It is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

1 shows a configuration of an imaging apparatus 10 according to an embodiment of the present invention. 1 shows a configuration of an image sensor 110 and a charge acquisition unit 120 according to an embodiment of the present invention. An example of the update order of each pixel in the image recording part 140 which concerns on embodiment of this invention is shown. 6 shows another example of the update order of each pixel in the image recording unit 140 according to the embodiment of the present invention. The structure of the image pick-up element 110 and the electric charge acquisition part 120 which concern on the modification of embodiment of this invention is shown.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Imaging device 100 Lens part 110 Image pick-up element 120 Charge acquisition part 130 Image update part 140 Image recording part 150 Movie output part 160 Movie recording part 170 Detection part 180 Flash part 200 Pixel 210 VCCD
220 HCCD
230 Charge detector

Claims (9)

An imaging apparatus that captures a moving image with an imaging element in which a plurality of pixels are arranged in a matrix,
An image recording unit for recording the immediately preceding frame image in the moving image;
An acquisition unit that acquires a gradation value from a group of pixels discontinuous in the row direction and the column direction among all the pixels in each of the plurality of phases;
In any one of the phases, a detection unit that detects whether flash light is irradiated to the subject by another device during imaging;
An image updating unit that updates gradation data of each pixel in the frame image recorded in the image recording unit with each gradation value extracted from each pixel in the phase in which the irradiation of the flash light is not detected. When,
A moving image output unit that sequentially outputs the frame images updated by the image update unit.   The imaging device is a matrix in which a plurality of pixels each storing received light as charges are arranged in a matrix.
  The acquisition unit sequentially extracts charges from a group of pixels discontinuous in the row direction and the column direction among all the pixels in each of the plurality of phases, thereby extracting charges from the plurality of pixels.
  The image updating unit is configured to perform gradation of each pixel in the frame image recorded in the image recording unit in accordance with each charge amount extracted from each pixel in the phase where the flash light irradiation is not detected. Update data
  The imaging device according to claim 1. The imaging device according to claim 2 , wherein the acquisition unit extracts charges from the group of pixels arranged at predetermined intervals in a row direction and a column direction in each of the plurality of phases. The acquisition unit
In the first phase, electric charges are taken out from the group of pixels arranged every other row and column.
The imaging apparatus according to claim 3 , wherein in the second phase subsequent to the first phase, charge is extracted from each pixel located in the next row and next column of each pixel from which charge has been extracted in the first phase. The acquisition unit
In the third phase subsequent to the second phase, the charge is taken out from each pixel located in the same row and the previous column of each pixel from which the charge has been taken out in the second phase,
5. The imaging device according to claim 4 , wherein in the fourth phase following the third phase, charge is extracted from each pixel located in the next row and previous column of each pixel from which charge has been extracted in the third phase. The acquisition unit
In the third phase following the second phase, the charge is taken out from each pixel located in the same row and the previous column as each pixel from which the charge has been taken out in the second phase,
5. The imaging device according to claim 4 , wherein in the fourth phase following the third phase, charge is extracted from each pixel located in the previous row and next column of each pixel from which charge has been extracted in the third phase. The acquisition unit
Charges acquired from the pixels arranged at predetermined intervals in the row direction among the pixels arranged in the corresponding column in each phase provided for each column of the image sensor A plurality of vertical CCDs that output in series,
In each of the phases, charges for one pixel per the vertical CCDs output in synchronization from two or more vertical CCDs arranged at predetermined intervals in the column direction among the plurality of vertical CCDs are arranged in parallel. The image pickup apparatus according to claim 3 , further comprising: a horizontal CCD that acquires and outputs in series. The horizontal CCD is
In the first phase, a region connected to the vertical CCD in the odd column is set as a potential well, and a region connected to the vertical CCD in the even column is set as a potential barrier to obtain and output charges from the vertical CCD in the odd column,
In the second phase, charges are obtained and output from the vertical CCDs in the even columns using the regions connected to the vertical CCDs in the odd columns as potential barriers and the regions connected to the vertical CCDs in the even columns as potential wells. The imaging device according to claim 7 . An imaging method for imaging a moving image with an imaging device including an imaging device in which a plurality of pixels are arranged in a matrix,
An image recording stage for recording a previous frame image in the moving image;
Obtaining a gradation value from a group of pixels discontinuous in the row direction and the column direction among all the pixels in each of a plurality of phases;
In any one of the phases, a detection step of detecting whether flash light is irradiated on the subject by another device during imaging,
An image updating step of updating the gradation data of each pixel in the frame image recorded in the frame recording step with the respective gradation values extracted from each pixel in the phase where the flash light irradiation is not detected. When,
A moving image output step of sequentially outputting the frame images updated in the image update step.

Images