JP2007221423A - Imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus which is suitable for shooting a moving photographic subject and can produce a dynamic-range-enhanced image with less blurring due to moving of the photographic subject and less photogene. <P>SOLUTION: The imaging apparatus 1 includes an imaging device 2 in which whole pixels are divided into two pixel groups and in each of the two pixel groups, image signals with different exposure times in one frame can be read out, and further the exposure times of the two pixel groups can be replaced in each one frame; a frame memory 4 which records an image signal of the last frame; and separating circuits 5, 6 which separates the image signals into a prolonged exposure signal and a brief exposure signal. A motion detection circuit 9 detects a motion of the photographic subject from the present image signal and the image signal of the last frame. Interpolation signal generating circuits 10, 11 generate interpolation image signals according to a degree of the motion of the photographic subject. Selection circuits 12, 13 interpolate the present image signals by using the interpolation image signal to generate image signals of the whole pixels. A synthesizing circuit 14 generates a synthesized image signal of the whole pixels. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image pickup apparatus suitable for shooting a moving subject, and more particularly to an image pickup apparatus capable of shooting a dynamic range expanded image with less motion blur and afterimage.

  In recent years, a solid-state imaging device such as a CCD has been used as an imaging device used in an imaging apparatus such as a video camera. However, an imaging device using a solid-state imaging device has a narrow dynamic range with respect to the amount of incident light as compared to a silver salt imaging device. When the dynamic range of the image pickup apparatus is narrow, there is a problem that blackout or overexposure occurs in the captured image.

Therefore, conventionally, as an imaging apparatus capable of expanding the dynamic range, an apparatus that synthesizes an image having a wide dynamic range using a plurality of image signals having different exposure amounts is known (see, for example, Patent Document 1). In a conventional imaging apparatus, an appropriate exposure amount is obtained based on an image signal obtained by photographing the first frame of the imaging element and the exposure amount at that time. Based on this, photographing with the image sensor of the second frame is performed with an appropriate exposure amount or overexposure and underexposure. Then, the image signals of the first frame and the second frame are stored in the memory, and the image signals of the first frame and the second frame stored in the memory are combined to generate one image with an expanded dynamic range. To do.
JP-A-9-107499 (page 3-4, FIG. 2)

  However, in the conventional imaging apparatus, two image signals photographed with different exposure amounts (appropriate exposure amount or overexposure, underexposure) are read from the solid-state imaging device at time intervals for each frame period. For example, an overexposure (long exposure) image signal and an underexposure (short exposure) image signal are alternately read from the CCD every frame period. For this reason, when a moving subject is photographed, an image blur occurs between the long exposure image and the short exposure image. Therefore, the conventional imaging apparatus has a problem that motion blur and an afterimage occur in an image obtained by combining two images, and as a result, there is a problem that it is difficult to increase the frame rate.

  The present invention has been made to solve the above-described conventional problems, and is suitable for shooting a moving subject, and provides an imaging apparatus capable of obtaining a dynamic range enlarged image with less motion blur and afterimage. With the goal.

  In the imaging apparatus of the present invention, all the pixels are divided into two pixel groups, and each of the two pixel groups can read video signals having different exposure times within a predetermined frame period. An imaging means whose exposure time is changed every predetermined frame period, a recording means for recording a video signal obtained by the imaging means as a video signal before a predetermined frame period, and an image obtained by the imaging means Separating means for separating the signal into a long exposure signal having a long exposure time and a short exposure signal having a short exposure time; a current video signal obtained by the imaging means; and a predetermined frame period recorded in the recording means Based on the previous video signal, motion detection means for detecting the motion of the subject, and the current long exposure obtained by the separation means according to the degree of motion of the subject A first interpolation signal generation means for generating an interpolation long-time exposure signal from a signal and a long-time exposure signal before a predetermined frame period, and the current obtained by the separation means in accordance with the degree of movement of the subject Second interpolation signal generating means for generating a short-time exposure signal for interpolation from a short-time exposure signal and a short-time exposure signal before a predetermined frame period; and the current long-time exposure signal as the long-time exposure signal for interpolation. First interpolation synthesis means for generating a long-time exposure signal for all pixels by interpolation, and interpolating the current short-time exposure signal with the short-time exposure signal for interpolation, so that a short-time exposure signal for all pixels And a second interpolating and synthesizing unit that synthesizes the long-time exposure signal for all the pixels and the short-time exposure signal for all the pixels to generate a synthesized video signal for all the pixels. It has the composition provided.

  With this configuration, video signals (long exposure signal and short exposure signal) having different exposure times for each of the two pixel groups are read and separated into a long exposure signal and a short exposure signal. The exposure times of these two pixel groups are switched every predetermined frame period. Then, a long exposure signal for interpolation and a short exposure signal for interpolation are generated using the long exposure signal and the short exposure signal before a predetermined frame period, and the current long exposure signal and the short exposure signal are generated. Interpolate. In this manner, a composite video signal for all the pixels for one frame period is generated using the long exposure signal and the short exposure signal for two frame periods divided into two pixel groups. As a result, it is possible to obtain a composite video signal for all pixels with an expanded dynamic range. Also, with this configuration, when a moving subject is photographed, the movement of the subject is detected based on the current video signal and the video signal before a predetermined frame period. Then, according to the degree of movement of the subject, the long-time exposure signal for interpolation and the short-time exposure signal for interpolation are generated using the current long-time exposure signal and the short-time exposure signal, and the current long-time exposure signal and The short exposure signal is interpolated. Therefore, even when a moving subject is photographed, the current image (current long exposure signal and short exposure signal) and the interpolation image (interpolation long exposure signal and interpolation short exposure signal). ) Image blurring between Thereby, it is possible to suppress the occurrence of motion blur and afterimages in the generated long-time exposure signal and short-time exposure signal for all the pixels and the synthesized video signal for all the pixels, and the frame rate can be increased.

  Further, the imaging apparatus of the present invention includes a first interpolation pixel generating unit that generates a long exposure signal of an interpolation pixel group corresponding to the other pixel group based on a current long exposure signal of the one pixel group. And a second interpolation pixel generating means for generating a short-time exposure signal of an interpolation pixel group corresponding to the other pixel group based on a current short-time exposure signal of the one pixel group, The interpolation signal generation means combines the long exposure signal of the interpolated pixel group and the long exposure signal before the predetermined frame period according to the degree of movement of the subject to generate the interpolation long exposure signal. And the second interpolation signal generation means combines the short-time exposure signal of the interpolation pixel group and the short-time exposure signal before the predetermined frame period according to the degree of movement of the subject. And has a configuration for generating a short-time interpolation exposure signal.

  With this configuration, the long exposure signal and the short exposure signal for the interpolated pixel group are generated based on the current long exposure signal and the short exposure signal for one pixel group. Since the two pixel groups are switched every predetermined frame period, the interpolation pixel group corresponding to the other pixel group corresponds to one pixel group before the predetermined frame period. Then, in accordance with the movement of the subject, the interpolation pixel group, the long exposure signal and the short exposure signal before a predetermined frame period are combined to generate the interpolation long exposure signal and the interpolation short exposure signal. Thereby, the long exposure signal for interpolation and the short exposure signal for interpolation according to the movement of the subject can be generated using the current long exposure signal and the short exposure signal. Therefore, even when a moving subject is photographed, the current image (current long exposure signal and short exposure signal) and the interpolation image (interpolation long exposure signal and interpolation short exposure signal). ) Can be suppressed.

  In the photographing apparatus of the present invention, the motion detecting unit compares the long-time exposure signal of the interpolation pixel group with the long-time exposure signal before the predetermined frame period, and the short-time exposure signal of the interpolation pixel group. And the short-time exposure signal before the predetermined frame period are compared to detect the movement of the subject.

  With this configuration, it is possible to accurately detect the movement of the subject by comparing the interpolated pixel groups corresponding to each other with the long exposure signal and the short exposure signal before a predetermined frame period.

  In the photographing apparatus of the present invention, the movement detection unit may be configured to detect the amount of movement of the subject based on the current video signal obtained by the imaging unit and the video signal recorded in the recording unit before a predetermined frame period. The first interpolation signal generating means weights the long exposure signal of the interpolated pixel group and the long exposure signal before the predetermined frame period using the motion parameter. The second interpolation signal generating means generates the motion for the short-time exposure signal of the interpolated pixel group and the short-time exposure signal before the predetermined frame period. By performing a weighting process using parameters, the interpolation short-time exposure signal is generated.

  With this configuration, the motion parameter indicating the amount of movement of the subject is determined based on the current video signal and the video signal before a predetermined frame period. Then, by applying a weighting process using the motion parameter, it is possible to generate an interpolation long-time exposure signal and an interpolation short-time exposure signal corresponding to the motion of the subject. For example, when the amount of movement of the subject is large, the degree of weighting of the current long exposure signal and the short exposure signal is increased. Thus, the current long exposure signal and the short exposure signal can be used according to the movement of the subject, and the current image (the current long exposure signal and the short exposure signal) and the image for interpolation (for interpolation) Image blur between the long exposure signal and the interpolation short exposure signal) can be suppressed.

  In the imaging device of the present invention, in the imaging unit, one or more vertical pixel columns that constitute the one pixel group and one or more vertical pixel columns that constitute the other pixel group are horizontally arranged. The two pixel groups are formed alternately in the direction.

  With this configuration, the pixels in the vertical pixel column constituting two pixel groups (one pixel group and the other pixel group) are continuous in the vertical direction. Thereby, even when all the pixels are divided into two pixel groups, it is possible to prevent a decrease in resolution in the vertical direction of each pixel group. That is, the vertical resolution of each pixel group can be maintained at the same level as the resolution for all pixels.

  In the imaging apparatus of the present invention, in the imaging unit, one or more horizontal pixel rows constituting the one pixel group and one or more horizontal pixel rows constituting the other pixel group are vertically arranged. The two pixel groups are formed alternately in the direction.

  With this configuration, the pixels in the horizontal pixel row constituting the two pixel groups (one pixel group and the other pixel group) are continuous in the horizontal direction. Thereby, even when all the pixels are divided into two pixel groups, it is possible to prevent a decrease in resolution in the horizontal direction of each pixel group. That is, the horizontal resolution of each pixel group can be maintained at the same level as the resolution for all pixels.

  In the imaging device of the present invention, in the imaging unit, one or more pixel units constituting the one pixel group and one or more pixel units constituting the other pixel group are in the vertical direction and The two pixel groups are formed alternately arranged in the horizontal direction.

  With this configuration, the pixels of the two pixel groups (one pixel group and the other pixel group) are alternately arranged in the vertical direction and the horizontal direction for each pixel unit. That is, the pixels of the two pixel groups are arranged without deviation in the vertical direction and the horizontal direction. Thereby, the balance of the resolution of two pixel groups in the vertical direction and the horizontal direction is improved.

  The present invention is provided with an imaging means capable of alternately reading out video signals having different exposure times for each of two pixel groups every predetermined frame period, and a motion detection means for detecting the motion of the subject. By interpolating the current long exposure signal and the short exposure signal using the corresponding long exposure signal for interpolation and the short exposure signal for interpolation, a composite video signal for all pixels with an expanded dynamic range is obtained. In addition, it is possible to provide an imaging device that has the effect of suppressing the occurrence of motion blur and afterimages in the composite video signal for all pixels and increasing the frame rate.

  Hereinafter, an imaging device according to an embodiment of the present invention will be described with reference to the drawings. In this embodiment, the case of an imaging device used as a video camera or the like is illustrated.

(First embodiment)
FIG. 1 shows a block diagram of the imaging apparatus according to the first embodiment of the present invention. As shown in FIG. 1, the imaging device 1 temporarily receives an image sensor 2 such as a CCD or CMOS, a timing pulse generator 3 that controls the readout timing of the image sensor 2, and a video signal obtained by the image sensor 2. The frame memory 4 for recording is provided. For example, a video signal for one frame is recorded in the frame memory 4. Here, the imaging device 2 corresponds to the imaging means of the present invention, and the frame memory 4 corresponds to the recording means of the present invention.

  All the pixels of the image sensor 2 (for example, CCD) are divided into two pixel groups (a first pixel group and a second pixel group). FIG. 2 is an explanatory diagram for explaining an arrangement pattern of pixels of two pixel groups. In this case, as shown in FIG. 2, the two pixel groups include a vertical pixel column of the first pixel group configured by one column of pixels and a vertical pixel column of the second pixel group configured by one column of pixels. Are arranged alternately in the horizontal direction. For example, the vertical pixel column of the first pixel group is an odd column, and the vertical pixel column of the second pixel group is an even column. That is, all the pixels of the image sensor 2 are divided into two pixel groups for each column. In FIG. 2, only a part of all the pixels of the image sensor 2 is shown for convenience of explanation.

  The image pickup device 2 can read out video signals having different exposure times within one frame period for every two pixel groups, and the exposure times of the two pixel groups are changed every frame period. It is configured. In the present embodiment, in the image sensor 2, the timing pulse generator 3 controls the readout timing of each pixel group (that is, the accumulation time of pixel charges) independently in a time division manner.

  FIG. 3 is an explanatory diagram of the control operation of the pixel charge accumulation time and readout timing of the image sensor 2. As shown in FIG. 3, first, in the first frame, all the pixels are exposed for a long time to accumulate pixel charges, and only the first pixel group reads out the long-time exposure signal, Reset accumulated charge. Subsequently, all the pixels are exposed for a short time to accumulate pixel charges, and only the second pixel group reads the short-time exposure signal, and then the accumulated charges of all the pixels are reset. At this time, the long exposure signal of the first pixel group and the short exposure signal of the second pixel group read out in the first frame are temporarily recorded in the frame memory 4.

  Next, in the second frame, all the pixels are exposed for a long time to accumulate pixel charges, and only the second pixel group reads out the long-time exposure signal, and then the accumulated charges of all the pixels are reset. Subsequently, all the pixels are exposed for a short time to accumulate pixel charges, and only the first pixel group reads the short-time exposure signal, and then the accumulated charges of all the pixels are reset.

  In this way, in the image sensor 2, the long exposure signal and the short exposure signal are independently read out within one frame period for each of the two pixel groups, and the exposure times of the two pixel groups are It is replaced every frame period. In the figure, pixels from which the long-time exposure signal is read are indicated by white circles, and pixels from which the short-time exposure signal is read are indicated by hatched circles (for example, see FIG. 2).

  In the present embodiment, the long exposure signal and the short exposure signal are read out during one frame period. On the other hand, in a conventional imaging apparatus, a long exposure signal and a short exposure signal are read out every frame period. That is, while the conventional readout time difference between the long exposure signal and the short exposure signal is one frame, the deviation ΔT in the readout time between the long exposure signal and the short exposure signal in the present embodiment is It is extremely shorter than one frame period T (see FIG. 3).

  In addition, the imaging apparatus 1 includes a first separation circuit 5 to which a current video signal obtained by the imaging device 2 is input, and a second separation circuit to which a previous video signal recorded in the frame memory 4 is input. 6 is provided. The first separation circuit 5 separates the input current video signal into a long exposure signal and a short exposure signal. The second separation circuit 6 separates the input video signal of the previous frame into a long exposure signal and a short exposure signal. Here, the first separation circuit 5 and the second separation circuit 6 correspond to the separation means of the present invention.

  The imaging apparatus 1 also includes a first two-dimensional pixel interpolation circuit 7 to which the current long exposure signal separated by the first separation circuit 5 is input, and a current short time exposure separated by the first separation circuit 5. A second two-dimensional pixel interpolation circuit 8 to which signals are input is provided. In the first two-dimensional pixel interpolation circuit 7, using the long exposure signal of one pixel group, a long exposure signal of an interpolation pixel group that does not exist in one pixel group is generated by performing two-dimensional pixel interpolation. Is done. In the second two-dimensional pixel interpolation circuit 8, the short-time exposure signal of one pixel group is used to perform two-dimensional pixel interpolation using the short-time exposure signal of an interpolation pixel group that does not exist in one pixel group. Generated by. Here, the first two-dimensional pixel interpolation circuit 7 corresponds to the first interpolation pixel generation means of the present invention, and the second two-dimensional pixel interpolation circuit 8 corresponds to the second interpolation pixel generation means of the present invention.

  In the present embodiment, the two-dimensional pixel interpolation circuits 7 and 8 add pixel signals (long-time exposure signals or short-time exposure signals) of a plurality of pixels existing around the pixel to be interpolated (interpolated pixels). By averaging, a pixel signal of the interpolation pixel is generated. For example, as illustrated in FIG. 4, the pixel signal of the interpolation pixel (pixel illustrated by a broken line) is generated by averaging the pixel signals of pixels around the interpolation pixel (pixel illustrated by a solid line). In this manner, the pixel signal of the interpolation pixel group corresponding to the other pixel group is generated by the two-dimensional pixel interpolation using the pixel signal of one pixel group.

  Further, the imaging apparatus 1 uses the current frame video signal (long exposure signal and short exposure signal) and the previous frame video signal (long exposure signal and short exposure signal) to detect the movement of the subject. A motion detection circuit 9 for detection is provided. The motion detection circuit 9 includes a video signal (long exposure signal and short exposure signal) of the interpolation pixel group generated by the two-dimensional pixel interpolation circuits 7 and 8 and 1 separated by the second separation circuit 6. Video signals before the frame (long exposure signal and short exposure signal) are input. Here, the motion detection circuit 9 corresponds to the motion detection means of the present invention.

  The motion detection circuit 9 detects the motion (motion pixel) for each pixel between frames by comparing the two-dimensionally interpolated image of the current frame and pixels having the same phase in the spatially identical image. can do. For example, in the present embodiment, the long-time exposure signal (or short-time exposure signal) of the current frame subjected to two-dimensional pixel interpolation is compared with the long-time exposure signal (or short-time exposure signal) of the previous frame. , Motion pixels are detected. In this case, if a movement of a pixel between frames is detected in either one of the long-time exposure signal and the short-time exposure signal, the pixel having the movement is determined as a movement pixel.

  FIG. 5 is an explanatory diagram conceptually showing the detection of motion pixels. For example, in the example shown in FIG. 5, when the long exposure signal of the current frame is compared with the long exposure signal of the previous frame, pixels indicated by diagonal lines are detected as motion pixels. As a result, the movement amount Δx of the moving pixel is obtained as the movement amount of the subject. The movement parameter α (0 ≦ α ≦ 1) is determined according to the degree of movement of the subject, that is, according to the movement amount Δx of the subject. The movement parameter α is closer to 1 as the subject movement amount Δx is larger, and closer to 0 as the subject movement amount Δx is smaller. For example, in the case of a still image in which the subject does not move, the motion parameter α is 0.

  The imaging apparatus 1 also interpolates the first interpolation signal generation circuit 10 that generates an interpolation long-time exposure signal for interpolating the long-time exposure signal of the current frame, and the short-time exposure signal of the current frame. A second interpolation signal generation circuit 11 for generating a short-time exposure signal for interpolation. Here, the first interpolation signal generation circuit 10 corresponds to the first interpolation signal generation means of the present invention, and the second interpolation signal generation circuit 11 corresponds to the second interpolation signal generation means of the present invention. .

The first interpolation signal generation circuit 10 includes a long exposure signal of the interpolated pixel group generated using the long exposure signal of the current frame, and a long exposure of the previous frame separated by the second separation means. A signal is input, and a motion parameter α determined by the motion detection circuit 9 is input. Then, a long exposure signal for interpolation is synthesized by applying a weighting process using the motion parameter α to the long exposure signal of the interpolation pixel group of the current frame and the long exposure signal of the previous frame. Here, for example, a weighting process using Equation 1 below is performed.
Lc = (1−α) × Lp + α × Lr (Formula 1)
Here, Lc is a long-time exposure signal for interpolation, Lp is a long-time exposure signal one frame before, and Lr is a long-time exposure signal of the interpolation pixel group of the current frame.

The second interpolation signal generation circuit 11 includes a short-time exposure signal for the interpolated pixel group generated using the short-time exposure signal for the current frame, and a short-time exposure for the previous frame separated by the second separation unit. A signal is input, and a motion parameter α determined by the motion detection circuit 9 is input. Then, a short-time exposure signal for interpolation is synthesized by applying a weighting process using the motion parameter α to the short-time exposure signal of the interpolation pixel group of the current frame and the short-time exposure signal one frame before. Here, for example, a weighting process using Equation 2 below is performed.
Sc = (1−α) × Sp + α × Sr (Formula 2)
Here, Sc is a short-time exposure signal for interpolation, Sp is a short-time exposure signal one frame before, and Sr is a short-time exposure signal of the interpolation pixel group of the current frame.

  In the present embodiment, the video signals (interpolation long exposure signal and interpolation short exposure signal) generated in this way can also be referred to as interpolation pixels. On the other hand, the video signal (long exposure signal and short exposure signal) read out in the current frame (for example, the second frame) is not subjected to the interpolation process or the synthesis process as in the interpolation pixel described above. It can also be called a real pixel.

  The imaging apparatus 1 receives a first selection circuit 12 to which a long exposure signal for a current frame and a long exposure signal for interpolation are input, and a short exposure signal for a current frame and a short exposure signal for interpolation are input. A second selection circuit 13 is provided. The first selection circuit 12 alternately selects the long exposure signal and the interpolation long exposure signal for the current frame for each column pixel group. In this manner, the current long exposure signal is interpolated with the interpolation long exposure signal, and the long exposure signals for all the pixels are synthesized. In addition, the second selection circuit 13 alternately selects the short-time exposure signal and the interpolation short-time exposure signal for the current frame for each column pixel group. In this way, the current short-time exposure signal is interpolated with the interpolation short-time exposure signal, and the short-time exposure signals for all the pixels are synthesized. Here, the first selection circuit 12 corresponds to the first interpolation synthesis means of the present invention, and the second selection circuit 13 corresponds to the second interpolation synthesis means of the present invention.

  The imaging apparatus 1 also includes a synthesis circuit 14 to which a long exposure signal and a short exposure signal for all pixels are input. The synthesizing circuit 14 synthesizes the long-time exposure signal and the short-time exposure signal for all the pixels, and generates a synthesized video signal for all the pixels with an expanded dynamic range. Here, the synthesis circuit 14 corresponds to the video signal synthesis means of the present invention. Although not specifically shown, the synthesized video signal for all pixels is input to the gradation correction circuit and subjected to gradation correction, and then input to the luminance / color difference separation circuit to be separated into a luminance signal and a color difference signal. .

  The operation of the imaging apparatus 1 configured as described above will be described with reference to FIG.

  When a subject is shot using the imaging apparatus 1 according to the first embodiment of the present invention, an image of the video signal read from the imaging device 2 in the first frame is temporarily stored in the frame memory 4. . Then, a video signal is read from the image sensor 2 in the second frame. At this time, the exposure times of the two pixel groups in the first frame (previous frame) and the exposure times of the two pixel groups in the second frame (current frame) are switched (see FIG. 6).

  The video signal of the current frame is separated into a long exposure signal and a short exposure signal by the first separation circuit 5. The video signal of the previous frame (first frame) is separated into a long exposure signal and a short exposure signal by the second separation circuit 6.

  The long exposure signal of the current frame is two-dimensionally interpolated by the first two-dimensional pixel interpolation circuit 7 to generate a long exposure signal of the interpolated pixel group. Thereafter, in the first interpolation signal generation circuit 10, an interpolation long exposure signal is generated from the long exposure signal of the interpolation pixel group and the long exposure signal of the previous frame. At this time, the first interpolation signal generation circuit 10 performs weighting processing using the motion parameter α determined by the motion detection circuit 9, and generates an interpolation long-time exposure signal corresponding to the degree of movement of the subject. . Then, in the first selection circuit 12, the long exposure signal of the current frame is interpolated with the long exposure signal for interpolation, and a long exposure signal for all pixels is generated.

  On the other hand, the short-time exposure signal of the current frame is two-dimensionally interpolated by the second two-dimensional pixel interpolation circuit 8 to generate a short-time exposure signal of the interpolated pixel group. Thereafter, the second interpolation signal generation circuit 11 generates an interpolation short-time exposure signal from the short-time exposure signal of the interpolation pixel group and the short-time exposure signal of the previous frame. At this time, the second interpolation signal generation circuit 11 performs weighting processing using the motion parameter α determined by the motion detection circuit 9, and generates an interpolation short-time exposure signal corresponding to the degree of movement of the subject. . Then, in the second selection circuit 13, the short-time exposure signal of the current frame is interpolated with the short-time exposure signal for interpolation, and a short-time exposure signal for all pixels is generated.

  Finally, the long-time exposure signal and the short-time exposure signal for all the pixels are combined in the combining circuit 14 to obtain a combined video signal for all the pixels with an expanded dynamic range.

  According to the imaging apparatus 1 of the first embodiment of the invention as described above, the imaging device 2 capable of alternately reading out video signals having different exposure times for each of the two pixel groups every frame, and the movement of the subject. By providing a motion detection circuit 9 for detecting, by interpolating the current long exposure signal and the short exposure signal by using the long exposure signal for interpolation and the short exposure signal for interpolation according to the degree of movement of the subject. As a result, it is possible to obtain a composite video signal for all the pixels with an expanded dynamic range, suppress occurrence of motion blur and afterimages in the composite video signal for all the pixels, and increase the frame rate.

  That is, in the present embodiment, video signals (long exposure signal and short exposure signal) having different exposure times are read for each of two pixel groups, and separated into a long exposure signal and a short exposure signal. The exposure times of these two pixel groups are switched every frame. Then, a long exposure signal for interpolation and a short exposure signal for interpolation are generated using the long exposure signal and the short exposure signal one frame before, and the current long exposure signal and the short exposure signal are interpolated. . In this way, a composite video signal for all pixels for one frame is generated using the long exposure signal and the short exposure signal for two frames divided into two pixel groups. As a result, it is possible to obtain a composite video signal for all pixels with an expanded dynamic range.

  Further, in the present embodiment, when a moving subject is photographed, the movement of the subject is detected based on the current video signal and the video signal one frame before. Then, according to the degree of movement of the subject, the long-time exposure signal for interpolation and the short-time exposure signal for interpolation are generated using the current long-time exposure signal and the short-time exposure signal, and the current long-time exposure signal and The short exposure signal is interpolated. Therefore, even when a moving subject is photographed, the current image (current long exposure signal and short exposure signal) and the interpolation image (interpolation long exposure signal and interpolation short exposure signal). ) Image blurring between Thereby, it is possible to suppress the occurrence of motion blur and afterimages in the generated long-time exposure signal and short-time exposure signal for all the pixels and the synthesized video signal for all the pixels, and the frame rate can be increased.

  In this embodiment, the long exposure signal and the short exposure signal for the interpolated pixel group are generated based on the current long exposure signal and the short exposure signal for one pixel group. Since the two pixel groups are switched every frame, the interpolation pixel group corresponding to the other pixel group corresponds to one pixel group one frame before. Then, in accordance with the movement of the subject, the interpolation pixel group, the long exposure signal and the short exposure signal one frame before are synthesized, and the interpolation long exposure signal and the interpolation short exposure signal are generated. Thereby, the long exposure signal for interpolation and the short exposure signal for interpolation according to the movement of the subject can be generated using the current long exposure signal and the short exposure signal. Therefore, even when a moving subject is photographed, the current image (current long exposure signal and short exposure signal) and the interpolation image (interpolation long exposure signal and interpolation short exposure signal). ) Can be suppressed.

  Further, in the present embodiment, by comparing the interpolated pixel groups corresponding to each other with the long-time exposure signal and the short-time exposure signal one frame before, it is possible to accurately detect the movement of the subject.

  In the present embodiment, the motion parameter α indicating the movement amount Δx of the subject is determined based on the current video signal and the video signal one frame before. Then, by using the motion parameter α (0 ≦ α ≦ 1), weighting processing such as Equation 1 and Equation 2 described above is performed, so that the long exposure signal for interpolation and the short interpolation signal corresponding to the motion of the subject are obtained. A time exposure signal can be generated. For example, when the movement amount Δx of the subject is large, the motion parameter α is brought close to 1, and the degree of weighting of the current long exposure signal and the short exposure signal is increased. Thus, the current long exposure signal and the short exposure signal can be used according to the movement of the subject, and the current image (the current long exposure signal and the short exposure signal) and the image for interpolation (for interpolation) Image blur between the long exposure signal and the interpolation short exposure signal) can be suppressed.

  When the movement amount Δx of the subject is small, the motion parameter α is brought close to 0 to reduce the weighting degree of the current long exposure signal and the short exposure signal. That is, the long-time exposure signal for interpolation and the short-time exposure signal for interpolation are generated mainly using the long-time exposure signal and the short-time exposure signal one frame before. As a result, it is possible to obtain a high-resolution image using a highly reliable image signal that has not been two-dimensionally interpolated (long exposure signal and short exposure signal one frame before). As described above, in the present embodiment, an optimal interpolation pixel can be obtained according to the movement of the subject by changing the movement parameter α according to the movement of the subject.

  Further, in the present embodiment, the pixels in the vertical pixel row constituting the two pixel groups (first pixel group and second pixel group) are continuous in the vertical direction. Thereby, even when all the pixels are divided into two pixel groups, it is possible to prevent a decrease in resolution in the vertical direction of each pixel group. That is, the vertical resolution of each pixel group can be maintained at the same level as the resolution for all pixels.

  FIG. 7 shows a modification of the pixel arrangement pattern of two pixel groups. As shown in FIG. 7, the two pixel groups include a vertical pixel column of the first pixel group configured by two columns of pixels, and a vertical pixel column of the second pixel group configured by two columns of pixels. They may be arranged alternately in the horizontal direction. That is, all the pixels of the image sensor 2 may be divided into two pixel groups every two columns.

  Although not particularly shown here, the two pixel groups are a vertical pixel column of the first pixel group composed of three or more columns of pixels and a first pixel group composed of three or more columns of pixels. The vertical pixel columns of the two pixel group may be alternately arranged in the horizontal direction. That is, all the pixels of the image sensor 2 may be divided into two pixel groups for each of a plurality of columns.

(Second Embodiment)
Next, the imaging device 1 according to the second embodiment of the present invention will be described. The imaging device 1 of the present embodiment is different from the first embodiment in the arrangement pattern of the pixels of the two pixel groups. That is, the configuration and operation of the imaging apparatus 1 of the present embodiment are the same as the configuration of the first embodiment unless otherwise specified here.

  FIG. 8A is an explanatory diagram for explaining an arrangement pattern of pixels of two pixel groups in the present embodiment. As shown in FIG. 8A, the two pixel groups include a horizontal pixel row of the first pixel group configured by one row of pixels and a horizontal pixel row of the second pixel group configured by one row of pixels. Are arranged alternately in the vertical direction. For example, the horizontal pixel rows of the first pixel group are odd rows, and the horizontal pixel rows of the second pixel group are even rows. That is, all the pixels of the image sensor 2 are divided into two pixel groups for each row.

  In the imaging apparatus 1 according to the second embodiment of the invention as described above, it is possible to achieve the same effects as those of the first embodiment.

  In the present embodiment, the pixels in the horizontal pixel row constituting the two pixel groups (the first pixel group and the second pixel group) are continuous in the horizontal direction. Thereby, even when all the pixels are divided into two pixel groups, it is possible to prevent a decrease in resolution in the horizontal direction of each pixel group. That is, the horizontal resolution of each pixel group can be maintained at the same level as the resolution for all pixels.

  FIG. 8B shows a modification of the arrangement pattern of the pixels of the two pixel groups. As shown in FIG. 8B, the two pixel groups include a horizontal pixel row of the first pixel group constituted by two rows of pixels and a horizontal pixel row of the second pixel group constituted by two rows of pixels. May be arranged alternately in the vertical direction. That is, all the pixels of the image sensor 2 may be divided into two pixel groups every two rows.

  Although not specifically shown here, the two pixel groups are a horizontal pixel row of the first pixel group constituted by pixels of a plurality of rows of three or more rows and a second pixel group constituted of pixels of a plurality of rows of three or more rows. The horizontal pixel rows of the two pixel group may be arranged alternately in the vertical direction. That is, all the pixels of the image sensor 2 may be divided into two pixel groups for each of a plurality of rows.

(Third embodiment)
Next, an imaging apparatus 1 according to a third embodiment of the present invention will be described. The imaging device 1 of the present embodiment is different from the first embodiment in the arrangement pattern of the pixels of the two pixel groups. That is, the configuration and operation of the imaging apparatus 1 of the present embodiment are the same as the configuration of the first embodiment unless otherwise specified here.

  FIG. 9A is an explanatory diagram for explaining an arrangement pattern of pixels of two pixel groups in the present embodiment. As shown in FIG. 9A, two pixel groups are divided into a vertical pixel unit of a first pixel group composed of one pixel and a pixel unit of a second pixel group composed of a single pixel. It is alternately arranged in the direction and the horizontal direction. That is, all the pixels of the image sensor 2 are divided into two pixel groups like a checkered pattern.

  In the imaging apparatus 1 according to the third embodiment of the invention as described above, the same effects as those of the first embodiment can be obtained.

  In the present embodiment, the pixels of the two pixel groups (the first pixel group and the second pixel group) are alternately arranged in the vertical direction and the horizontal direction for each pixel unit. That is, the pixels of the two pixel groups are arranged without deviation in the vertical direction and the horizontal direction. Thereby, the balance of the resolution of two pixel groups in the vertical direction and the horizontal direction is improved.

  FIG. 9B shows a modified example of the arrangement pattern of the pixels of the two pixel groups. As shown in FIG. 9B, the two pixel groups include a pixel unit of a first pixel group configured by 2 rows × 2 columns of pixels and a second pixel configured by 2 rows × 2 columns of pixels. The pixel units of the group may be alternately arranged in the vertical direction and the horizontal direction.

  Although not specifically shown here, the two pixel groups are composed of a pixel unit of the first pixel group composed of a plurality of pixels of 3 rows × 3 columns or more and a plurality of pixels of 3 rows × 3 columns or more. The pixel units of the second pixel group thus formed may be alternately arranged in the vertical direction and the horizontal direction.

  The embodiments of the present invention have been described above by way of example, but the scope of the present invention is not limited to these embodiments, and can be changed or modified according to the purpose within the scope of the claims. is there.

  In the above description, the two separation circuits of the first separation circuit 5 and the second separation circuit 6 are used. That is, after the video signal obtained by the image sensor 2 is recorded in the frame memory 4, the video signal of the previous frame is separated by the second separation circuit 6 and the current video signal is separated by the first separation circuit 5. The case was illustrated. However, the scope of the present invention is not limited to this. For example, after separating a video signal using one separation circuit, a long exposure signal and a short exposure signal are recorded in the frame memory 4 and used as a long exposure signal and a short exposure signal one frame before. It may be used.

  In the above description, the motion detection circuit 9 detects the movement of the subject by comparing the current long exposure signal and short exposure signal with the long exposure signal and short exposure signal one frame before. The case of doing was illustrated. However, the scope of the present invention is not limited to this. For example, before the video signal is separated by the separation circuit, the current video signal and the video signal one frame before may be compared to detect the movement of the subject.

  As described above, the image pickup apparatus according to the present invention can obtain a high-resolution composite video signal with an expanded dynamic range, suppress the occurrence of motion blur and afterimages in the composite video signal, and increase the frame rate. And is useful as an imaging device used for a video camera or the like.

The block diagram of the imaging device in the 1st Embodiment of this invention Explanatory drawing of the pixel group in the 1st Embodiment of this invention Explanatory drawing for demonstrating operation | movement of the image pick-up element (CCD) in the 1st Embodiment of this invention. Explanatory drawing for demonstrating operation | movement of the two-dimensional pixel interpolation circuit in the 1st Embodiment of this invention Explanatory drawing for demonstrating operation | movement of the motion detection circuit in the 1st Embodiment of this invention. Explanatory drawing for demonstrating operation | movement of the imaging device in the 1st Embodiment of this invention Explanatory drawing of the modification of the pixel group in the 1st Embodiment of this invention (A) Explanatory drawing of pixel group in 2nd Embodiment of this invention (b) Explanatory drawing of the modification of the pixel group in 2nd Embodiment of this invention (A) Explanatory drawing of pixel group in 3rd Embodiment of this invention (b) Explanatory drawing of the modification of the pixel group in 3rd Embodiment of this invention

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Imaging device 2 Imaging element 3 Timing pulse generator 4 Frame memory 5 1st separation circuit 6 2nd separation circuit 7 1st 2D pixel interpolation circuit 8 2nd 2D pixel interpolation circuit 9 Motion detection circuit 10 1st signal for interpolation Generation circuit 11 Second interpolation signal generation circuit 12 First selection circuit 13 Second selection circuit 14 Synthesis circuit

Claims (7)

All the pixels are divided into two pixel groups, and each of the two pixel groups can read video signals having different exposure times within a predetermined frame period, and the exposure times of the two pixel groups are the predetermined frames. Imaging means to be replaced every period;
Recording means for recording the video signal obtained by the imaging means as a video signal before a predetermined frame period;
Separating means for separating the video signal obtained by the imaging means into a long exposure signal having a long exposure time and a short exposure signal having a short exposure time;
A motion detection means for detecting a motion of a subject based on a current video signal obtained by the imaging means and a video signal recorded in the recording means before a predetermined frame period;
First interpolation signal generation for generating an interpolation long-time exposure signal from the current long-time exposure signal obtained by the separating unit and the long-time exposure signal before a predetermined frame period in accordance with the degree of movement of the subject. Means,
Second interpolation signal generation for generating an interpolation short-time exposure signal from the current short-time exposure signal obtained by the separation unit and the short-time exposure signal before a predetermined frame period according to the degree of movement of the subject. Means,
First interpolation synthesis means for interpolating the current long exposure signal with the interpolation long exposure signal to generate a long exposure signal for all pixels;
Interpolating the current short-time exposure signal with the interpolation short-time exposure signal to generate a short-time exposure signal for all pixels;
Video signal synthesis means for synthesizing the long-time exposure signal for all the pixels and the short-time exposure signal for all the pixels to generate a synthesized video signal for all the pixels;
An imaging apparatus comprising: First interpolation pixel generating means for generating a long exposure signal of an interpolation pixel group corresponding to the other pixel group based on a current long exposure signal of one of the two pixel groups;
Second interpolation pixel generating means for generating a short-time exposure signal of an interpolation pixel group corresponding to the other pixel group based on a current short-time exposure signal of the one pixel group;
With
The first interpolation signal generation means includes:
In accordance with the degree of movement of the subject, the long exposure signal of the interpolation pixel group and the long exposure signal before the predetermined frame period are combined to generate a long exposure signal for interpolation,
The second interpolation signal generation means includes:
A short exposure signal for interpolation is generated by synthesizing the short exposure signal of the interpolation pixel group and the short exposure signal before the predetermined frame period in accordance with the degree of movement of the subject. The imaging device according to claim 1. The motion detection means includes
The long-time exposure signal of the interpolation pixel group and the long-time exposure signal before the predetermined frame period are compared, and the short-time exposure signal of the interpolation pixel group and the short-time exposure signal before the predetermined frame period are compared. The imaging apparatus according to claim 2, wherein the movement of the subject is detected. The motion detection means includes
Based on the current video signal obtained by the imaging means and the video signal before the predetermined frame period recorded in the recording means, a motion parameter indicating the amount of movement of the subject is determined,
The first interpolation signal generation means includes:
By applying a weighting process to the long exposure signal of the interpolation pixel group and the long exposure signal before the predetermined frame period using the motion parameter, the interpolation long exposure signal is generated,
The second interpolation signal generation means includes:
The interpolation short-time exposure signal is generated by weighting the short-time exposure signal of the interpolated pixel group and the short-time exposure signal before the predetermined frame period using the motion parameter. The imaging device according to any one of claims 1 to 3. In the imaging means,
The two pixel groups are formed by alternately arranging one or more vertical pixel columns constituting the one pixel group and one or more vertical pixel columns constituting the other pixel group in the horizontal direction. The imaging apparatus according to claim 1, wherein the imaging apparatus is provided. In the imaging means,
The two pixel groups are formed by alternately arranging one or a plurality of horizontal pixel rows constituting the one pixel group and one or a plurality of horizontal pixel rows constituting the other pixel group in the vertical direction. The imaging apparatus according to claim 1, wherein the imaging apparatus is provided. In the imaging means,
One or a plurality of pixel units constituting the one pixel group and one or a plurality of pixel units constituting the other pixel group are alternately arranged in the vertical direction and the horizontal direction, and the two pixel groups are The imaging apparatus according to claim 1, wherein the imaging apparatus is formed.

Images