JP4415196B2 - Imaging apparatus and imaging method - Google Patents

Description

The present invention relates to an imaging apparatus and an imaging method . Specifically, a digital low-pass filter is used on the image signal of the captured image, and a frame thinning process is performed on the image signal that has been subjected to filter processing in the time axis direction or filter processing to obtain an image signal having a desired frame period. Is.

  In a conventional movie production or the like, a film video camera is used. In this film video camera, as shown in FIG. 6, a mirror 81 is mounted at a position other than the open portion of a rotating disk-shaped shutter 80, and an imaging lens 70 and an optical viewfinder are placed on the optical axis of the mirror 81. A ground glass 82 is placed. Here, when the shutter 80 is in the open position, the photographed image is formed on the film 83 and the film 83 is exposed, and the photographed image display on the optical viewfinder is stopped. Further, when the shutter 80 is at the non-open position, the film 83 is moved forward, and the captured image is displayed on the ground glass 82 of the optical viewfinder by the mirror 81.

  Here, a special image effect can be obtained by adjusting the exposure time by adjusting the shutter opening time or changing the frame frame feed speed. For example, if an image shot at a high speed with a high frame feed rate is reproduced at a normal speed, a high-speed operation such as when a water droplet falls on the water surface can be observed easily and in detail. In addition, if an image shot at a low speed with a slow frame advance rate is played back at a normal speed, the realism of the fighting scene can be enhanced, and the sense of speed in a car chase can be enhanced.

  Also, in television program production, etc., digitization of shooting, editing, and transmission of programs has been attempted, but in the production of movies, etc. due to the increase in image quality and the lowering of equipment prices accompanying the advancement of digital technology. Digitization is in progress.

  Here, when shooting using a video camera using a solid-state imaging device such as a CCD (Charge Coupled Device), if the exposure time is controlled using the electronic shutter function of the solid-state imaging device, the shutter opening in the film video camera Can simulate time control. Further, by controlling the time required for reading a signal from the solid-state imaging device, it is possible to imitate the film feed time interval control in the film video camera. Furthermore, since the electronic viewfinder displays an image based on a signal read from the solid-state imaging device, an image that is easy to see can be displayed without being interrupted, unlike an optical viewfinder.

By the way, when it is assumed that low-speed shooting is performed, the solid-state image sensor performs exposure as shown in FIG. 7a by controlling the exposure time by using the electronic shutter function, and the time required for reading is shown in FIG. 7b. By making it longer, the time interval of the obtained images becomes longer, and a low-speed captured image can be obtained .

However, in a solid-state image sensor, it is known that a noise component called dark current is generated in the photosensor unit that performs photoelectric conversion. When the exposure time is increased, not only the image signal but also the dark current is proportional to the exposure time. And get bigger. For this reason, as shown in FIG. 8, as the exposure time increases, the margin of the dynamic range decreases. For example, when the luminance of the subject is high, the captured image signal is saturated and the high luminance portion is crushed. A captured image with good tone reproducibility cannot be obtained. Also, in order to increase the margin of the dynamic range, if the illumination is dimmed or the aperture of the photographic lens is adjusted to reduce the amount of light incident on the solid-state image sensor, only the photographic image signal is reduced without changing the dark current level. As a result, the S / N ratio is deteriorated. For this reason, it is impossible to obtain a photographed image with good image quality with less noise.

  Further, in the CCD, noise components such as dark current and smear generated in a transfer line for reading out electric charges generated by photoelectric conversion in the photosensor portion increase in proportion to the time required for signal reading. For this reason, if the time required for reading is increased, the dynamic range margin is further reduced due to these effects, and the S / N ratio is also deteriorated.

Therefore, the present invention provides an imaging apparatus and an imaging method capable of preventing a reduction in the margin of the dynamic range even during low-speed shooting and obtaining a good shot image without causing deterioration of the S / N ratio. It is.

An image pickup apparatus according to the present invention includes an image pickup unit that has an image pickup element and generates an image signal of a photographed image having a readout period equal to or longer than the shortest frame period in which the image pickup element can operate, and filter processing in the time axis direction of the image signal The digital low-pass filter that lowers the dynamic resolution of the image based on the image signal and the frame decimation process on the image signal obtained by performing the filter processing, and the image signal of the desired output frame period And a control means for controlling the image signal generation operation in the imaging means, the filter characteristics of the digital low-pass filter, the readout operation in the imaging means, and the frame decimation processing operation in the thinning processing means. the control means, when the output frame period is in the range of 2 times the shortest frame period, read out as a no thinning The cycle is varied, when the output frame period, (the n 2 or more positive integer) n of the shortest frame period times larger than, and the case of the shortest frame period of (n + 1) times or less, the readout from the imaging means The imaging means is controlled so that the period is 1 / n times the output frame period, and the thinning processing means is controlled so that the thinning processing means performs frame thinning processing every n frames.

In addition, the imaging method according to the present invention performs a first step of generating an image signal of a captured image having a readout period equal to or longer than the shortest frame period in which the image sensor can operate, and a filtering process in the time axis direction of the image signal. The second step of reducing the dynamic resolution of the image based on the image signal and the frame decimation process on the image signal obtained from the second step to generate an image signal having a desired output frame period 3 and the output frame period is within a range twice as long as the shortest frame period, the readout period is varied so that the thinning is not performed in the third step, and the output frame period is the shortest frame period. period (the n 2 or more positive integer) of n times larger than, and in the case of the shortest frame period of (n + 1) times or less is read in a first step The out period as the 1 / n times the output frame period, and performs frame skipping every n frames in the third step.

In the present invention, the image signal generated by the image pickup means is subjected to filter processing in the time axis direction using a digital low-pass filter, thereby reducing the dynamic resolution of the image and the coefficient of the digital low-pass filter. The amount of decrease in dynamic resolution can be adjusted by changing. In addition, when the output frame period is within the range of twice the shortest frame period for the image signal with reduced dynamic resolution, the readout period is varied so as not to perform decimation, and the output frame period is When it is greater than n (n is a positive integer greater than or equal to 2 ) times the shortest frame period in which the image sensor can operate and less than (n + 1) times the shortest frame period, the readout period is 1 / n times the output frame period Then, frame decimation is performed every n frames, and an image signal with a reduced dynamic resolution in the output frame period is output.

According to this invention, by performing filtering in the time axis direction with a digital low-pass filter on the image signal, the dynamic resolution of the image based on the image signal is reduced, and the filtered image signal is processed. When the output frame period is in the range of twice the shortest frame period, the readout period is varied so as not to perform decimation, and the output frame period is n (n is the shortest frame period at which the image sensor can operate). If it is greater than 2 (a positive integer) times and less than (n + 1) times the shortest frame period, the readout period is set to 1 / n times the output frame period, and frame decimation is performed every n frames, An image signal in which the dynamic resolution of the output frame period is lowered is output. For this reason, it is possible to suppress a decrease in the margin of the S / N ratio and the dynamic range.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 shows the configuration of a video camera. The light incident through the imaging lens 10 enters the imaging unit 11 and forms a subject image on the imaging surface. The imaging unit 11 is configured using a solid-state imaging device such as a CCD, and reads out a signal based on a subject image obtained by photoelectric conversion based on a driving signal RC from a driving unit 52 to be described later, and picks up three primary color imaging signals of the captured image. Spa is generated and supplied to the preprocessing unit 12.

  The preprocessing unit 12 performs processing for removing noise components from the imaging signal Spa, for example, correlated double sampling processing, and supplies the noise-removed imaging signal Spa to the A / D conversion unit 13 as the imaging signal Spb. The A / D conversion unit 13 converts the image pickup signal Spb into a digital image signal DVa and supplies the digital image signal DVa to the feedback clamp unit 14. The feedback clamp unit 14 detects an error between the black level signal during the blanking period and the reference signal and supplies it to the A / D conversion unit 13 to obtain an image signal DVa having a required magnitude at a stable black level. The A / D conversion operation is controlled so that it can be performed. The correction processing unit 15 performs shading correction, correction processing for defects in the image sensor, and the like on the image signal DVa. The image signal DVa subjected to the correction processing by the correction processing unit 15 is supplied to the speed correspondence processing unit 20 as the image signal DVb.

  FIG. 2 shows a configuration of the speed correspondence processing unit 20. The image signal DVb supplied from the correction processing unit 15 is supplied to the subtractor 211 and adder 213 of the digital low-pass filter 21 and the terminal a of the switch 23. The subtracter 211 is supplied with an image signal DVf from a 1-frame delay circuit 214 described later, and subtracts the image signal DVb from the image signal DVf. The subtraction signal DVc obtained by the subtractor 211 is supplied to the multiplier 212.

  The multiplier 212 is supplied with a coefficient k for adjusting the filter characteristics so as to obtain the same effect as the adjustment of the exposure time length from the control unit 50, and multiplies the subtraction signal DVc by the coefficient k. The multiplication signal DVd obtained by the multiplier 212 is supplied to the adder 213.

  The adder 213 adds the image signal DVb supplied from the correction processing unit 15 and the multiplication signal DVd supplied from the multiplier 212 to generate an image signal DVe. The image signal DVe obtained by the adder 213 is supplied to the 1-frame delay circuit 214, the frame thinning circuit 22, and the terminal b of the switch 23.

  The 1-frame delay circuit 214 delays the image signal DVe by 1 frame and supplies it to the subtractor 211 as the image signal DVf. The frame decimation circuit 22 performs frame decimation based on the decimation control signal CTa from the control unit 50, generates an image signal DVg having a desired number of frames, and supplies it to the main line side signal processing unit 31 in FIG. The movable terminal c of the switch 23 is connected to the monitor-side signal processing unit 41 of FIG. 1, and selects either the image signal DVb or the image signal DVe based on the control signal CTb from the control unit 50 to select the image signal. It is supplied to the monitor side signal processor 41 as DVh.

  The main line side signal processing unit 31 performs γ processing and contour compensation processing on the video camera output on the image signal DVg, and also performs processing such as knee correction processing. The image signal DVg subjected to this signal processing is supplied to the main line side signal output processing unit 32 as an image signal DVj.

  The main line side signal output processing unit 32 converts the image signal DVj into a signal CMout corresponding to a device connected to the video camera and outputs the signal CMout. For example, when a device corresponding to a component signal or a device corresponding to a composite signal is connected, the signal is converted into a signal CMout corresponding to each device and output. When an image signal is transmitted through a serial digital interface or the like standardized as SMPTE292M, a transmission signal corresponding to the interface standard is generated and output as a signal CMout.

  The monitor-side signal processing unit 41 performs γ processing, contour compensation processing, and the like on the image signal DVh, and performs processing such as knee correction processing. Here, the monitor-side signal processing unit 41 performs signal processing according to the image display device used to monitor the captured image. For example, the γ correction characteristic, the knee correction characteristic, the contour compensation characteristic, and the like are switched depending on whether the image display device is configured using a cathode ray tube or a liquid crystal display element. . The image signal DVh subjected to signal processing by the monitor side signal processing unit 41 is supplied to the monitor side signal output processing unit 42 as an image signal DVk.

  The monitor-side signal output processor 42 converts the supplied image signal DVk into a signal MTout corresponding to the image display device for captured image monitoring and outputs the signal MTout. For example, when the image display device uses an analog signal, the image signal DVk is converted into an analog signal and output as a signal MTout.

  An operation unit 51 is connected to the control unit 50, and when the user of the video camera operates the operation unit 51, an operation signal PS corresponding to the user operation is supplied from the operation unit 51 to the control unit 50. The control unit 50 generates various control signals CT based on the operation signal PS and controls the operation of each unit, thereby operating the video camera in accordance with the user's operation. In addition, when the photographing unit or the shutter opening time is switched by the operation unit 51, the control unit 50 supplies the coefficient k corresponding to the switched shutter opening time to the speed correspondence processing unit 20 to switch the switched shutter. The speed correspondence processing unit 20 generates an image signal of an image whose dynamic resolution is lowered according to the opening time. Further, a control signal TC for setting a signal readout frame period in the imaging unit 11 is generated according to the switched imaging speed and supplied to the driving unit 52, and the control signals CTa and CTb are controlled according to the switched imaging speed. Is generated and supplied to the speed corresponding processing unit 20. The drive unit 52 generates a drive signal RC based on the control unit TC and supplies the drive signal RC to the imaging unit 11 to read out the signal at a set frame period. The speed correspondence processing unit 20 controls the control signals CTa, Based on CTb, frame thinning and signal selection are performed, and an image signal corresponding to the imaging speed switched from the main line side signal output processing unit 32 is output, and the monitor side signal output processing unit 42 performs user operations. A corresponding image signal is output.

  Next, the operation will be described. In the speed correspondence processing unit 20 shown in FIG. 2, the relationship between the image signal DVb and the image signal DVe obtained by performing the filter processing in the time axis direction with the digital low-pass filter 21 is as shown in Expression (1), and the amplitude characteristic When the amplitude at the angular frequency ω = 0 is normalized to “1”, the angular frequency ω is expressed by Equation (2). When the frame period of the image signal DVb is “T”, the angular frequency ω is “ω = π / T”.

  When the step response is obtained by varying the coefficient k within the range of “0” to “1” in the filter characteristics of the digital low-pass filter 21 of the speed corresponding processing unit 20, the characteristics shown in FIG. As it becomes larger, the response becomes slower. In addition, as shown in FIG. 4, the frequency characteristics are also narrowed as the coefficient k increases. That is, as the coefficient k increases, the image based on the image signal DVe increases afterimages and loses high-frequency components, resulting in an image with a reduced dynamic resolution similar to the case where the exposure time is increased. That is, by adjusting the filter characteristics of the digital low-pass filter 21 and performing the filter processing of the image signal DVb, an image similar to the photographed image when the shutter opening time is adjusted to a desired exposure time is obtained. A signal can be generated. That is, the shutter opening time adjustment can be imitated by changing the coefficient k of the digital low-pass filter 21.

  For this reason, during low-speed shooting, the coefficient k of the digital low-pass filter 21 is changed to reduce the moving image so that the image signal DVe of the same image as that obtained when the exposure time is lengthened and the moving resolution is lowered can be obtained. Adjust the amount.

  Further, the frame thinning circuit 22 performs frame thinning on the image signal DVe according to the shooting speed. The frame decimation circuit 22 performs frame decimation and adjusts the decimation number to obtain an image signal having a desired frame period. That is, the frame number thinning circuit 22 can imitate the frame number variable function.

  In this way, by performing frame decimation in the frame decimation circuit 22 and adjusting the decimation number, the image from the frame decimation circuit 22 is the same as that when the exposure time is long and the dynamic resolution is lowered, and a desired frame period is obtained. Image signal DVg can be output.

  Further, if the movable terminal c of the switch 23 is set to the terminal a and the image signal DVb is supplied to the image display device, an image in which the dynamic resolution is not reduced and the frame is not thinned can be displayed. It is possible to perform low-speed shooting by following correctly. Furthermore, if the movable terminal c is set to the terminal b and the image signal DVe is supplied to the image display device, it can be confirmed how much the dynamic resolution is reduced by the display image of the image display device.

  For example, as shown in FIG. 5, the solid-state imaging device of the imaging unit 11 performs exposure and signal readout at the frame period Fa as shown in FIGS. 5A and 5B, and the frame thinning circuit 22 of the speed correspondence processing unit 20 Assuming that 3 frames are thinned out per 4 frames as shown in 5C, the image signal DVg supplied to the main line side signal processing unit 31 has a frame period of “Fa / 4” and is exposed based on the coefficient k. This is an image signal of an image with reduced dynamic resolution similar to the case where the time is increased.

  When the image signal DVb is selected with the movable terminal c of the switch 23 as the terminal a side, the image display device for the image monitor has a frame period of “Fa” and a decrease in dynamic resolution as shown in FIG. 5D. Undisplayed images can be displayed. Further, if the image signal DVe is selected with the movable terminal c of the switch 23 as the terminal b side, how much the image is based on the image signal DVe having the frame period “Fa” and supplied to the main line side signal processing unit 31. You can check if the dynamic resolution is degraded.

  When low-speed shooting is not performed, the coefficient k is set to “0” and the frame thinning circuit 22 does not perform thinning. At this time, since the image signal DVb and the image signal DVe are equal and no thinning is performed, the image signal DVb is supplied to the main line side signal processing unit 31 and the monitor side signal processing unit 41, and a predetermined frame is obtained. A rate image signal CMout can be output from the video camera. In addition, an image whose dynamic resolution is not reduced can be displayed on the image display device. Furthermore, if the electronic shutter function is activated by controlling the signal readout operation from the solid-state imaging device, the exposure time can be shortened, so that high-speed shooting is possible.

  By the way, in the video camera described above, the signal CMout having a desired frame rate is generated by changing the number of thinnings in the frame thinning circuit 22. However, the image pickup unit is not only based on the frame thinning number but also based on the control signal TC. It is also possible to generate a signal with a desired frame rate by varying the signal readout frame period at 11.

In this case, switching of the thinning-out number and switching of the signal readout frame period at the imaging unit 11 are performed so that the signal readout frame period at the imaging unit 11 does not exceed a predetermined multiple of the shortest frame period at which the imaging element can operate. Do. For example, when the frame period (hereinafter referred to as “output frame period”) To of the signal CMout is within a range twice the shortest frame period Tmin that can be operated by the image sensor (Tmin <To ≦ 2Tmin), decimation is not performed. The signal readout frame period Ti in the imaging unit 11 is varied to generate a signal with a desired frame rate. Next, when the output frame period To is more than twice the shortest frame period Tmin and within three times (2Tmin <To ≦ 3Tmin), the signal readout frame period Ti is halved to thin out every two frames. Do. When the output frame period To exceeds 3 times the shortest frame period Tmin and is within 4 times (3Tmin <To ≦ 4Tmin), the signal readout frame period Ti is multiplied by 1/3 and thinning is performed every three frames. Similarly, when the output frame period To exceeds n (n is a positive integer equal to or greater than 2) times the shortest frame period Tmin and is within (n + 1) times (nTmin <To ≦ (n + 1) Tmin), If the readout frame period Ti is multiplied by 1 / n and thinning is performed every n frames, the signal readout frame period Ti can always be kept short.

  In this way, even if the output frame period To is lengthened, the signal readout frame period Ti does not become longer than the predetermined period. Therefore, the output frame period To becomes longer and the influence of dark current, smear, etc. in the imaging unit 11 is also caused. Without becoming large, it is possible to suppress the deterioration of the S / N ratio of the signal from the imaging unit 11 and the decrease in the margin of the dynamic range in the imaging unit 11. Further, the number of frames to be thinned out can be reduced by increasing the signal readout frame period. Further, when the image signal DVb is selected with the movable terminal c of the switch 23 as the terminal a side in order to monitor the captured image with the electronic viewfinder, the frame period of the image signal DVb does not become longer than a predetermined period. Therefore, it is possible to suppress a decrease in dynamic resolution.

It is a figure which shows the structure of a video camera. It is a figure which shows the structure of a speed corresponding | compatible process part. It is a figure which shows a step response characteristic. It is a figure which shows a frequency characteristic. It is a figure which shows the operation | movement by low speed imaging | photography. It is a figure which shows operation | movement of a film video camera. It is a figure which shows the case where low-speed imaging is performed by lengthening read-out time. It is a figure which shows the dynamic range margin in short time exposure and long time exposure.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10,70 ... Imaging lens, 11 ... Imaging part, 12 ... Pre-processing part, 13 ... A / D conversion part, 14 ... Feedback clamp part, 15 ... Correction processing part, DESCRIPTION OF SYMBOLS 20 ... Speed corresponding | compatible process part, 21 ... Digital low-pass filter, 22 ... Frame thinning circuit, 23 ... Switch, 31 ... Main line side signal processing part, 32 ... Main line side signal output Processing unit 41 ... Monitor side signal processing unit 42 ... Monitor side signal output processing unit 50 ... Control unit 51 ... Operation unit 52 ... Drive unit 80 ... Shutter , 81 ... Mirror, 82 ... Ground glass, 83 ... Film, 211 ... Subtractor, 212 ... Multiplier, 213 ... Adder, 214 ... Frame delay circuit

Claims (4)

An image pickup unit having an image pickup device and generating an image signal of a shot image having a read cycle equal to or longer than the shortest frame cycle in which the image pickup device can operate;
A digital low-pass filter that reduces the dynamic resolution of an image based on the image signal by performing filtering in the time axis direction of the image signal;
A decimation processing means for performing a frame decimation process on the image signal obtained by performing the filtering process, and generating an image signal having a desired output frame period;
A control means for controlling an image signal generation operation in the imaging means, a filter characteristic of the digital low-pass filter, a readout operation in the imaging means, and a frame decimation processing operation in the decimation processing means;
When the output frame period is in a range twice as long as the shortest frame period , the control unit varies the read period so that no decimation is performed, and the output frame period is n (the shortest frame period). When n is a positive integer greater than or equal to 2 and less than (n + 1) times the shortest frame period, the readout period from the imaging means is 1 / n times the output frame period. controlling said imaging means, that controls the thinning processing unit so that said thinning processing means performs frame skipping process for every n frames
Imaging device. The digital low-pass filter has one frame delay means,
When the image signal is “DVb” and the output of the one-frame delay means is “DVf”,
Shall be the output of the digital low pass filter receives an input to the one-frame delay means the operation result of "(1-k) × DVb + k × DVf " (k is a coefficient in the range of 0 to 1)
請 Motomeko 1 imaging device according. A first step of generating an image signal of a captured image having a readout period equal to or longer than the shortest frame period in which the image sensor can operate;
A second step of reducing the dynamic resolution of the image based on the image signal by performing time-axis filtering of the image signal;
A third step of performing frame thinning processing on the image signal obtained in the second step to generate an image signal having a desired output frame period;
When the output frame period is in a range twice as long as the shortest frame period, the readout period is varied so that thinning is not performed in the third step, and the output frame period is n (the shortest frame period). If n is a positive integer greater than or equal to 2 ) and less than or equal to (n + 1) times the shortest frame period, the read period is set to 1 / n times the output frame period in the first step; Let row frame skipping every n frames in the third step
An imaging method. In the second step, 1-frame delay means is used,
When the image signal is “DVb” and the output of the one-frame delay means is “DVf”,
Shall be the output of the second step as well as input to the one-frame delay means the operation result of "(1-k) × DVb + k × DVf " (k is a coefficient in the range of 0 to 1)
請 Motomeko 3 imaging method according.

Images