JP2008072448A - Imaging apparatus, imaging method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus capable of correcting an out-of-focus photographed image by camera shake during photographing without requiring an image buffer memory, without causing sense of incongruity and with little noise. <P>SOLUTION: The imaging apparatus is provided with an amplifier circuit 11 for amplifying a video signal output from an imaging element 10, a correction signal generation circuit 13 for generating a correction signal based on a video signal and camera shake track when a photographed image represented by the video signal is photographed, a correction signal shaping circuit 14 for shaping the correction signal by coring, and a video signal correction circuit 15 for using the correction signal shaped by the correction signal shaping circuit 14 to correct the video signal, and a control circuit 19 for setting a coring threshold in accordance with the gain of the amplifier circuit 11. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an imaging apparatus, an imaging method, and a program that correct a captured image that is blurred due to blurring during imaging.

  Conventionally, as an imaging device that corrects a blurred image due to camera shake, a plurality of frames of captured images are continuously captured to detect the amount of deviation between the captured images, and the amount of deviation is equivalent to each captured image. In some cases, a corrected image of one frame is generated after being corrected (see, for example, Patent Document 1).

  As shown in FIG. 10, a conventional imaging device 50 includes an imaging device 51, an amplification circuit 52 that amplifies the video signal output from the imaging device 51, and the video signal output from the amplification circuit 52 as digital image data. An A / D conversion circuit 53 for conversion, a signal processing circuit 54 for performing white balance processing and interpolation processing on digital image data, a switch 55, an image buffer memory 56 for storing digital image data, and digital image data A shift amount detection circuit 57 that detects a shift amount between captured images and a CPU (Central Processing Unit) 58 are provided.

  The switch 55 is configured to switch the output destination of the digital image data so that the digital image data of each frame photographed continuously is stored in different memory areas of the image buffer memory 56.

  The shift amount detection circuit 57 detects the shift amount between the captured image represented by the digital image data of the first captured frame and the captured image represented by the digital image data of each captured frame. .

  The CPU 58 corrects each digital image data based on the shift amount of the captured image of each frame detected by the shift amount detection circuit 57, and converts the corrected digital image data into the digital image data of the first captured frame. The synthesized digital image data is output.

  Here, as shown in FIG. 11A, it is assumed that the optical axis of the imaging device 50 has moved from L1 to L5 during t during the imaging of the imaging device 50. Accordingly, in the image sensor 51, as shown in FIG. 11B, the photographed captured image is blurred by the blur amount W1.

  Here, as shown in FIG. 11C, the imaging device 50 exposes the imaging element 51 at intervals of t / 4 while the optical axis moves from L1 to L5, and the optical axis changes from L1 to L2. Four shots are taken while moving, from L2 to L3, from L3 to L4, and from L4 to L5.

  As a result, as shown in FIGS. 11D to 11G, the blur amount W2 of the captured image at each exposure is suppressed to approximately ¼ of W1.

  The digital image data converted by the A / D conversion circuit 53 from the video signal of each frame amplified in this way is stored in each memory area of the image buffer memory 56 and each frame detected by the shift amount detection circuit 57. The CPU 58 synthesizes the digital image data of the first frame so that the amount of deviation between the captured images is eliminated.

As described above, the conventional imaging device 50 corrects a captured image blurred by blurring during shooting while suppressing noise by combining images.
JP 2004-266648 A

  However, the above-described conventional imaging device has a problem that an image buffer memory is necessary because a plurality of captured images are stored in a memory and then corrected.

  In addition, when there is a subject that moves within a shooting range between a plurality of frames, the conventional imaging device shifts the positional relationship between the subject and the background between the shot images of each frame, thereby Since the boundary between the subject of the photographed image and the background is shifted four times and synthesized, there is a problem that the corrected image is uncomfortable.

  The present invention has been made in order to solve the conventional problems, and does not require an image buffer memory, and can correct a captured image blurred by blurring during shooting without causing a sense of incongruity and reducing noise. An object is to provide an imaging method and a program.

  An imaging apparatus according to the present invention includes an amplifying unit that amplifies a video signal output from an imaging element, and a correction that generates a correction signal based on the video signal and a blur locus when a captured image represented by the video signal is captured. A signal generation unit; a correction signal shaping unit that shapes the correction signal by coring; a video signal correction unit that corrects the video signal using the correction signal shaped by the correction signal shaping unit; and the coring And a threshold value setting means for setting the threshold value according to the gain of the amplifying means.

  With this configuration, the image pickup apparatus of the present invention corrects the video signal using the correction signal shaped based on the threshold value corresponding to the gain of the amplifying unit, so that it is not necessary to use an image buffer memory, and the image pickup device is free from image blur. A blurred photographed image can be corrected without causing a sense of incongruity and with little noise.

  The threshold value setting means may set the coring threshold value larger as the gain of the amplification means increases.

  The image pickup apparatus of the present invention further includes a light / dark ratio detection unit that detects a light / dark ratio of a captured image represented by the video signal, and the threshold setting unit is configured to reduce the ratio detected by the light / dark ratio detection unit. The coring threshold may be set large.

  With this configuration, the imaging device according to the present invention changes the coring threshold value between a contour portion having a high contrast ratio that is visually noticeable and a portion having a flat contrast that is not visually noticeable, thereby sharpening the contour portion. Since the degree of noise is increased and noise in a portion with a flat gradation is reduced, an image with good image quality can be obtained.

  The image pickup apparatus of the present invention has a light / dark ratio detecting means for detecting a light / dark ratio of a photographed image represented by the video signal output from the image sensor, and the video signal and the photographed image represented by the video signal are photographed. Correction signal generating means for generating a correction signal based on a blur locus, correction signal shaping means for shaping the correction signal by coring, and the video signal using the correction signal shaped by the correction signal shaping means. And a threshold value setting means for setting the coring threshold according to the ratio detected by the light / dark ratio detection means.

  With this configuration, the image pickup apparatus of the present invention corrects the video signal using the correction signal shaped based on the threshold value corresponding to the light / dark ratio of the photographed image. A photographed image blurred by blurring can be corrected with less noise and less noise.

  The threshold setting means may set the coring threshold to a larger value as the ratio detected by the light / dark ratio detection means becomes smaller.

  With this configuration, the imaging apparatus of the present invention can increase the sharpness of the contour portion and reduce noise in a portion with a flat gradation, so that an image with good image quality can be obtained.

  Further, in order to solve the conventional problems, the present invention provides, in addition to the above-described imaging apparatus, an imaging method for realizing the function of the imaging apparatus, a program for causing a computer to execute the function of the imaging apparatus, and A recording medium recording this program is provided.

  The present invention can provide an imaging apparatus, an imaging method, and a program that can correct a captured image that is blurred due to blurring at the time of shooting without causing a sense of incongruity and with less noise, without requiring an image buffer memory.

  Embodiments of the present invention will be described below with reference to the drawings.

(Embodiment 1)
FIG. 1 shows a block diagram of the imaging apparatus according to Embodiment 1 of the present invention.

  In FIG. 1, an imaging apparatus 1 includes an imaging device 10, an amplification circuit 11 that amplifies a minute video signal output from the imaging device 10, and an A / D conversion that digitizes the video signal output from the amplification circuit 11. A circuit 12, a correction signal generation circuit 13 that generates a correction signal for correcting the video signal, a correction signal shaping circuit 14 that shapes the correction signal, and a video signal correction circuit 15 that corrects the video signal using the correction signal. A signal processing circuit 16 that performs signal processing such as gamma correction processing, edge enhancement processing, and interpolation processing on the corrected video signal, an image display unit 17 that displays an image represented by the video signal subjected to the signal processing, A blur locus detection circuit 18 that detects a blur locus of the image sensor 10 and a control circuit 19 that sets a gain and a threshold value that are referred to by the correction signal shaping circuit 14. Eteiru.

  The blur locus detection circuit 18 includes, for example, an angular velocity sensor, and detects the blur locus of the image sensor 10 based on the angular velocity detected by the angular velocity sensor.

  The correction signal generation circuit 13 derives a video signal correction function based on the blur trajectory detected by the blur trajectory detection circuit 18, and based on the correction function using the video signal digitized by the A / D conversion circuit 12 as a variable. Thus, a correction signal is generated.

  The correction signal shaping circuit 14 includes, for example, a coring filter, and suppresses noise having a level lower than the threshold set by the control circuit 19 among noises included in the correction signal.

  The video signal correction circuit 15 corrects the video signal by adding the video signal digitized by the A / D conversion circuit 12 and the correction signal shaped by the correction signal shaping circuit 14. .

  FIG. 2 is a conceptual diagram illustrating an example of signals in each unit of the imaging apparatus 1. In each signal shown in FIG. 2, the horizontal axis corresponds to the pixel position in the captured image, and the vertical axis corresponds to the value of the signal.

  A video signal representing a photographed image photographed by the image sensor 10 without blurring is as shown in FIG. On the other hand, when there is a shake at the time of shooting, as shown in FIG. 2B, the video signal representing the shot image is inclined.

  When there is a blur at the time of shooting, a high-frequency component such as a contour portion in the shot image is attenuated. Therefore, the correction signal generation circuit 13 generates a correction signal for restoring the high-frequency component of the video signal when added to the video signal. It is designed to generate.

  Therefore, as shown in FIG. 2C, the high-frequency component of the correction signal increases, and the noise included in the correction signal increases. Further, when the video signal is corrected by adding the correction signal in this state to the video signal, the noise included in the corrected video signal increases as shown in FIG.

  In particular, when the shooting location is dark and the captured image needs to be brightened, the control circuit 19 sets the gain of the amplifier circuit 11 to be large, so that noise is also amplified. The image quality is significantly deteriorated.

  Therefore, as shown in FIG. 3, when the gain of the amplifier circuit 11 is set to g1 and the coring threshold value by the correction signal shaping circuit 14 is set to TH1, the control circuit 19 When the gain is set to g2 larger than g1, the coring threshold value by the correction signal shaping circuit 14 is set to TH2 larger than TH1.

  As a result, as shown in FIG. 2 (e), noise included in the correction signal shaped by the correction signal shaping circuit 14 is reduced and corrected by the video signal correction circuit 15 as shown in FIG. 2 (f). Noise contained in the video signal is also reduced.

  The operation of the imaging apparatus 1 configured as described above will be described with reference to FIG.

  First, the gain of the amplifier circuit 11 is set by the control circuit 19 (S1), and when shooting is performed (S2), the blur locus of the image sensor 10 while the image sensor 10 is exposed is detected by the blur locus detector circuit 18. It is detected (S3).

  Next, a correction function is derived by the correction signal generation circuit 13 on the basis of the blur locus detected by the blur locus detection circuit 18 (S4), and the correction is performed using the video signal digitized by the A / D conversion circuit 12 as a variable. A correction signal is generated by the correction signal generation circuit 13 based on the function (S5).

  Next, as described with reference to FIG. 3, the threshold value for coring by the correction signal shaping circuit 14 is set by the control circuit 19 in accordance with the gain of the amplifier circuit 11 (S6). Here, as the gain of the amplifier circuit 11 is set larger, the coring threshold value by the correction signal shaping circuit 14 is set larger.

  The correction signal generated by the correction signal generation circuit 13 is shaped by coring by the correction signal shaping circuit 14 having a threshold value set by the control circuit 19 (S7), and digitized by the A / D conversion circuit 12 It is added to the video signal (S8).

  The video signal is subjected to signal processing by the signal processing circuit 16 (S9) and displayed as an image on the image display unit 17 (S10).

  Since the imaging apparatus 1 according to the first embodiment of the present invention corrects a video signal using a correction signal shaped based on a threshold corresponding to the gain of the amplifier circuit 11, it does not require an image buffer memory. It is possible to correct a photographed image blurred by blurring during photographing without causing a sense of incongruity and with less noise.

  In the present embodiment, the control circuit 19 constitutes threshold setting means in the present invention.

  Further, the A / D conversion circuit 12, the correction signal generation circuit 13, the correction signal shaping circuit 14, the video signal correction circuit 15, the signal processing circuit 16, and the control circuit 19 execute a program in which the operation of the imaging device 1 described above is described. You may comprise integrally by processors, such as DSP (Digital Signal Processor) to perform.

  In FIG. 3, the relationship between the coring threshold value TH and the gain g of the amplifier circuit 11 is shown as a linear function, but the present invention is not limited to this.

  Further, although the video signal correction circuit 15 has been described as correcting the video signal by adding the video signal and the correction signal, the present invention is not limited to this.

  The blur locus detection circuit 18 has been described as detecting the blur locus of the image sensor 10 based on the angular velocity detected by the angular velocity sensor. However, the present invention is not limited to this, and for example, image processing is performed from a captured image. Thus, the blur locus of the image sensor 10 may be detected.

(Embodiment 2)
FIG. 5 shows a block diagram of the imaging apparatus according to Embodiment 2 of the present invention. In the present embodiment, the same components as those of the imaging device 1 according to Embodiment 1 of the present invention are denoted by the same reference numerals, and description thereof is omitted.

  In FIG. 5, an imaging device 21 includes an imaging device 10, an amplifier circuit 11, an A / D conversion circuit 12, a correction signal generation circuit 13, a correction signal shaping circuit 14, a video signal correction circuit 15, and signal processing. The light / dark ratio (hereinafter referred to as “light / dark ratio”) of each pixel of the captured image represented by the video signal output from the circuit 16, the image display unit 17, the blur locus detection circuit 18, and the A / D conversion circuit 12. And a light / dark ratio detection circuit 29 for detection.

  FIG. 6 is a conceptual diagram showing a region of a captured image that is referred to when the light / dark ratio detection circuit 29 detects the light / dark ratio, and shows an area of 3 horizontal pixels × 3 vertical lines as an example.

  In FIG. 6, a to i indicate pixel values of 3 pixels × 3 lines. Since the video signal output from the A / D conversion circuit 12 represents a horizontal and vertical two-dimensional image, in this embodiment, the light / dark ratio detection circuit 29 sets the range of 3 pixels × 3 lines horizontally and vertically. The pixel is moved pixel by pixel to detect the light / dark ratio for all pixels.

  For example, the light / dark ratio can be expressed by the following formula (1).

Light / dark ratio = (maximum value of pixel values a to i) / (minimum value of pixel values a to i) Expression (1)
If the light / dark ratio is larger than a predetermined threshold, the center of the 3 pixel × 3 line area (pixel value e) is a contour, and if the light / dark ratio is small, the area has no light / dark difference and the density is flat. It can be judged that there is.

  The light / dark ratio detection circuit 29 calculates a coring threshold value corresponding to the detected light / dark ratio, and sets the calculated threshold value in the correction signal shaping circuit 14. Here, the light / dark ratio detection circuit 29 is configured to increase the threshold value as the detected light / dark ratio decreases.

  FIG. 7 is a conceptual diagram illustrating an example of a signal in each unit of the imaging device 21. In each signal shown in FIG. 7, the horizontal axis corresponds to the pixel position in the captured image, and the vertical axis corresponds to the value of the signal.

  A video signal representing a photographed image taken by the image sensor 10 without blurring is as shown in FIG. On the other hand, when there is a shake at the time of shooting, as shown in FIG. 7B, the video signal representing the shot image is inclined.

  When there is a blur at the time of shooting, a high frequency component such as a contour portion in the shot image is attenuated. Therefore, the correction signal generation circuit 13 generates a correction signal for restoring the high frequency component of the video signal when added to the video signal. It is designed to generate.

  Therefore, as shown in FIG. 7C, the high-frequency component of the correction signal increases, and the noise included in the correction signal increases. Further, when the correction signal in this state is added to the video signal to correct the video signal, as shown in FIG. 7D, noise included in the corrected video signal also increases. In particular, when the gain of the amplifier circuit 11 is high, noise is further amplified, so that the image quality of the image represented by the corrected video signal is significantly deteriorated.

  Therefore, as shown in FIG. 8, the light / dark ratio detection circuit 29 sets the correction signal shaping circuit 14 when the detected light / dark ratio is R1, that is, when it is determined that the light / dark ratio is small and the density is flat. Focusing on noise reduction by increasing the threshold, and when the light / dark ratio is R2, that is, when the light / dark ratio is large and the contour is judged to be a contour, blur correction is performed by setting a small threshold to the correction signal shaping circuit 14. The emphasis is on.

  As described above, the light / dark ratio detection circuit 29 corrects the correction signal according to the light / dark ratio detected while shifting the area of 3 pixels × 3 lines as shown in FIG. 6 pixel by pixel over the entire area of the captured image. The threshold of coring by the circuit 14 is calculated.

  As a result, as shown in FIG. 7E, the noise in the flat portion of the correction signal shaped by the correction signal shaping circuit 14 is reduced, and as shown in FIG. In the video signal corrected by the circuit 15, noise included in the video signal corresponding to a flat portion where the density is not affected by the blur is also reduced.

  The operation of the imaging device 21 configured as described above will be described with reference to FIG.

  First, when shooting is performed (S21), the blur locus of the image sensor 10 while the image sensor 10 is exposed is detected by the blur locus detection circuit 18 (S22). Further, the light / dark ratio detection circuit 29 detects the light / dark ratio of each pixel of the photographed image (S23).

  Next, a correction function is derived by the correction signal generation circuit 13 based on the shake locus detected by the shake locus detection circuit 18 (S24), and the correction is performed using the video signal digitized by the A / D conversion circuit 12 as a variable. A correction signal is generated by the correction signal generation circuit 13 based on the function (S25).

  Next, the coring threshold value by the correction signal shaping circuit 14 is set according to the light / dark ratio by the light / dark ratio detection circuit 29 as described with reference to FIG. 8 (S26). Here, as the contrast ratio decreases, the coring threshold value by the correction signal shaping circuit 14 is set larger.

  The correction signal generated by the correction signal generation circuit 13 is shaped by coring by the correction signal shaping circuit 14 having a threshold value set by the light / dark ratio detection circuit 29 (S27) and digitized by the A / D conversion circuit 12. The added video signal is added (S28).

  The video signal is subjected to signal processing by the signal processing circuit 16 (S29) and displayed as an image on the image display unit 17 (S30).

  Since the imaging device 21 according to the second embodiment of the present invention corrects a video signal using a correction signal shaped based on a threshold value corresponding to the light / dark ratio detected by the light / dark ratio detection circuit 29, the image Without requiring a buffer memory, it is possible to correct a captured image that is blurred due to blurring during shooting without causing a sense of incongruity and with less noise.

  In addition, the imaging device 21 increases the sharpness of the contour portion by changing the coring threshold value between a contour portion having a large contrast ratio that is visually conspicuous and a portion having a flat contrast that is not visually conspicuous. In order to reduce noise in a portion with a flat gradation, an image with good image quality can be obtained.

  The A / D conversion circuit 12, the correction signal generation circuit 13, the correction signal shaping circuit 14, the video signal correction circuit 15, the signal processing circuit 16, and the light / dark ratio detection circuit 29 describe the operation of the imaging device 21 described above. You may comprise integrally by processors, such as DSP which performs a program.

  The imaging device 21 further includes the control circuit 19 described in the first embodiment of the present invention, and when the gain g of the amplifier circuit 11 is large and the light / dark ratio R of the photographed image is small, the coring threshold value. You may comprise so that TH may be set large.

  With this configuration, the imaging device 21 performs image stabilization in a dark environment, and even when it is desired to brighten a captured image, the amount of correction is particularly small in a portion with a light gray level. Reduction performance can be improved.

  In FIG. 6, the light / dark ratio detection circuit 29 has been described as detecting the light / dark ratio of each pixel with reference to an area of horizontal 3 pixels × vertical 3 lines, but is limited to the size of this area. There is no. In addition, the light / dark ratio detection circuit 29 has been described as detecting the light / dark ratio based on the above-described equation (1), but the present invention is not limited to this equation, and the light / dark ratio is detected based on another equation. It may be.

  In FIG. 8, the relationship between the coring threshold value TH and the light / dark ratio R is shown by a linear function, but the present invention is not limited to this.

  As described above, the imaging apparatus according to the present invention has an effect that a captured image blurred due to blurring at the time of shooting can be corrected with less noise and no noise, without requiring an image buffer memory. For example, it is useful as an imaging device that corrects a photographed image that is blurred due to camera shake.

1 is a block diagram of an imaging apparatus according to Embodiment 1 of the present invention. FIG. 2 is a conceptual diagram illustrating an example of a signal of each part of the imaging apparatus according to Embodiment 1 of the present invention. 6 is a graph showing the relationship between the coring threshold set in the correction signal shaping circuit of the imaging apparatus according to Embodiment 1 of the present invention and the gain of the amplifier circuit; FIG. 3 is a flowchart for explaining the operation of the imaging apparatus according to Embodiment 1 of the present invention. The block diagram of the imaging device in Embodiment 2 of this invention The conceptual diagram which shows the area | region of the picked-up image referred when the light / dark ratio detection circuit of the imaging device in Embodiment 2 of this invention detects a light / dark ratio Conceptual diagram showing an example of signals of each part of the imaging apparatus according to Embodiment 2 of the present invention. The graph which shows the relationship between the threshold value of coring set to the correction signal shaping circuit of the imaging device in Embodiment 2 of this invention, and the light / dark ratio detected by the light / dark ratio detection circuit Flow chart for explaining the operation of the imaging apparatus according to the second embodiment of the present invention. Block diagram of a conventional imaging device Conceptual diagram for explaining the operation of a conventional imaging apparatus

Explanation of symbols

DESCRIPTION OF SYMBOLS 1, 21, 50 Image pick-up device 10, 51 Image pick-up element 11, 52 Amplification circuit 12, 53 A / D conversion circuit 13 Correction signal generation circuit 14 Correction signal shaping circuit 15 Video signal correction circuit 16, 54 Signal processing circuit 17 Image display part 18 blur locus detection circuit 19 control circuit 29 light / dark ratio detection circuit 55 switch 56 image buffer memory 57 deviation amount detection circuit 58 CPU

Claims (8)

Amplifying means for amplifying the video signal output from the image sensor;
Correction signal generating means for generating a correction signal based on a blur locus when the video signal and a captured image represented by the video signal are captured;
Correction signal shaping means for shaping the correction signal by coring;
Video signal correction means for correcting the video signal using the correction signal shaped by the correction signal shaping means;
An imaging apparatus comprising: threshold setting means for setting the coring threshold according to the gain of the amplification means. The imaging apparatus according to claim 1, wherein the threshold value setting unit sets the coring threshold value larger as the gain of the amplification unit increases. A light / dark ratio detecting means for detecting a light / dark ratio of a captured image represented by the video signal;
The imaging apparatus according to claim 1, wherein the threshold value setting unit sets the coring threshold value larger as the ratio detected by the light / dark ratio detection unit decreases. A light / dark ratio detecting means for detecting a light / dark ratio of a captured image represented by the video signal output from the image sensor;
Correction signal generating means for generating a correction signal based on a blur locus when the video signal and a captured image represented by the video signal are captured;
Correction signal shaping means for shaping the correction signal by coring;
Video signal correction means for correcting the video signal using the correction signal shaped by the correction signal shaping means;
An imaging apparatus comprising: a threshold setting unit configured to set the coring threshold according to a ratio detected by the light / dark ratio detection unit. The imaging apparatus according to claim 4, wherein the threshold setting unit sets the coring threshold to a larger value as the ratio detected by the light / dark ratio detection unit decreases. An amplification step for amplifying the video signal output from the image sensor;
A correction signal generating step for generating a correction signal based on the video signal and a blur locus when the captured image represented by the video signal is captured;
A correction signal shaping step for shaping the correction signal by coring;
A video signal correcting step for correcting the video signal using the correction signal shaped in the correction signal shaping step;
An imaging method comprising: a threshold setting step of setting the coring threshold according to a gain in the amplification step. A correction signal generating step for generating a correction signal based on a video signal output from the image sensor and amplified by the amplifying means, and a blur locus when the captured image represented by the video signal is captured;
A correction signal shaping step for shaping the correction signal by coring;
A video signal correcting step for correcting the video signal using the correction signal shaped in the correction signal shaping step;
A program for causing a computer to execute a threshold setting step for setting the coring threshold according to the gain of the amplification means. A recording medium on which the program according to claim 7 is recorded.

Images