JP2010239493A - Image pickup apparatus, and method of correcting video signal - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image pickup apparatus 11 capable of obtaining high-definition video signals, and to provide a method of correcting video signals for obtaining high-definition video signals. <P>SOLUTION: The image pickup apparatus includes: a CCD 102 for photographing a subject 10 to output the video signal of a video image 20; a distance information acquisition section 108 for acquiring photographing distance; and a noise reduction section 110 for reducing noise, based on photographing distance of an area of interest 22, namely a video image of a specific subject 10A for composing one portion of the video image 20, and photographing distance of a proximity region 23 of the area of interest 22. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an imaging device and a video signal correction processing method that perform signal correction processing on a video signal obtained by imaging, and in particular, an imaging device that performs noise reduction processing based on distance information from a subject, and The present invention relates to a video signal correction processing method for performing noise reduction processing based on distance information between a subject and an imaging apparatus.

  In an imaging apparatus such as a digital camera, noise is included in the video signal when the video signal obtained by imaging is converted from an analog signal to a digital signal. In addition, when there is a defective pixel in an image sensor such as a CCD, fixed pattern noise is included in the video signal. For this reason, in the imaging apparatus, noise reduction processing is performed in order to reduce noise in the video signal.

  The noise reduction process is performed by a smoothing process that reduces the amplitude of the change in the video signal. However, the smoothing process also smoothes the edge component of the video signal, which may cause a side effect of degrading the image quality.

  Tomasi et al. Disclosed a noise reduction process using a bilateral filter in the IEEE International Computer Vision Conference Proceedings (1998, pages 839 to 846). The bilateral filter calculates an average and performs smoothing processing based on the weight based on the distance from the target pixel and the weight based on the signal difference. For this reason, by using a bilateral filter, it is possible to realize a noise reduction process with little image quality degradation that preserves the edge component of the video signal.

C. Tomasi and R. Manduchi, Bilateral filtering for gray and color images, Proc. IEEE. Proceedings of IEEE International Computer Vision Conference Conf. Computer Vision), (India), 1998, pages 839-846.

  However, even with noise reduction processing using a bilateral filter, it is not possible to achieve a sufficient effect for minute edge components with a small difference in video signals, and the minute edge components are smoothed to deteriorate the image quality. There was a problem that occurred.

  In order to achieve the above object, the imaging apparatus of the present invention is a distance between an imaging unit that captures a subject and outputs a video signal of a video image, and the subject and the imaging unit corresponding to the video signal. Ranging means for obtaining a shooting distance; the shooting distance of the video signal of the region of interest that is the video image of a specific subject that forms part of the video image; and the adjacent region adjacent to the region of interest Correction processing means for performing correction processing on the video signal of the region of interest based on the shooting distance of the video signal.

  The video signal correction processing method according to the present invention includes a shooting step of shooting a subject by an imaging unit and outputting a video signal of a video image, and a distance between the subject corresponding to the video signal and the imaging unit. A distance information acquisition step of acquiring a certain shooting distance; the shooting distance of the video signal of the attention area which is the video image of a specific subject constituting a part of the video image; and the adjacent area of the attention area. A correction processing method for a video signal, comprising: a correction processing step for performing correction processing on the video signal in the region of interest based on the shooting distance of the video signal.

  The present invention provides an imaging apparatus capable of obtaining a high-quality video signal and a video signal correction processing method capable of obtaining a high-quality video signal.

It is a block diagram of the imaging device of 1st Embodiment of this invention. It is explanatory drawing which shows the structure of a primary color filter. It is a block diagram of the noise reduction part of the imaging device of 1st Embodiment of this invention. It is explanatory drawing for demonstrating a video image. It is a block diagram of the weight count calculation part of the imaging device of 1st Embodiment of this invention. It is a block diagram of the weight count calculation part of the imaging device of 2nd Embodiment of this invention. It is a block diagram of the imaging device of 3rd Embodiment of this invention. It is a block diagram of the weight count calculation part of the imaging device of 3rd Embodiment of this invention. It is the figure which plotted the relationship between dispersion | distribution v of the absolute value of the distance difference in 3rd Embodiment of this invention, and distribution coefficient (sigma) d. It is a flowchart for demonstrating the flow of the noise reduction process of the imaging device of 3rd Embodiment of this invention.

<First Embodiment>
Hereinafter, an imaging device 11 according to a first embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a configuration diagram of an imaging apparatus 11 according to the first embodiment of the present invention. In the image pickup apparatus 11, a CCD 102, which is an image pickup unit, takes an image of a subject 10 including specific subjects 10A and 10B through a lens system 100 and a diaphragm 101, and outputs an analog video signal. That is, the video signal is a collection of pixel signals of the number of pixels of the CCD 102, and hereinafter, the pixel signal may be simply expressed as a pixel.

  In the present specification, the term “subject 10” simply means that the image is captured on the entire screen shot by the CCD 102, and a specific subject 10A, for example, a nearby person, or a specific subject 10B, for example, It does not mean a distant mountain. The analog video signal of the subject 10 output from the CCD 102 is converted into a digital video signal by the A / D converter 104. The video signal output from the A / D converter 104 is transferred via the buffer 105 to the photometry evaluation unit 106, the distance information acquisition unit 108 that is a distance measurement unit, and the noise reduction unit 110 that is a noise reduction unit. Is done.

  The photometric evaluation unit 106 is connected to the aperture 101 and the CCD 102, and the distance information acquisition unit 108 is connected to the lens control unit 107 and a relative distance calculation unit 109 that is a relative distance calculation unit. The lens control unit 107 is connected to the AF motor 103. The noise reduction unit 110 is connected to the interpolation unit 111, and the interpolation unit 111 is connected to the signal processing unit 112. The signal processing unit 112 is connected to the compression unit 113, and the compression unit 113 is connected to the output unit 114.

  A control unit 115 that is a control unit configured by a microcomputer or the like includes an A / D converter 104, a photometric evaluation unit 106, a lens control unit 107, a distance information acquisition unit 108, a relative distance calculation unit 109, The noise reduction unit 110, the interpolation unit 111, the signal processing unit 112, the compression unit 113, and the output unit 114 are bidirectionally connected. Further, the control unit 115 includes a power switch (not shown), a shutter button (not shown), and an external I / F unit 116 having an interface for switching shooting conditions such as a scene mode at the time of shooting. And connected in both directions. It should be noted that the shutter button of the image pickup apparatus 11 is composed of a two-stage push button switch that enters the pre-shooting mode when pressed halfway and enters the main shooting mode when pressed fully. In addition, the scene mode is a setting for the imaging device to automatically select a shutter speed, aperture value, and / or specific subject that is advantageous for various shooting scenes, and to perform shooting under appropriate conditions. For example, there are a portrait mode, a landscape mode, a close-up (macro shooting) mode, and the like, which can be selected by the photographer via the external I / F unit 116 before shooting starts.

Next, a signal flow in the imaging apparatus 11 will be described with reference to FIG.
The imaging device 11 is set with photographing conditions such as ISO sensitivity and shutter speed via the external I / F unit 116, and then a shutter button (not shown) that is a part of the external I / F unit 116 is half-pressed. This will enter pre-shooting mode. A video signal photographed by the CCD 102 via the lens system 100 and the diaphragm 101 is converted from an analog signal to a digital signal by the A / D converter 104 and stored in the buffer 105. That is, a photographing step is performed in which a subject is photographed by the imaging means and a video signal is output.

  As will be described later, the imaging device 11 performs the imaging step again even in the main imaging mode. Then, the imaging device 11 reduces noise of the second video signal output in the second shooting step in the main shooting mode based on the first video signal output in the first shooting step in the pre-shooting mode. Processing is performed. Of course, all processing may be performed using a video signal obtained by only one shooting step.

  In the imaging apparatus 11 of the present embodiment, a single-plate CCD having a Bayer-type primary color filter will be described as an example of the CCD 102. Here, FIG. 2 is an explanatory diagram showing the configuration of a Bayer-type primary color filter, which has a structure in which a number of CCD elements are two-dimensionally arranged. ", The position (coordinates) in the vertical direction is indicated by" j ". A Bayer-type primary color filter has 2 × 2 elements as a basic unit, and any one of four types of R, G1, G2, or B color filters is arranged in each CCD element. G1 and G2 are filters having the same optical characteristics. Then, a pixel 19 (see FIG. 4) consisting of one video signal is output by the basic unit 19A of 2 × 2 elements.

  The video signal stored in the buffer 105 is transferred to the photometric evaluation unit 106, the distance information acquisition unit 108, and the noise reduction unit 110. The photometric evaluation unit 106 calculates the luminance level in the video signal and controls the electronic shutter speed and the like of the aperture 101 and the CCD 102 so as to achieve proper exposure.

  A distance information acquisition unit 108 that is a distance measuring unit calculates a shooting distance d that is a distance from the imaging device 11 to the subject 10 with respect to a video signal corresponding to each pixel. That is, a distance information acquisition step for calculating the shooting distance d between the subject 10 and the imaging device 11 corresponding to the video signal is executed. The distance information acquisition unit 108 calculates, for example, a “blur parameter” representing a “blur” state from a plurality of images having different degrees of defocus. Here, the blur parameter is an index indicating the defocus state of the luminance signal, and is a parameter correlated with the dispersion value of PSF (Point Spread Function).

  Since the relationship between the dispersion value of the PSF and the shooting distance d is modeled by a predetermined relational expression, the distance information acquisition unit 108 determines the subject 10 from the imaging device 11 for each pixel of the video signal based on the blur parameter. The shooting distance d is calculated. However, since the relational expression differs depending on conditions such as the lens configuration, zoom, and diaphragm, the relational expression is calculated using a relational expression corresponding to each condition.

  As described above, the distance information acquisition unit 108 calculates the shooting distance d for each pixel of the video signal, that is, calculates the shooting distance d as many as the number of elements (number of pixels) of the CCD 102. However, in the imaging device 11, the video signal is divided into a plurality of areas and the focus point is obtained for each of the divided areas to increase the processing efficiency, thereby obtaining the shooting distance d of the video signal at an arbitrary position. It is also possible to create a distance map to represent. Information on the shooting distance d acquired by the distance information acquisition unit 108 is transferred to the lens control unit 107 and the relative distance calculation unit 109.

  The lens control unit 107, which is a specific distance acquisition unit, controls the AF motor 103 based on the shooting distance information obtained via the distance information acquisition unit 108, and the specific subject 10A, For example, a focused image focused on a person is obtained. In other words, the lens control unit 107 performs a specific distance information acquisition step of acquiring distance information of a specific distance that is a focus distance at which a focused image is obtained.

  The relative distance calculation unit 109, which is a relative distance calculation unit, is based on the shooting distance d obtained from the distance information acquisition unit 108 and the specific distance obtained by the lens control unit 107, and the entire video signal corresponding to each pixel. A relative distance calculating step for calculating a relative distance r is performed.

  Then, the imaging device 11 performs the actual shooting when the shutter button is fully pressed via the external I / F unit 116. That is, the second photographing step of photographing the subject by the imaging unit and outputting the second video signal is repeated again. The video signal on which noise reduction processing is performed in the noise reduction step described below is the second video signal. The video signal photographed by the main photographing CCD 102 is transferred to the buffer 105 and temporarily stored in the same manner as in the pre-photographing mode. The actual photographing is performed based on the exposure condition obtained by the photometric evaluation unit 106, the focusing condition obtained by the lens control unit 107, and the like, and these photographing conditions are transferred to the control unit 115.

  The noise reduction unit 110, which is a correction processing unit of the imaging device 11 and is a noise reduction unit, adaptively reduces noise with respect to the video signal obtained from the buffer 105 based on the relative distance r obtained from the relative distance calculation unit 109. A noise reduction step for processing is performed. Note that “adaptive” means that the video signal of each pixel is appropriately processed according to each situation.

  The video signal after the noise reduction processing is transferred to the interpolation unit 111. The interpolation unit 111 generates a three-plate video signal by a known interpolation process, and transfers it to the signal processing unit 112. In the signal processing unit 112, known color conversion processing and gradation conversion processing are performed on the video signal, and the video signal is transferred to the compression unit 113. The compression unit 113 performs a known compression process such as JPEG and transfers the video signal to the output unit 114. The output unit 114 records and stores the compressed video signal in a medium such as a memory card.

  Here, the structure of the noise reduction part 110 is demonstrated using FIG. FIG. 3 is a configuration diagram of the noise reduction unit 110. As shown in FIG. 3, the noise reduction unit 110 includes an image buffer 500, a local region extraction unit 501, a distance difference calculation unit 502, an upper limit setting unit 503, a clipping unit 504, a weighting factor calculation unit 505, And an addition averaging unit 506. The buffer 105 is connected to the image buffer 500. The image buffer 500 is connected to the local area extraction unit 501. The local area extraction unit 501 is connected to the distance difference calculation unit 502 and the addition average unit 506. The distance information acquisition unit 108 is connected to the distance difference calculation unit 502. The distance difference calculation unit 502 and the upper limit setting unit 503 are connected to the clipping unit 504.

  The clipping unit 504 is connected to the weighting factor calculation unit 505, and the weighting factor calculation unit 505 is connected to the addition averaging unit 506. The addition averaging unit 506 is connected to the interpolation unit 111. The control unit 115 is bidirectionally connected to the local region extraction unit 501, the distance difference calculation unit 502, the upper limit setting unit 503, the clipping unit 504, the weight coefficient calculation unit 505, and the addition average unit 506.

  The video signal from the buffer 105 is temporarily stored in the image buffer 500 and then transferred to the local region extraction unit 501. The local area extraction unit 501 extracts a local area having a predetermined pixel unit size, for example, a local area of 5 × 5 pixels, centered on each target pixel constituting the target area in the subject video. Is transferred to the distance difference calculation unit 502 and the addition averaging unit 506.

  Here, FIG. 4 is an explanatory diagram for explaining a video image. The attention pixel 21 is, for example, the pixel 19 of the image of the specific subject 10 </ b> A at the in-focus position, and the pixel 19 constituting the attention region 22. In other words, the attention area 22 is an area of the video image of the specific subject 10A at the in-focus position in the entire video image 20 of the subject 10. The attention area 22 is entirely composed of focused pixels, whereas the local area 24 is centered on the attention pixel 21 that is a focused pixel, but is composed of focused and non-focused pixels. The neighborhood region 23 is composed of non-focused pixels. As is apparent from FIG. 4, the neighboring region 23 is a region adjacent to the attention region 22. More specifically, the neighboring region 23 may be a region having a width of “(the number of pixels in the local region−1) / 2” as the number of pixels along the outer edge of the region of interest 22. In other words, the neighboring region 23 is a region of non-focused pixels in the range from the focused pixel to N pixels. For example, N is an integer of 1 or more. 4 shows a part of the image 20A of the video image 20 including one attention area 22, the video image 20 may include a plurality of attention areas 22.

  The distance obtained by the distance information acquisition unit 108 is transferred to the distance difference calculation unit 502.

  A distance difference calculation unit 502 serving as a distance difference calculation unit calculates an absolute value of a distance difference corresponding to each pixel by subtracting a distance corresponding to the target pixel from a distance corresponding to each pixel in the local region, and performs clipping. The data is transferred to the unit 504.

  The upper limit setting unit 503 serving as an upper limit setting unit is a distance difference upper limit that is an upper limit value of the distance difference absolute value based on information such as a focusing condition or a scene mode obtained from the external I / F unit 116 via the control unit 115. Set the value. The upper limit setting unit 503 is provided in order to prevent the absolute value of the distance difference from becoming infinite and difficult to calculate, and can also be manually set via the control unit 115. For example, when the scene mode is the macro shooting mode, the distance difference between the attention area and the neighboring area is reduced as a whole, so the upper limit value of the distance difference is set to a small value. On the other hand, when a landscape and a person are photographed at the same time, the distance difference between the attention area and the neighboring area becomes large, so the distance difference upper limit value is set to a large value. The distance difference upper limit value set by the upper limit setting unit 503 is transferred to the clipping unit 504.

  A clipping unit 504 serving as clipping means performs clipping processing on the absolute value of the distance difference corresponding to each pixel in the local area transferred from the distance difference calculation unit 502 based on the distance difference upper limit value. Here, the clipping process is a process of replacing the distance difference absolute value exceeding the distance difference upper limit value with the distance difference upper limit value. The distance difference absolute value of the pixels in the local region after the clipping processing is transferred to the weighting factor calculating unit 505 which is a weighting factor calculating unit.

  Here, FIG. 5 is a configuration diagram of a weighting coefficient calculation unit of the imaging apparatus 11. As shown in FIG. 5, the weighting coefficient calculation unit 505 includes a function calculation unit 600 and a coefficient ROM 601 that records distribution coefficients. The weighting coefficient calculation unit 505 calculates a weighting coefficient for each pixel in the local region based on the distance difference absolute value after the clipping process.

  As shown in FIG. 5, the clipping unit 504 and the coefficient ROM 601 are connected to the function calculation unit 600. The function calculation unit 600 is connected to the addition averaging unit 506. The control unit 115 is bidirectionally connected to the function calculation unit 600. The absolute value of the distance difference in the local area after the clipping process is transferred to the function calculation unit 600.

  The function calculation unit 600 reads a predetermined distribution coefficient from the coefficient ROM 601 and calculates a weighting coefficient based on the distance difference absolute value corresponding to each pixel in the local region. For example, the weighting coefficient is obtained using the following formula 1.

ここで、（ｉ、ｊ）は注目画素の座標を、(ｊ＋ｍ、ｊ＋ｎ)は注目画素（ｉ、ｊ）から水平方向にm、垂直方向にn画素だけ離れた近傍画素の座標を、Wは局所領域の画素サイズ（例えば、Ｗ＝５）を、ｗ（ｊ＋ｍ、ｊ＋ｎ)は座標(ｊ＋ｍ、ｊ＋ｎ)の画素における重み係数を示している。また、d（ｉ、ｊ）は座標（ｉ、ｊ）の画素における距離情報の値を表し、σdは距離差に対する重み係数を表すガウス分布の標準偏差を表す分布係数である。分布係数σdは予め係数ＲＯＭ６０１に記録しておく。 (Formula 1)

Here, (i, j) is the coordinate of the pixel of interest, (j + m, j + n) is the coordinate of a neighboring pixel separated from the pixel of interest (i, j) by m in the horizontal direction and n pixels in the vertical direction, and W is The pixel size (for example, W = 5) of the local area, and w (j + m, j + n) indicates the weighting coefficient in the pixel at the coordinates (j + m, j + n). Further, d (i, j) represents a value of distance information in the pixel at coordinates (i, j), and σd is a distribution coefficient representing a standard deviation of a Gaussian distribution representing a weighting coefficient for the distance difference. The distribution coefficient σd is recorded in the coefficient ROM 601 in advance.

  As is clear from Equation 1, in the calculation process in the weighting factor calculation unit 505, when the distance information of the pixel of interest and the absolute value of the distance difference between neighboring pixels increase, the weighting factor for the neighboring pixel decreases accordingly. Since the addition average is calculated using video signals of neighboring pixels having a small distance difference absolute value, a smoothing process is performed in which an edge component is stored in a region having a large distance difference absolute value.

  The weighting factor calculated by the weighting factor calculation unit 505 is transferred to the addition averaging unit 506 which is an addition averaging unit. Based on the weighting factor calculated by the weighting factor calculation unit 505, the addition averaging unit 506 calculates an addition average for the video signal of each pixel in the local region obtained by the local region extraction unit 501. The calculated addition average is replaced with the video signal of the target pixel. The video signal of the target pixel after the averaging is transferred to the interpolation unit 111.

  In the imaging apparatus 11 according to the present embodiment described above, the intensity of noise reduction processing is changed between a region having a large distance difference absolute value and a region having a small distance difference, thereby excessively reducing a small edge component having a small video signal difference. Since no processing is performed, a high-quality video signal can be obtained.

  In addition, in the video signal correction processing method of the present embodiment, a high-quality video signal can be obtained.

<Modification of First Embodiment>
Hereinafter, an imaging device 11B according to a modification of the first embodiment of the present invention will be described with reference to the drawings. Since the imaging device 11B of the present modification is similar to the imaging device 11 of the first embodiment, the same components are denoted by the same reference numerals and description thereof is omitted.

The weighting factor calculation unit 505 of the imaging device 11 according to the first embodiment is configured such that the function calculation unit 600 calculates a weighting factor using a function.
On the other hand, the weighting factor calculation unit 505B of the imaging apparatus 11B of the present modification refers to a weighting factor by referring to a table ROM that is a factor recording table in which the correspondence between the distance difference absolute value and the weighting factor is recorded in advance. Ask for.

  As illustrated in FIG. 6, the weighting factor calculation unit 505B of the imaging device 11B includes a table calculation unit 602 and a table ROM 603.

  The clipping unit 504 and the table ROM 603 are connected to the table calculation unit 602. The table calculation unit 602 is connected to the addition averaging unit 506. The control unit 115 is bidirectionally connected to the table calculation unit 602. A table calculation unit 602 serving as a table calculation unit refers to the table ROM 603 and calculates a weighting coefficient corresponding to the distance difference absolute value in the local area transferred from the clipping unit 504. The table (correspondence table) recorded in the table ROM 603 is predetermined. For example, the weighting coefficient is calculated using the function of Equation 1 for the absolute value of the distance difference sampled at a constant interval. Then, it can be determined by associating the relationship between the distance difference absolute value and the weighting coefficient. The weighting factor calculated by the weighting factor calculation unit 505B is transferred to the addition averaging unit 506.

  The imaging device 11B of the present modification described above has the same effect as the imaging device 11 of the first embodiment, and the table arithmetic processing is easy to implement. Thus, a faster and lower cost system can be constructed. In addition, the video signal correction processing method of the present modification not only has the same effect as the video signal correction processing method of the first embodiment, but also compared with the video signal correction processing method of the first embodiment. It is possible to construct a faster and lower cost system.

<Second Embodiment>
Hereinafter, an image pickup apparatus 11C according to a second embodiment of the present invention will be described with reference to the drawings. Since the imaging device 11C of the present embodiment is similar to the imaging device 11 of the first embodiment, the same components are denoted by the same reference numerals and description thereof is omitted.

  FIG. 7 is a configuration diagram illustrating a configuration of the noise reduction unit 110C of the imaging device 11C according to the present embodiment. The noise reduction unit 110C has a configuration in which a signal difference calculation unit 507 and a coordinate difference calculation unit 508 are added to the configuration of the noise reduction unit 110 of the first embodiment shown in FIG.

  As shown in FIG. 7, the local region extraction unit 501 is connected to a signal difference calculation unit 507 and a coordinate difference calculation unit 508. The signal difference calculation unit 507 and the coordinate difference calculation unit 508 are connected to the weight coefficient calculation unit 505. The control unit 115 is bidirectionally connected to the signal difference calculation unit 507 and the coordinate difference calculation unit 508. The video signal in the local area obtained by the local area extracting unit 501 is transferred to the signal difference calculating unit 507 and the coordinate difference calculating unit 508. A signal difference calculation unit 507 serving as a signal difference calculation unit calculates an absolute value of a difference between the video signal of each pixel in the local region and the video signal of the target pixel. The processing of the signal difference calculation unit 507 is expressed by the following equation (2).

ここで、f（ｉ、ｊ）は座標（ｉ、ｊ）の画素の映像信号を、dS(ｊ＋ｍ、ｊ＋ｎ)は座標(ｊ＋ｍ、ｊ＋ｎ)の画素における注目画素の映像信号との差の絶対値を表す。局所領域内の各画素に対応する映像信号の差の絶対値は、重み係数算出部５０５へ転送される。 (Formula 2)

Here, f (i, j) is the video signal of the pixel at coordinates (i, j), and dS (j + m, j + n) is the absolute value of the difference from the video signal of the pixel of interest at the pixel at coordinates (j + m, j + n). Represents. The absolute value of the difference between the video signals corresponding to each pixel in the local region is transferred to the weighting coefficient calculation unit 505.

  A coordinate difference calculation unit 508 serving as a coordinate difference calculation unit calculates a distance between pixels that is a difference between the two-dimensional coordinate value of each pixel in the local region and the two-dimensional coordinate value of the target pixel. Note that the distance between the pixels is referred to as “the absolute value of the difference in coordinate values” in order to distinguish from the shooting distance d, which is the distance between the subject 10 and the imaging device 11C. The absolute value of the difference between the coordinate values is expressed by the following equation (3).

ここで、ｄＣ（ｊ＋ｍ、ｊ＋ｎ)は座標(ｊ＋ｍ、ｊ＋ｎ)の画素における注目画素の座標値との差の絶対値を表す。算出した座標値の差の絶対値は、重み係数算出部５０５Ｃへ転送される。 (Formula 3)

Here, dC (j + m, j + n) represents the absolute value of the difference from the coordinate value of the pixel of interest in the pixel at coordinates (j + m, j + n). The absolute value of the calculated coordinate value difference is transferred to the weighting coefficient calculation unit 505C.

  FIG. 8 is a configuration diagram of a weighting coefficient calculation unit of the imaging apparatus 11C. The weighting factor calculation unit 505C, which is a weighting factor calculation unit of the imaging device 11C, is an absolute value of a distance difference, an absolute value of a difference between video signals, and coordinates transferred from the clipping unit 504, the signal difference calculation unit 507, and the coordinate difference calculation unit 508. Based on the absolute value of the value difference, a weighting factor for each pixel in the local region is calculated.

  As shown in FIG. 8, the configuration of the weighting factor calculation unit 505C is the same as the configuration of the weighting factor calculation unit 505 of the imaging apparatus 11 according to the first embodiment shown in FIG. The second coefficient ROM 606 is added.

  The clipping unit 504 is connected to the variance calculation unit 604 and the function calculation unit 600. The variance calculation unit 604 and the coefficient ROM 601 are connected to the coefficient setting unit 605. The signal difference calculation unit 507 and the coordinate difference calculation unit 508 are connected to the function calculation unit 600. The coefficient setting unit 605 and the second coefficient ROM 606 are also connected to the function calculation unit 600. The control unit 115 is bidirectionally connected to the variance calculation unit 604 and the coefficient setting unit 605.

  A variance calculation unit 604 serving as a variance calculation unit calculates the variance v of the distance difference absolute value in the local area obtained via the clipping unit 504 and transfers the variance v to the coefficient setting unit 605. A coefficient setting unit 605 serving as an adjustment unit reads a distribution coefficient σd, which is a standard deviation of a Gaussian distribution representing a weighting coefficient for the distance difference absolute value, from the coefficient ROM 601, and uses the distance difference absolute value variance v calculated by the variance calculation unit 604. Accordingly, the value of the distribution coefficient σd is converted.

  Specifically, when the variance v of the absolute value of the distance difference is large, the coefficient setting unit 605 determines that the pixel serving as the edge component is included in the local region, and decreases the value of the distribution coefficient σd. Reduce the strength of noise reduction. On the other hand, when the variance v is small, it is determined that the local region is a flat region with a small amplitude of the video signal, and the value of the distribution coefficient σd is increased to increase the strength of the noise reduction processing.

  FIG. 9 shows a plot of the relationship between the variance v of the absolute value of the distance difference and the distribution coefficient σd. In the processing of the coefficient setting unit 605 shown in FIG. 9, conversion is performed so that the distribution coefficient σd decreases linearly with respect to the variance v. However, when the variance v is greater than or equal to a predetermined value vth, the value of the distribution coefficient σd is fixed, so-called clipping. This is to prevent the distribution coefficient σd from becoming a minute value when the variance v becomes large and the noise reduction processing to be substantially not performed.

  The distribution coefficient σd set by the coefficient setting unit 605 is transferred to the function calculation unit 600. In the function calculation unit 600, the distribution coefficient σd calculated by the coefficient setting unit 605, the distance difference absolute value transferred from the clipping unit 504, the absolute value of the difference between the video signals transferred from the signal difference calculation unit 507, and coordinates Based on the absolute value of the difference between the coordinate values transferred from the difference calculation unit 508, a weighting coefficient for each pixel in the local region is calculated using (Equation 4) below.

σ1は座標値の差の絶対値に対する重み係数を表すガウス分布の標準偏差を表す係数を示し、σ2は映像信号の差の絶対値に対する重み係数を表すガウス分布の標準偏差を表す係数を示す。σ1およびσ2は予め決められている所定の値で第２係数ＲＯＭ６０６に保存されている。関数演算部６００は（式４）の計算のときに第２係数ＲＯＭ６０６からσ1およびσ2を読み込む。算出した重み係数は、加算平均部５０６へ転送される。 (Formula 4)

σ1 represents a coefficient representing the standard deviation of the Gaussian distribution representing the weighting coefficient with respect to the absolute value of the difference between the coordinate values, and σ2 represents a coefficient representing the standard deviation of the Gaussian distribution representing the weighting coefficient with respect to the absolute value of the difference between the video signals. σ 1 and σ 2 are stored in the second coefficient ROM 606 as predetermined values. The function calculation unit 600 reads σ 1 and σ 2 from the second coefficient ROM 606 during the calculation of (Expression 4). The calculated weighting coefficient is transferred to the addition averaging unit 506.

  Here, the flow of noise reduction processing in the imaging device 11C of the third embodiment will be described with reference to FIG. FIG. 10 is a flowchart for explaining the flow of noise reduction processing in the imaging device 11C of the third embodiment.

  Hereinafter, a description will be given according to the flowchart of FIG.

<Step S20> Shooting Step In the imaging device 11C, a shooting step for obtaining a focused image focused on the specific subject 10A is executed under the control of the control unit 115 in accordance with the setting of the scene mode and the like. That is, the video signal of the video image of the subject 10 including the specific subject 10A is output by the CCD 102 as the imaging means.

<Step S21> Shooting distance acquisition step
The distance information acquisition unit 108 calculates the shooting distance between the imaging device 11C and the subject 10 for each of the video signals of the individual pixels constituting the video signal.

<Step S22> Local region extraction step
The local region extraction unit 501 extracts a local region having a size of, for example, 5 × 5 pixels centered on the target pixel constituting the specific subject 10A.

<Step S23> Shooting distance difference calculation step
The distance difference calculation unit 502 calculates the absolute value of the difference from the shooting distance of the target pixel for each pixel in the local region.

<Step S24> Distance difference upper limit setting step
The control unit 115 reads the upper limit value of the absolute value of the distance difference set in the upper limit setting unit.

<Step S25> Clipping processing step
The clipping unit 504 performs clipping processing on the absolute value of the distance difference for each pixel in the local region based on the upper limit value obtained in step S24.

<Step S26> Video signal difference calculation step
The signal difference calculation unit 507 calculates the absolute value of the difference from the video signal of the target pixel for each pixel in the local region.

<Step S27> Coordinate difference calculation step
The coordinate difference calculation unit 508 calculates the absolute value of the difference from the coordinate value of the target pixel for each pixel in the local region.

<Step S28> Weighting factor calculation step
The weighting factor calculation unit 505C calculates the weighting factor for each pixel in the local region based on the absolute value of the distance difference, the absolute value of the difference between the video signals, and the absolute value of the difference between the coordinate values obtained from Step 25, Step 26, and Step 27. Is calculated.

<Step S29> Addition averaging step
The addition averaging unit 206 calculates a weighted addition average in the local region based on the weighting coefficient obtained from Step 28, and replaces the video signal of the target pixel with the calculated addition average.

<Step S30>
It is determined whether or not the processing for all the pixels has been completed. If the processing has not been completed (No), the process returns to Step 22, and the same processing is repeated until the processing for all the pixels is completed. When the noise reduction process for all the pixels is completed (Yes), the noise reduction unit 110 transfers the processed video signal to the interpolation unit 111.

<Step S31>
Known interpolation processing is performed in the interpolation unit 111, and color conversion, gradation conversion, edge enhancement processing, and the like are performed in the signal processing unit 112.

<Step S32>
A known compression process such as JPEG is performed in the compression unit 113.

<Step S32>
The processed video signal is output to the output unit 114, and the processing of the imaging device 11C ends.

Since the imaging device 11C according to the third embodiment adaptively controls the intensity of the noise reduction processing based on the distance information, similarly to the imaging device 11 according to the first embodiment, the optimum noise reduction processing for each region of the video signal. And a high-quality video signal can be obtained. In addition, since the imaging device 11C determines the weighting coefficient at the time of addition averaging according to the absolute value of the distance difference from the target pixel, for example, if the absolute value of the distance difference is large in the local region, a minute edge component Therefore, it is possible to perform noise reduction processing with little deterioration.
In addition to the effects of the video signal correction processing method of the first embodiment, the video signal correction processing method of the third embodiment performs noise reduction processing with less deterioration even for a minute edge component. It becomes possible.

  In the imaging device 11C, the standard deviation of the Gaussian distribution is converted according to the variance of the absolute value of the distance difference. However, the imaging device 11C is not limited to this. For example, in the weighting factor calculation unit 505C shown in FIG. A configuration is also possible in which distance information in the local region obtained from the extraction unit 501 is transferred to the variance calculation unit 604, and the variance calculation unit 604 calculates the variance of the distance information. In the case of this configuration, the standard deviation of the Gaussian distribution is converted based on the dispersion of distance information.

  The imaging device 11 or the like is intended for a single-plate CCD in which a Bayer-type primary color filter is disposed on the front surface, but is not limited thereto, and a single-plate CCD in which a complementary color filter is disposed on the front surface, It is also possible to apply to a three-plate CCD.

  In the imaging apparatus 11 and the like, the weighting coefficient is calculated based on the Gaussian distribution. However, the weighting coefficient is not limited to this, and can be calculated based on the Laplace distribution or a rational function.

  Furthermore, although the imaging apparatus 11 and the like are premised on processing by hardware, it is not necessary to be limited to such a configuration. For example, a video signal from the CCD 102 may be unprocessed raw data, information at the time of shooting from the control unit 115 may be output as header information, and separately processed by software.

  The imaging device 11 and the like are configured to control the noise reduction processing for the video signal based on the distance information. However, the imaging device 11 is not limited to the noise reduction processing, and for example, the distance information may be used when interpolating a pixel defect. Is possible. When interpolating pixel defects, distance information for the entire video signal is obtained in advance by stereo distance measurement, etc., and the pixel position and distance information associated with the defect are associated with each other and have a distance information equivalent to that of the pixel defect portion. Interpolation can also be performed using pixel information.

  The present invention is not limited to the above-described embodiments and modifications, and various changes and modifications can be made without departing from the scope of the present invention.

DESCRIPTION OF SYMBOLS 10, 10A, 10B ... Subject, 11, 11B, 11C ... Imaging device, 20 ... Video image, 21 ... Target pixel, 22 ... Target region, 23 ... Near region, 24 ... Local region, 100 ... Lens system, 102 ... CCD DESCRIPTION OF SYMBOLS 103 ... Motor 104 ... Converter 105 ... Buffer 106 ... Photometry evaluation part 107 ... Lens control part 108 ... Distance information acquisition part 109 ... Relative distance calculation part 110 ... Noise reduction part 110C ... Noise reduction 111: Interpolation unit, 112 ... Signal processing unit, 113 ... Compression unit, 114 ... Output unit, 115 ... Control unit, 116 ... External I / F unit, 206 ... Addition averaging unit, 500 ... Image buffer, 501 ... Local Area extraction unit 502 ... Distance difference calculation unit 503 ... Upper limit setting unit 504 ... Clipping unit 505 ... Coefficient calculation unit 505B ... Coefficient calculation unit 505C ... Coefficient calculation unit 5 6 ... averaging unit, 507 ... signal difference calculation section, 508 ... coordinate difference calculation section, 600 ... function calculator, 602 ... table calculation unit, 604 ... variance calculation unit, 605 ... coefficient setting unit

Claims (18)

Imaging means for photographing a subject and outputting a video signal of a video image;
Ranging means for obtaining a shooting distance corresponding to the video signal, which is a distance between the subject and the imaging means;
Based on the shooting distance of the video signal of a region of interest that is the video image of a specific subject constituting a part of the video image, and the shooting distance of the video signal of an adjacent region adjacent to the region of interest An imaging apparatus comprising: correction processing means for performing correction processing on the video signal in the region of interest.   The imaging apparatus according to claim 1, wherein the correction processing is noise reduction processing, and the correction processing means is noise reduction means. The noise reduction means includes
A distance difference calculating means for calculating a distance difference which is a difference between the shooting distance of the video signal of the attention area and the shooting distance of the video signal of the adjacent area;
A weighting factor calculating means for calculating a weighting factor for the video signal of the region of interest and the video signal of the adjacent region based on the distance difference;
Based on the weighting factor, an addition average video signal is obtained by calculating a weighted addition average of the video signal of the attention area and the video signal of the adjacent area, and replacing the addition average video signal with the video signal of the attention area The imaging apparatus according to claim 2, further comprising: means. The noise reduction means includes
Signal difference calculating means for calculating a video signal difference which is a difference between the video signal of the region of interest and the video signal of the adjacent region;
Coordinate difference calculating means for calculating a coordinate value difference which is a difference between the coordinate value in the video image of the region of interest and the coordinate value in the video image of the adjacent region;
Weighting factor calculating means for calculating a weighting factor based on the distance difference, the video signal difference, and the coordinate value difference;
Based on the weighting factor, an addition average video signal obtained by weighted and averaging the video signal of the attention area and the video signal of the adjacent area is calculated, and the addition average video signal is replaced with the video signal of the attention area. The imaging apparatus according to claim 3, further comprising addition averaging means. The weight coefficient calculating means includes
5. The imaging apparatus according to claim 3, further comprising weight coefficient calculation function means for calculating a weight coefficient for the video signal using a distribution coefficient representing a standard deviation of the distance difference distribution. . The weight coefficient calculating means includes
A coefficient recording table for recording the weighting coefficient corresponding to the absolute value of the distance difference;
The imaging apparatus according to claim 3, further comprising: a table calculating unit that calculates the weighting coefficient based on the coefficient recording table. The distance difference calculating means includes
Distance difference upper limit setting means for setting a distance difference upper limit value with respect to the absolute value of the distance difference;
5. A clipping unit that replaces the absolute value of the distance difference with the upper limit value of the distance difference when the absolute value of the distance difference exceeds the upper limit value of the distance difference. The imaging device described. The weight coefficient calculating means includes
Variance calculating means for calculating variance of the shooting distance corresponding to video signals of the attention area and the adjacent area of the attention area;
The imaging apparatus according to claim 5, further comprising an adjusting unit that adjusts the distribution coefficient based on the variance of the shooting distance.   5. The imaging apparatus according to claim 3, wherein the weighting coefficient calculating unit calculates a smaller weighting coefficient as the absolute value of the distance difference is larger. A shooting step of shooting a subject by an imaging means and outputting a video signal of a video image;
A distance information acquisition step of acquiring a shooting distance which is a distance between the subject corresponding to the video signal and the imaging means;
Based on the shooting distance of the video signal of the region of interest that is the video image of the specific subject that forms part of the video image, and the shooting distance of the video signal of the region adjacent to the region of interest, And a correction processing step for performing correction processing on the video signal in the region of interest.   The video signal correction processing method according to claim 10, wherein the correction processing is noise reduction processing. The correction processing step includes
A distance difference calculating step of calculating a distance difference which is a difference between the shooting distance of the video signal of the attention area and the shooting distance of the video signal of the adjacent area;
A weighting factor calculating step for calculating a weighting factor for the video signal of the region of interest and the video signal of the adjacent region based on the distance difference;
An averaging step of calculating an addition average video signal obtained by weighted and averaging the video signal of the region of interest and the video signal of the adjacent region based on the weighting factor, and replacing the addition average video signal with the video signal of the region of interest The video signal correction processing method according to claim 11, further comprising: The correction processing step includes
A signal difference calculating step of calculating a video signal difference which is a difference between the video signal of the region of interest and the video signal of the adjacent region;
A coordinate difference calculating step of calculating a coordinate value difference that is a difference between a coordinate value in the video image of the region of interest and a coordinate value in the video image of the adjacent region;
A weighting factor calculating step for calculating a weighting factor based on the distance difference, the video signal difference, and the coordinate value difference;
Based on the weighting factor, an addition average video signal obtained by weighted and averaging the video signal of the attention area and the video signal of the adjacent area is calculated, and the addition average video signal is replaced with the video signal of the attention area. 12. The video signal correction processing method according to claim 11, further comprising: an averaging step. The weighting factor calculating step includes:
14. The video signal according to claim 12, further comprising a weighting factor calculation function step for calculating a weighting factor for the video signal using a distribution factor representing a standard deviation of the distance difference distribution. Correction processing method. The weighting factor calculating step includes:
14. The video according to claim 12, further comprising a table calculation step of calculating the weighting factor based on a coefficient recording table that records the weighting factor corresponding to the absolute value of the distance difference. Signal correction processing method. The distance difference calculating step includes:
A distance difference upper limit setting step for setting a distance difference upper limit value with respect to the absolute value of the distance difference;
The clipping step of replacing the absolute value of the distance difference with the upper limit value of the distance difference when the absolute value of the distance difference exceeds the upper limit value of the distance difference. The video signal correction processing method described. The weighting factor calculating step includes:
A variance calculating step for calculating variance of the shooting distance corresponding to video signals of the attention area and the adjacent area of the attention area;
The video signal correction processing method according to claim 14, further comprising an adjustment step of adjusting the distribution coefficient based on the variance of the shooting distance.   14. The video signal correction processing method according to claim 12, wherein the weighting factor calculating step calculates a smaller weighting factor as the absolute value of the distance difference is larger.

Images