JP4785635B2 - Image processing apparatus and method - Google Patents

Description

  The present invention relates to image processing.

  In digital video cameras and digital cameras, the high frequency component of the image signal deteriorates due to the limited aperture ratio of the lens and the light receiving portion of the image sensor. For this reason, aperture correction processing is performed to compensate for the high frequency components of the image, enhance the contour of the video signal, and increase the sharpness of the image.

  As an example of a conventional aperture correction signal generation processing circuit, there is a configuration that includes a horizontal aperture processing circuit and a vertical aperture processing circuit, and synthesizes outputs of horizontal and vertical aperture processing circuits (for example, Patent Document 1). reference). The horizontal aperture processing circuit is formed by cascading a low-pass filter (hereinafter referred to as LPF) in the vertical direction and a band-pass filter (hereinafter referred to as BPF) in the horizontal direction. The vertical aperture processing circuit is formed by cascading LPFs in the horizontal direction and BPFs in the vertical direction.

Japanese Patent Laid-Open No. 11-17983 (page 4, FIG. 1)

  However, in the configuration of the above-described prior art, when noise is superimposed on a flat portion of an image, the noise is stretched in the vertical direction or the horizontal direction by the low-pass filter processing included in the aperture processing circuit, and the graininess of the noise There was a problem of getting worse.

  An object of the present invention is to perform an aperture correction process with reduced distortion without impairing the granularity of noise in a flat portion of an image.

The image processing apparatus of the present invention has a vertical and horizontal bandpass filter and a vertical and horizontal lowpass filter, and emphasizes the vertical and horizontal contours based on the input image, respectively. Aperture correction signal generating means for generating an aperture correction signal in the horizontal direction , contour estimation means for estimating whether or not the target pixel of the input image and its surrounding pixels form a diagonal contour, the target pixel of the input image, and A contour direction estimating means for estimating the degree of whether the contour direction of the pixel of interest is a vertical direction or a horizontal direction based on the signal of the surrounding pixels, and a narrow band low-pass filter in the aperture correction signal generating means The processed vertical and horizontal aperture correction signals are weighted according to the degree of the vertical direction and the horizontal direction. Give synthesis produces a narrowband aperture correction signal performed, the aperture correction signal generation means the combined wideband aperture correction signal in the vertical direction and the horizontal direction of the aperture correction signal processed by the wideband low-pass filter at a predetermined ratio generate the and a synthesizing means for, said combining means, said outputs the narrowband aperture correction signal is estimated to constitute an oblique direction of the contour by the contour estimation means, constituting a diagonal contour by the contour estimator When it is estimated that the wideband aperture correction signal is not generated, the wideband aperture correction signal is output .

  Since the phenomenon that the noise component diffuses in a specific direction in the flat portion of the input image is reduced, the graininess of the noise can be kept good. In addition, it is possible to synthesize an aperture correction signal with reduced distortion in consideration of the direction of the edge, and to add an edge with uniform strength in all directions.

Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.
(First embodiment)
FIG. 1 is a block diagram showing a schematic configuration example of an aperture correction signal generation processing circuit according to the first embodiment of the present invention. In FIG. 1, 1 is a control coefficient generation unit, 2 is an aperture signal generation unit, and 3 is an aperture adaptive control unit. In this aperture correction signal generation processing circuit, the control coefficient generation unit 1 and the aperture signal generation unit 2 process the signals of the respective pixels fetched from the image input terminals in parallel, and then the aperture adaptive control unit 3 performs the final processing. A correct aperture correction signal is generated. Hereinafter, the contour is referred to as an edge.

  Hereinafter, the configuration and operation of the control coefficient generation unit 1, the aperture signal generation unit 2, and the aperture adaptive control unit 3 will be described. First, the control coefficient generation unit 1 will be described. In the control coefficient generation unit 1, 11 is a vertical direction correlation detection processing means, 12 is a horizontal direction correlation detection processing means, 13 is an edge direction estimation processing means, 14 is an oblique 45 ° direction correlation detection processing means, and 15 is an oblique 135 ° direction. Correlation detection processing means 16 is a non-correlation estimation processing means. The control coefficient generation unit 1 outputs an edge direction estimation coefficient T indicating the edge direction of the target pixel, and an uncorrelation determination flag D indicating whether there is a correlation between the target pixel and surrounding pixels.

  Here, the operation in each process included in the control coefficient generation unit 1 will be described using the local region of the image shown in FIG. 2 as an example. The vertical direction correlation detection processing means 11 detects the correlation between the target pixel P22 and the adjacent pixels P12 and P32 in the vertical direction, and outputs a vertical correlation value vdiff. The horizontal direction correlation detection processing means 12 detects the correlation between the target pixel P22 and the adjacent pixels P21 and P23 in the horizontal direction, and outputs a horizontal correlation value hdiff. The oblique 45 ° direction correlation detection processing means 14 detects the correlation between the target pixel P22 and adjacent pixels P13 and P31 in the oblique 45 ° direction, and outputs an oblique 45 ° direction correlation value d1_diff. The oblique 135 ° direction correlation detection processing means 15 detects the correlation between the pixel of interest P22 and adjacent pixels P11 and P33 in the oblique 135 ° direction, and outputs an oblique 135 ° direction correlation value d2_diff.

  The vertical correlation value vdiff, the horizontal correlation value hdiff, the diagonal 45 ° direction correlation value d1_diff, and the diagonal 135 ° direction correlation value d2_diff are calculated by, for example, calculations such as equations (1) to (4).

hdiff = | P22 − P21 − P23 | (1)
vdiff = | P22 − P12 − P32 | (2)
d1_diff = | P22 − P13 − P31 | (3)
d2_diff = | P22 − P11 − P33 | (4)

  Further, the edge direction estimation processing means 13 sets predetermined thresholds th1, th2, -th1, -th2 (where | th2 |> | th1 |) for the difference between the vertical correlation value vdiff and the horizontal correlation value hdiff. The edge direction at the target pixel is estimated, and an edge direction estimation coefficient T is generated. That is, the edge direction estimation processing means 13 estimates the edge direction of the target pixel based on the signals of the target pixel of the input image and its surrounding pixels, and generates an edge direction estimation coefficient T.

  Here, the procedure of edge direction estimation processing using the difference between the horizontal correlation value hdiff and the vertical correlation value vdiff will be described using the flowchart of FIG. In step S01, the edge direction estimation process is started.

  In step S02, conditional branching using hdiff-vdiff and threshold value th2 is performed. In step S02, if hdiff−vdiff is greater than or equal to th2, in step S03, the edge direction estimation coefficient T is set to 1.0. If the condition in step S02 is not satisfied, the process proceeds to the conditional branch in step S04. .

  In step S04, conditional branching using hdiff-vdiff and threshold-th2 is performed. If hdiff−vdiff is smaller than −th2, −1.0 is set to the edge direction estimation coefficient T in step S05, and if the condition in step S04 is not satisfied, the process proceeds to the conditional branch in step S06.

  In step S06, conditional branching using hdiff-vdiff and threshold values th1 and th2 is performed. If hdiff−vdiff is greater than or equal to th1 and smaller than th2, in step S07, T = (hdiff−vdiff−th1) / (th2−th1) is set as the edge direction estimation coefficient T. If the condition in step S06 is not satisfied, the process proceeds to the conditional branch in step S08.

  In step S08, conditional branching using hdiff−vdiff and threshold values −th1 and −th2 is performed. If hdiff−vdiff is equal to or larger than −th2 and smaller than −th1, T = (hdiff−vdiff + th1) / (th1−th2) is set as the edge direction estimation coefficient T in step S09. If the condition of step S09 is not satisfied, T = 0.0 is set as the edge direction estimation coefficient T.

  FIG. 12 is a graph showing the relationship between the edge direction estimation coefficient T and hdiff-vdiff obtained by the above procedure. As shown in FIG. 13, when the edge direction is defined as an angle θ between the edge direction and the horizontal axis in the target pixel, the edge direction estimation coefficient T and the angle θ indicating the edge direction are The relationships can be associated as shown in FIG.

  Here, when the edge direction estimation coefficient T = 0.0, the pixel of interest may be included in an oblique line pattern as shown in FIG. 5, or may be an isolated point as shown in FIG. Therefore, the non-correlation estimation processing means 16 uses the edge direction estimation coefficient T, the diagonal 45 ° direction correlation value d1_diff, and the diagonal 135 ° direction correlation value d2_diff. Then, the uncorrelated estimation processing means 16 determines whether the pixel of interest is an edge having a correlation in a specific direction or an isolated point uncorrelated with surrounding pixels, and outputs a non-correlation determination flag D.

  That is, the non-correlation estimation processing means 16 estimates whether or not the pixel of interest of the input image and its surrounding pixels constitute an edge in a specific direction, and outputs a non-correlation determination flag D. The uncorrelation determination flag D becomes 0 when it is estimated that an edge is formed, and becomes 0 when it is estimated that an edge is not formed.

  Here, the processing of the uncorrelated estimation processing means 16 will be described with reference to the flowchart of FIG. From FIG. 15, uncorrelated estimation processing is started in step S20, and it is determined in step S21 whether the edge direction estimation coefficient T is zero. If the edge direction estimation coefficient T is not 0, it is assumed that there is a correlation in a specific direction closer to the vertical direction or the horizontal direction, and 0 (correlated) is set in the uncorrelation determination flag D in step S25. On the other hand, when the edge direction estimation coefficient T is 0 in step S21, the process proceeds to the conditional branch in step S22. If the absolute value of the difference between the diagonal 45 ° direction correlation value d1_diff and the diagonal 135 ° direction correlation value d2_diff is greater than or equal to th3 in step S22, 0 (correlated) is set in the uncorrelation determination flag D in step S23. If the condition is not satisfied in step S22, 1 (no correlation) is set to the uncorrelation determination flag D in step S24.

  When the non-correlation determination flag D is D = 0, the target pixel is an oblique edge, and when D = 1, the target pixel is isolated point noise that is uncorrelated with surrounding pixels. .

  Next, the aperture signal generation unit 2 will be described with reference to FIG. 21 is a vertical BPF processing means, 22 is a horizontal narrowband LPF processing means, 23 is a horizontal wideband LPF processing means, 24 is a horizontal BPF processing means, 25 is a vertical narrowband LPF processing means, and 26 is a vertical wideband. LPF processing means. BPF is a band pass filter, and LPF is a low pass filter. The narrow band LPF processing units 22 and 25 are LPFs having a narrower frequency band than the wide band LPF processing units 23 and 26.

  A system in which the vertical BPF processing unit 21 and the horizontal narrowband LPF processing unit 22 are connected in cascade emphasizes the edge in the vertical direction based on the input image, and generates the first vertical aperture correction signal vapc_n. A system in which the vertical BPF processing unit 21 and the horizontal broadband LPF processing unit 23 are connected in cascade emphasizes the edge in the vertical direction based on the input image, and generates the second vertical aperture correction signal vapc_w. A system in which the horizontal BPF processing unit 24 and the vertical narrowband LPF processing unit 25 are connected in cascade enhances the horizontal edge based on the input image and generates the first horizontal aperture correction signal hapc_n. A system in which the horizontal BPF processing unit 24 and the vertical wideband LPF processing unit 26 are connected in cascade emphasizes the edge in the horizontal direction based on the input image, and generates the second horizontal aperture correction signal hapc_w.

  The vertical direction BPF processing means 21 and the horizontal direction BPF processing means 24 each perform filter processing for extracting the mid-band component included in the image. An example of the frequency amplitude characteristics of the BPF filter used here is shown in FIG.

  The horizontal narrowband LPF processing means 22, the horizontal wideband LPF processing means 23, the vertical narrowband LPF processing means 25, and the vertical wideband LPF processing means 26 perform the following filter processing, respectively. That is, the LPF processing means 22, 23, 25, and 26 perform a filter process that allows only a low-frequency component to pass through for the frequency component that is orthogonal to the frequency component extracted by the preceding band-pass filter process. An example of the frequency characteristics of each LPF is shown in FIG.

  Next, the aperture adaptive control unit 3 will be described. As shown in FIG. 1, in the aperture adaptive control unit 3, 31 is a first aperture synthesis processing means, 32 is a second aperture synthesis processing means, and 33 is a switching means.

  The first aperture synthesis processing means 31 uses the aforementioned edge direction estimation coefficient T so as to generate an aperture correction signal suitable for the direction of the edge at the pixel of interest. Then, the first aperture synthesis processing means 31 synthesizes the first horizontal aperture correction signal hapc_n and the first vertical aperture correction signal vapc_n as shown in equations (5) and (6), and the aperture Outputs correction signal apc1.

apc1 = hapc_n × T + vapc_n × (1.0-T) (when 1.0 ≧ T ≧ 0.0) (5)
apc1 = vapc_n × | T | + hapc_n × (1.0-| T |) (when 0.0> T ≥ -1.0) (6)

  Here, FIG. 7 shows a region extracted by a horizontal aperture correction signal and a region extracted by a vertical aperture correction signal for a CZP (circular zone plate) image as shown in FIG. It is the schematic diagram drawn on the frequency space. In FIG. 7, the horizontal axis indicates the frequency component in the horizontal direction, and the vertical axis indicates the frequency component in the vertical direction. Moreover, only the first quadrant is displayed in consideration of the symmetry of CZP. When the horizontal aperture correction signal and the vertical aperture correction signal are combined, if the LPF band processed in the orthogonal direction with respect to the BPF is wide, as shown in FIG. 7, the overlapping portion of the two types of aperture correction signals , The aperture correction signal is distorted. As a result, rattling occurs at the edge of the diagonal line. Therefore, in the first aperture synthesis processing means 31 of the present embodiment, the first horizontal aperture correction signal hapc_n in which the LPF band orthogonal to the BPF is set to a narrow band, and the first vertical direction The aperture correction signal vapc_n is synthesized. Thereby, uniform edge enhancement without distortion can be performed in any direction from the horizontal direction to the vertical direction.

  By the way, when the LPF process in the orthogonal direction is performed with respect to the BPF process, the spatial correlation between the pixel of interest and the surrounding pixels increases. Therefore, when noise is superimposed on a flat region without an edge, the noise component is stretched in the direction in which the LPF process is performed, and there is a problem that the graininess is deteriorated and the image quality is deteriorated. In particular, as the LPF band becomes narrower, the adverse effect of the LPF on noise becomes more conspicuous. Therefore, the second aperture synthesis processing means 32 uses the second horizontal aperture correction signal hapc_w and the second vertical aperture correction signal vapc_w in which the LPF band in the orthogonal direction is set to a wide band. Then, the second aperture synthesis processing means 32 outputs an aperture correction signal apc2 having a noise granularity by, for example, synthesizing as in Expression (7).

    apc2 = (hapc_w + vapc_w) / 2 (7)

  In the switching unit 33, the output of the first aperture synthesis processing unit 31 considering the edge direction and the second aperture synthesis processing unit 32 considering the graininess of the noise in the flat portion are determined by the uncorrelation determination flag D. The output is switched and the final aperture correction signal is output.

  When the non-correlation determination flag D is 0, the switching unit 33 can be regarded as having a correlation with respect to a specific direction between the target pixel and the surrounding pixels, and thus is generated by the first aperture synthesis processing unit 31. Selects and outputs an aperture correction signal apc1 in consideration of the edge direction.

  When the uncorrelation determination flag D is 1, the pixel of interest and the surrounding pixels can be regarded as uncorrelated, that is, a flat portion having no edge. Therefore, the switching means (selection means) 33 selects the aperture correction signal apc2 generated by the second aperture synthesis processing means 32 so as not to impair the granularity of noise when noise is present in the area. And output.

  As described above, the aperture correction signal generation circuit according to the first embodiment outputs the aperture correction signal apc2 that does not impair the granularity of noise in the flat portion when the pixel of interest and the surrounding pixels are uncorrelated. . When there is a correlation in a specific direction between the pixel of interest and the surrounding pixels, it is possible to output the aperture correction signal apc1 along the direction of the correlation. Thereby, it is possible to generate an aperture correction signal having good noise granularity and no distortion.

  Next, an imaging apparatus using the aperture correction signal generation processing circuit of the above embodiment will be described.

  FIG. 17A is a block diagram illustrating an imaging apparatus using the aperture correction signal generation processing circuit of the present embodiment. The imaging device is a digital video camera, a digital camera, or the like. In FIG. 17A, 201 is an optical system including a lens and a diaphragm, 202 is a mechanical shutter (illustrated as a mechanical shutter), and 203 is a solid-state imaging device. 204 is a CDS circuit that performs analog signal processing, 205 is an A / D converter that converts an analog signal into a digital signal, and 206 is a signal that operates the image sensor 203, the CDS circuit 204, and the A / D converter 205. This is a timing signal generation circuit that generates the timing signal. Reference numeral 207 denotes a drive circuit for the optical system 201, mechanical shutter 202, and image sensor 203; 208, a signal processing circuit that performs signal processing necessary for captured image data; and 209, an image memory that stores the image data subjected to signal processing. It is. 210 is an image recording medium (shown as a recording medium) that can be removed from the imaging device, 211 is a recording circuit that encodes and compresses signal-processed image data, and records the image data on the image recording medium 210, and 212 is a signal processing This is an image display device for displaying the image data. Reference numeral 213 denotes a display circuit that displays an image on the image display device 212, and reference numeral 214 denotes a system control unit that controls the entire imaging device. 215 is a non-volatile storage for storing a program describing a control method executed by the system control unit 214, control data such as parameters and tables used when executing the program, and correction data such as a scratch address Memory (ROM). 216 transfers and stores the program, control data, and correction data stored in the non-volatile memory 215, and becomes a volatile memory (RAM) used when the system control unit 214 controls the imaging apparatus. Yes.

  Hereinafter, a photographing operation using the mechanical shutter 202 using the imaging device configured as described above will be described. Prior to the photographing operation, necessary programs, control data, and correction data are transferred from the nonvolatile memory 215 to the volatile memory 216 and stored at the start of the operation of the system control unit 214 such as when the imaging apparatus is turned on. Shall. These programs and data are used when the system control unit 214 controls the imaging apparatus. In addition, these programs and data are transferred as necessary from the nonvolatile memory 215 to the volatile memory 216, or the system control unit 214 directly reads the data in the nonvolatile memory 215. use.

  First, the optical system 201 drives a diaphragm and a lens according to a control signal from the system control unit 214 to form a subject image set to an appropriate brightness on the image sensor 203. Next, the mechanical shutter 202 is driven by the control signal from the system control unit 214 so as to shield the image sensor 203 in accordance with the operation of the image sensor 203 so that a necessary exposure time is reached. At this time, when the image pickup element 203 has an electronic shutter function, it may be used together with the mechanical shutter 202 to secure a necessary exposure time. The image sensor 203 is driven by a drive pulse based on the operation pulse generated by the timing signal generation circuit 206 controlled by the system control unit 214, and converts the subject image into an electrical signal by photoelectric conversion as an analog image signal. Output. The analog image signal output from the image sensor 203 is subjected to an operation pulse generated by the timing signal generation circuit 206 controlled by the system control unit 214, and the clock synchronization noise is removed by the CDS circuit 204. The A / D converter In 205, it is converted into a digital image signal. Next, a signal processing circuit 208 controlled by the system control unit 214 generates a luminance / color difference signal from the digital image signal (denoted as RAW) output from the A / D converter 205.

  Here, the configuration and operation of the signal processing circuit 208 including the aperture correction signal generation processing circuit of the present embodiment will be described with reference to FIG.

  In FIG. 17B, 301 is a luminance / color difference conversion processing circuit, 302 is an aperture correction signal generation processing circuit, 303 is a base clip processing circuit, 304 is a gain processing circuit, and 305 is an addition circuit.

  The luminance / color difference conversion processing circuit 301 extracts the color difference signals Cr and Cb and the low-frequency luminance signal YL from the digital image signal output from the A / D converter 205.

  The aperture correction signal generation processing circuit 302 differs from the digital image signal output from the A / D converter 205 in two types of vertical aperture correction signals having different orthogonal LPF bands and different LPF bands in the orthogonal direction. Two types of horizontal aperture correction signals are generated. Then, the aperture correction signal generation processing circuit 302 detects the degree of uncorrelation between the edge direction of the pixel of interest and the surrounding pixels of the pixel of interest. Then, the plurality of aperture correction signals are adaptively combined according to the degree of non-correlation between the edge direction of the pixel of interest and the peripheral pixels of the pixel of interest.

  As described above, as described above, when the correlation between the target pixel and the surrounding pixels is low, the vertical aperture correction signal with a wide band of orthogonal LPFs and the horizontal direction with a wide band of orthogonal LPFs. Aperture correction signals are averaged and output. When the correlation between the target pixel and surrounding pixels is high, a vertical aperture correction signal with a narrow orthogonal LPF band and a horizontal aperture correction signal with a narrow orthogonal LPF band are synthesized along the edge direction. And output.

  The base clip processing circuit 303 regards the adaptively synthesized aperture correction signal output from the aperture correction signal generation processing circuit 302 as 0 when the signal amplitude is smaller than a predetermined value. Clip. The amplitude level regarded as noise is controlled by the system control unit 214 so as to be an appropriate value according to the shooting mode and the luminance level of the subject.

  The gain processing circuit 304 adjusts the aperture correction signal by applying a gain to the aperture correction signal clipped with a predetermined small amplitude component, which is an output of the base clip processing circuit 303. The gain level is controlled by the system control unit 214 so as to be an appropriate value according to the shooting mode and the luminance level of the subject.

  The adding circuit 305 adds the gain-adjusted aperture correction signal, which is the output of the gain processing circuit 304, to the low-frequency luminance signal YL, which is the output of the luminance / color difference conversion processing circuit 301, and obtains the final luminance signal Y. Generate.

  As described above, the signal processing circuit 208 generates image data including the luminance signal Y and the color difference signals Cr and Cb from the digital image signal (denoted as RAW) output from the A / D converter 205.

  Here, the image memory 209 is used to temporarily store a digital image signal during signal processing or to store a digital image signal subjected to signal processing. The image data signal-processed by the signal processing circuit 208 and the image data stored in the image memory 209 are encoded and compressed into data suitable for the image recording medium 210 by the recording circuit 211 and recorded on the image recording medium 210. The Data suitable for the image recording medium 210 is, for example, file system data having a hierarchical structure. Further, the image data signal-processed by the signal processing circuit 208 and the image data stored in the image memory 209 are converted in resolution by the signal processing circuit 208 into image data by a resolution conversion processing circuit (not shown). Thereafter, the image data is converted into a signal suitable for the image display device 212 (for example, an NTSC analog signal) by the display circuit 213 and displayed on the image display device 212.

  The signal processing circuit 208 may output the digital image signal as it is to the image memory 209 or the recording circuit 211 without performing signal processing by the control signal from the system control unit 214 as it is.

  Further, when requested by the system control unit 214, the signal processing circuit 208 outputs information on digital image signals and image data generated in the signal processing process to the system control unit 214. The information is, for example, information such as the spatial frequency of the image, the average value of the designated area, the data amount of the compressed image, or information extracted from them. Further, the recording circuit 211 outputs information such as the type and free capacity of the image recording medium 210 to the system control unit 214 when requested by the system control unit 214.

  Further, a reproduction operation when image data is recorded on the image recording medium 210 will be described. The recording circuit 211 reads image data from the image recording medium 210 by a control signal from the system control unit 214. Similarly, when the image data is a compressed image, the signal processing circuit 208 performs an image expansion process according to a control signal from the system control unit 214 and stores it in the image memory 209. The image data stored in the image memory 209 is subjected to resolution conversion processing by the signal processing circuit 208, converted to a signal suitable for the image display device 212 by the display circuit 213, and displayed on the image display device 212. .

  With the configuration described above, it is possible to generate a video signal to which an aperture correction signal with good noise granularity and no distortion is added.

(Second Embodiment)
FIG. 9 is a block diagram illustrating a schematic configuration example of an aperture correction signal generation processing circuit according to the second embodiment of the present invention. In FIG. 9, 1 is a control coefficient generation unit, 2 is an aperture signal generation unit, and 3 is an aperture adaptive control unit. Also in this embodiment, as in the first embodiment, the control coefficient generation unit 1 and the aperture signal generation unit 2 process each pixel signal captured from the image input terminal in parallel, and then the aperture adaptive control unit. In step 3, a final aperture correction signal is generated.

  This embodiment is different from the first embodiment in the uncorrelated estimation processing means 17 included in the control coefficient generation unit 1 and the third aperture synthesis processing means 34 included in the aperture adaptive control unit 3. The third aperture synthesis processing unit 34 synthesizes the output of the first aperture synthesis processing unit 31 and the output of the second aperture synthesis processing unit 32.

  Hereinafter, the configuration and operation of parts different from the first embodiment will be specifically described. The uncorrelated estimation processing means 17 uses the edge direction estimation coefficient T, the diagonal 45 ° direction correlation value d1_diff, the diagonal 135 ° direction correlation value d2_diff, and the predetermined thresholds th3 and th4, so that the pixel of interest is uncorrelated with the surrounding pixels. The uncorrelation degree Dk indicating the degree of the flat portion is output. However, th3> th4.

  That is, the non-correlation estimation processing means 17 estimates an edge composition degree indicating the degree to which the target pixel of the input image and its peripheral pixels form an edge in a specific direction. The edge configuration degree is represented by 1-Dk. The degree of decorrelation Dk is 0 when it is estimated that an edge is formed, and 1 when it is estimated that no edge is formed.

  Here, the procedure for calculating the degree of decorrelation Dk will be described using the flowchart of FIG. In FIG. 16, the decorrelation estimation process is started in step S30, and it is determined in step S31 whether the edge direction estimation coefficient T is zero. In step S31, if the edge direction estimation coefficient T is not 0, it is assumed that there is a correlation in a specific direction closer to the vertical direction or the horizontal direction, and 0 (correlated) is set to the uncorrelation degree Dk in step S37. To do. On the other hand, when the edge direction estimation coefficient T is 0 in step S31, the process proceeds to the conditional branch in step S32. If the absolute value of the difference between the diagonal 45 ° direction correlation value d1_diff and the diagonal 135 ° direction correlation value d2_diff is greater than or equal to th3 in step S32, 0 (correlated) is set to the uncorrelation degree Dk in step S33. If the condition is not satisfied in step S32, the process proceeds to the conditional branch in step S34.

  When the absolute value of the difference between the diagonal 45 ° direction correlation value d1_diff and the diagonal 135 ° direction correlation value d2_diff is greater than or equal to th4 and smaller than th3 in step S34, the process proceeds to step S35. In step S35, Dk = − (| d1_diff−d2_diff | −th3) / (th3−th4) is set as the decorrelation degree Dk. When the condition of step S34 is not satisfied, Dk = 1.0 (no correlation) is set in step S36.

  The non-correlation degree Dk takes an arbitrary value from 0.0 to 1.0. When Dk = 0, it is estimated that there is a correlation in a specific direction between the pixel of interest and the surrounding pixels, and Dk = 1.0. In some cases, it is estimated that the pixel of interest is an isolated point uncorrelated with surrounding pixels.

  The aperture adaptive control unit 3 uses the decorrelation degree Dk generated here. The third aperture synthesis processing unit 34 synthesizes the output apc1 of the first aperture synthesis processing unit 31 and the output acp2 of the second aperture synthesis processing unit 32 as shown in Expression (8), and obtains the final aperture. The correction signal apc_out is output.

    apc_out = apc2 x Dk + apc1 x (1.0 -Dk) (8)

  Here, the first aperture synthesis processing means 31 generates an aperture correction signal in consideration of the direction of the correlation in the pixel of interest, and the second aperture synthesis processing means 32 impairs the granularity of noise in the flat portion. An aperture correction signal synthesized so as not to be output is output.

  Therefore, by performing the synthesis process as shown in Expression (8), when the pixel of interest is uncorrelated with the surrounding pixels (when Dk takes a value close to 1.0), the granularity of noise in the flat portion is reduced. Aperture correction signal apc2 that does not damage is dominant. When there is a correlation in a specific direction between the pixel of interest and the surrounding pixels (when Dk takes a value close to 0.0), the aperture correction signal apc1 along the direction of the correlation becomes dominant.

  As described above, also in the configuration of the second embodiment, it is possible to generate an aperture correction signal with good noise granularity and no distortion for all pixels.

  Next, an imaging apparatus using the aperture correction signal generation processing circuit of the above embodiment will be described. A block diagram showing an imaging apparatus using the aperture correction signal generation processing circuit of the present embodiment is as shown in FIG. In this imaging apparatus, the configuration and operation of parts other than the aperture correction signal generation processing circuit in FIG. 17B are the same as those in the first embodiment, and therefore the description is omitted and the aperture correction signal generation processing circuit 302 is omitted. Only the outline of the process will be described.

  The aperture correction signal generation processing circuit 302 differs from the digital image signal output from the A / D converter 205 in two types of vertical aperture correction signals having different orthogonal LPF bands and different LPF bands in the orthogonal direction. Two types of horizontal aperture correction signals are generated. Then, the aperture correction signal generation processing circuit 302 detects the degree of uncorrelation between the edge direction of the pixel of interest and the surrounding pixels of the pixel of interest. Then, the plurality of aperture correction signals are adaptively combined according to the degree of non-correlation between the edge direction of the pixel of interest and the peripheral pixels of the pixel of interest.

  Specifically, as described above, along the edge direction, a vertical aperture correction signal with a narrow orthogonal LPF bandwidth and a horizontal aperture correction signal with a narrow orthogonal LPF bandwidth are combined. The result of the first aperture synthesis process is generated. Then, a result of a second aperture synthesis process is generated by averaging the vertical aperture correction signal with a wide band of orthogonal LPFs and the horizontal aperture correction signal with a wide band of orthogonal LPFs. Then, the results of the first and second aperture synthesis processes are further synthesized according to the degree of uncorrelation in the pixel of interest, and a final aperture correction signal is output.

  With the configuration described above, it is possible to generate a video signal to which an aperture correction signal with good noise granularity and no distortion is added.

(Third embodiment)
FIG. 10 is a block diagram illustrating a schematic configuration example of an aperture correction signal generation processing circuit according to the third embodiment of the present invention.

  In FIG. 10, 1 is a control coefficient generation unit, 2 is an aperture signal generation unit, and 35 is a fourth aperture synthesis processing means. In the present embodiment, similarly to the first and second embodiments, the control coefficient generation unit 1 and the aperture signal generation unit 2 process the signal of each pixel captured from the image input terminal in parallel, A final aperture correction signal is generated by the four aperture synthesis processing means 35.

  Hereinafter, the configuration and operation of the control coefficient generation unit 1, the aperture signal generation unit 2, and the fourth aperture synthesis processing unit 35 will be described.

  The control coefficient generation unit 1 includes a vertical direction correlation detection processing unit 11, a horizontal direction correlation detection processing unit 12, an edge direction estimation processing unit 13, an oblique 45 ° direction correlation detection processing unit 14, an oblique 135 ° direction correlation detection processing unit 15, The uncorrelated estimation processing means 17 is comprised. Then, the control coefficient generation unit 1 outputs an edge direction estimation coefficient T that indicates the direction of the edge of the pixel of interest and an uncorrelation degree Dk that indicates the degree to which the pixel of interest is a flat part uncorrelated with the surrounding pixels.

  The configurations and operations of the vertical direction correlation detection processing means 11, the horizontal direction correlation detection processing means 12, the oblique 45 ° direction correlation detection processing means 14, the oblique 135 ° direction correlation detection processing means 15, and the edge direction estimation processing means 13 are as follows. This is the same as the embodiment. That is, the processing means 11, 12, 14, 15, and 13 generate the vertical correlation value vdiff, the horizontal correlation value hdiff, the diagonal 45 ° direction correlation value d1_diff, the diagonal 135 ° direction correlation value d2_diff, and the edge direction estimation coefficient T, respectively. To do.

  The correspondence relationship between the edge direction estimation coefficient T and the angle θ indicating the direction of the edge is the same as in the first embodiment.

  By the way, when the edge direction estimation coefficient T = 0.0, the pixel of interest may be included in an oblique line pattern as shown in FIG. 5, or may be an isolated point as shown in FIG. Therefore, in the non-correlation estimation processing means 17, the pixel of interest is a flat part that is uncorrelated with the surrounding pixels by processing using the diagonal 45 ° direction correlation value d1_diff, the diagonal 135 ° direction correlation value d2_diff, and the edge direction estimation coefficient T. The uncorrelation degree Dk indicating the degree is output.

  The degree of uncorrelation Dk takes an arbitrary value from 0.0 to 1.0. When Dk = 0, there is a high possibility that the pixel of interest is an oblique edge, and when Dk = 1.0, the pixel of interest is not a neighboring pixel. It can be considered that there is a high possibility of being a correlated isolated point. Since the decorrelation estimation processing means 17 is the same as that of the second embodiment, description thereof is omitted.

  Next, the aperture signal generation unit 2 will be described. The aperture signal generation unit 2 includes a vertical BPF processing unit 21, a horizontal LPF processing unit 27, a horizontal BPF processing unit 24, and a vertical LPF processing unit 28, and includes a vertical aperture correction signal vapc and a horizontal aperture correction. Output the signal hapc.

  The vertical aperture correction signal vapc is generated by a system in which the vertical BPF processing means 21 and the horizontal LPF processing means 27 are connected in cascade. The horizontal direction aperture correction signal hapc is generated by a system in which the horizontal direction BPF processing means 24 and the vertical direction LPF processing means 28 are connected in cascade. The vertical BPF processing means 21 and the horizontal LPF processing means 24 are the same as those in the first embodiment.

  The feature of the third embodiment is that the pass frequency band of the LPF processing performed by the horizontal direction LPF processing unit 27 and the vertical direction LPF processing unit 28 is controlled by using the degree of decorrelation Dk.

  When the degree of decorrelation Dk is 1 or close to 1 (when it is determined that the pixel of interest is a flat part uncorrelated with the surrounding pixels), the horizontal LPF processing unit 27 and the vertical LPF processing unit 28 Increase the LPF bandwidth. As a result, it is possible to prevent the noise from being stretched in the horizontal direction or the vertical direction and the graininess from being impaired.

  On the other hand, when the degree of decorrelation Dk is 0 or close to 0 (when it is determined that the pixel of interest and its surrounding pixels constitute an edge), the horizontal direction LPF processing unit 27 and the vertical direction LPF processing unit 28 Reduce the LPF bandwidth. Thereby, an aperture correction signal suitable for the direction of the edge can be generated.

  Here, LPF band limiting methods using uncorrelated Dk include, for example, a method in which LPFs of a plurality of types of bands are selected based on uncorrelation degree Dk, or non-correlated between narrowband LPFs and wideband LPFs. A method of synthesizing using the degree Dk is mentioned. However, it may be other than that. Further, the decorrelation determination flag D of the first embodiment may be used instead of the decorrelation degree Dk.

  Finally, the fourth aperture synthesis processing means 35 will be described. The fourth aperture synthesis processing means 35 uses the edge direction estimation coefficient T to generate an aperture correction signal apc_out suitable for the edge direction at the target pixel.

  As shown in Expressions (9) to (11), when T> 0.0, the weights of hapc and vapc are weighted by T so that the horizontal aperture correction signal hapc is dominant and output. When T <0.0, the hepc and vapc are weighted with | T | and output so that the aperture correction signal vapc in the vertical direction becomes dominant. When T = 0.0, the average of hapc and vapc is output.

apc_out = hapc × T + vapc × (1.0 − T) (when T> 0.0) (9)
apc_out = vapc × | T | + hapc × (1.0 − | T |) (when T <0.0) (10)
apc_out = (hapc + vapc) /2.0 (when T = 0.0) (11)

  In the above equations (9) to (11), hapc and vapc are aperture correction signals that are appropriately band-limited by the decorrelation degree Dk. The distortion of the aperture correction signal due to interference with vapc is small. In addition, when the target pixel is a flat portion, it is possible to reduce the loss of graininess due to the noise component being stretched in the horizontal direction or the vertical direction by LPF processing.

  As described above, in the aperture correction signal generation processing circuit of the third embodiment, it is possible to perform aperture correction processing with good noise granularity and no distortion for all pixels.

  Next, an imaging apparatus using the aperture correction signal generation processing circuit of the above embodiment will be described. A block diagram showing an imaging apparatus using the aperture correction signal generation processing circuit of the present embodiment is as shown in FIG. In this imaging apparatus, the configuration and operation of the parts other than the aperture correction signal generation processing circuit 302 in FIG. 17B are the same as those in the first embodiment, and therefore the description is omitted and the aperture correction signal generation processing circuit is omitted. Only the outline of the processing at 302 will be described.

  The aperture correction signal generation processing circuit 302 generates a vertical aperture correction signal and a horizontal aperture correction signal from the digital image signal output from the A / D converter 205. Then, the aperture correction signal generation processing circuit 302 detects the degree of uncorrelation between the edge direction of the pixel of interest and the surrounding pixels of the pixel of interest. Here, the vertical aperture correction signal and the horizontal aperture correction signal are output with the LPF band in the orthogonal direction adjusted according to the degree of uncorrelation between the pixel of interest and the surrounding pixels, respectively. . Then, the vertical aperture correction signal in which the orthogonal LPF band is adaptively controlled and the horizontal aperture correction signal are further synthesized according to the direction of the edge in the target pixel, and finally combined. Output as a correct aperture correction signal.

  With the configuration described above, it is possible to generate a video signal to which an aperture correction signal with good noise granularity and no distortion is added.

  According to the first to third embodiments, the phenomenon of noise components diffusing in a specific direction is reduced in the flat portion of the input image, so that the noise granularity can be kept good. In addition, when there is a correlation between the target pixel and the surrounding pixels, an aperture correction signal without distortion can be synthesized in consideration of the direction of the correlation, and an edge with uniform intensity in all directions Can be added.

  Note that the aperture correction signal generation processing circuits in FIGS. 1, 9, and 10 are not limited to hardware configurations, and may be executed by software of a program. In that case, the ROM 215 stores a program for causing the system control unit (computer) 214 of FIG. 17A to execute the processing method of the circuits of FIGS. The system control unit (computer) 214 executes the same processing as the aperture correction signal generation processing circuit of FIGS. 1, 9, and 10 by executing the program in the ROM 215. The program can be recorded on a recording medium readable by the system control unit 214 (computer) such as a memory card.

  The present embodiment can also be realized by the computer of the system control unit 214 executing a program. Also, means for supplying a program to a computer, for example, a computer-readable recording medium such as a CD-ROM recording such a program, or a transmission medium such as the Internet for transmitting such a program is also applied as an embodiment of the present invention. Can do. A computer program product such as a computer-readable recording medium in which the above program is recorded can also be applied as an embodiment of the present invention. The above program, recording medium, transmission medium, and computer program product are included in the scope of the present invention. As the recording medium, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.

  The above-described embodiments are merely examples of implementation in carrying out the present invention, and the technical scope of the present invention should not be construed in a limited manner. That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

It is a block diagram which shows the structural example of the aperture correction signal generation processing circuit of the 1st Embodiment of this invention. It is a schematic diagram which shows the arrangement | sequence of the pixel sampled in the grid | lattice form. It is a figure which shows the frequency characteristic of a horizontal direction BPF process and a vertical direction BPF process. It is a figure which shows the frequency characteristic of a horizontal direction LPF process and a vertical direction LPF process. It is a figure which shows the example of an image pattern in case a focused pixel is an edge of a diagonal direction in a local area | region. It is a figure which shows the example of an image pattern in case a focused pixel is an isolated point in a local area | region. It is the schematic diagram which synthesize | combined the area | region extracted with a horizontal aperture correction circuit and a vertical aperture correction circuit. It is a figure which shows a CZP (circular zone plate) image. It is a block diagram which shows the structural example of the aperture correction signal generation processing circuit of the 2nd Embodiment of this invention. It is a block diagram which shows the structural example of the aperture correction signal generation processing circuit of the 3rd Embodiment of this invention. It is a flowchart which shows an edge direction estimation process. 6 is a graph showing a relationship of an edge direction estimation coefficient T with respect to a difference between a vertical correlation value and a horizontal correlation value. It is the model which defined angle (theta) which shows the direction of an edge. 5 is a graph showing a relationship between an angle θ indicating the direction of an edge and a correlation coefficient T. It is a flowchart of the uncorrelation estimation process of 1st Embodiment. It is a flowchart which shows the decorrelation estimation process of 2nd Embodiment. It is a block diagram which shows the structural example of the imaging device using an aperture correction signal generation processing circuit.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Control coefficient generation unit 2 Aperture signal generation unit 3 Aperture adaptive control unit 16 Uncorrelated estimation processing means
vdiff vertical correlation value
hdiff horizontal correlation value
d1_diff Oblique 45 ° correlation value
d2_diff Oblique 135 ° direction correlation value T Edge direction estimation coefficient D Uncorrelated flag
LPF low pass filter
BPF bandpass filter
vapc_n Vertical aperture correction signal
vapc_w Vertical aperture correction signal
hapc_n Horizontal aperture correction signal
hapc_w Horizontal aperture correction signal
apc1 Aperture correction signal
apc2 Aperture correction signal 201 Optical system 202 Mechanical shutter 203 Image sensor 204 CDS circuit 205 A / D converter 206 Timing signal generation circuit 207 Drive circuit 208 Signal processing circuit 209 Image memory 210 Image recording medium 211 Recording circuit 212 Image display device 213 Display Circuit 214 System control unit 215 Nonvolatile memory (ROM)
216 Volatile memory (RAM)
301 YC matrix processing circuit 302 Aperture correction signal generation processing circuit 303 Base clip processing circuit 304 Gain processing circuit 305 Adder circuit

Claims (7)

It has a vertical and horizontal bandpass filter and vertical and horizontal low pass filter, generating an aperture correction signal in the vertical and horizontal directions emphasized the respective contour of the vertical and horizontal directions based on the input image Aperture correction signal generating means for performing,
Contour estimation means for estimating whether the pixel of interest of the input image and its surrounding pixels constitute a diagonal contour;
Contour direction estimation means for estimating the degree of whether the direction of the contour of the pixel of interest is a vertical direction or a horizontal direction based on signals of the pixel of interest of the input image and surrounding pixels; and
The aperture correction signal generating means performs a weighted synthesis of the vertical and horizontal aperture correction signals processed by the narrow band low-pass filter in accordance with the degree of the vertical direction or the horizontal direction, and performs narrow band aperture. generating a correction signal, the combined wideband aperture correction signal by a predetermined ratio of vertical and horizontal aperture correction signal processed by the wideband low-pass filter in said aperture correction signal generation means and a combining means that generates ,
Said combining means, said said outputting the narrow-band aperture correction signal is estimated to constitute an oblique direction of the contour by the contour estimation means, is estimated not to constitute a diagonal contour by the contour estimator broadband An image processing apparatus that outputs an aperture correction signal . It has vertical and horizontal bandpass filters and vertical and horizontal lowpass filters, and generates vertical and horizontal aperture correction signals by emphasizing vertical and horizontal contours based on the input image, respectively. Aperture correction signal generating means for performing,
Contour estimation means for estimating the degree to which the pixel of interest of the input image and its peripheral pixels form a diagonal contour;
Contour direction estimation means for estimating the degree of whether the direction of the contour of the pixel of interest is a vertical direction or a horizontal direction based on signals of the pixel of interest of the input image and surrounding pixels; and
The aperture correction signal generating means performs a weighted synthesis of the vertical and horizontal aperture correction signals processed by the narrow band low-pass filter in accordance with the degree of the vertical direction or the horizontal direction, and performs narrow band aperture. Generating a correction signal, and combining the vertical and horizontal aperture correction signals processed by the wideband low-pass filter in the aperture correction signal generation unit at a predetermined ratio to generate a wideband aperture correction signal;
The synthesizing unit increases the weighting of the narrowband aperture correction signal as the degree of constructing the oblique contour estimated by the contour estimation unit increases, and decreases the degree of constructing the oblique contour as the wideband aperture correction. An image processing apparatus characterized in that the narrowband aperture correction signal and the wideband aperture correction signal are combined and output so that the weighting of the signal is increased . The image processing apparatus according to claim 1 or 2 ,
An imaging apparatus comprising: an imaging element that generates an image signal by photoelectric conversion. With vertical and horizontal bandpass filter and vertical and horizontal low pass filter, generating an aperture correction signal in the vertical and horizontal directions emphasized the respective contour of the vertical and horizontal directions based on the input image Aperture correction signal generation step to perform,
A contour estimation step for estimating whether or not the pixel of interest of the input image and its peripheral pixels constitute a diagonal contour;
A contour direction estimation step for estimating a degree of whether the direction of the contour of the target pixel is a vertical direction or a horizontal direction based on signals of the target pixel of the input image and its surrounding pixels;
In the aperture correction signal generating step, the vertical and horizontal aperture correction signals processed by the narrow band low-pass filter are weighted and synthesized corresponding to the degree of the vertical direction or the horizontal direction. generating a correction signal, the combined wideband aperture correction signal in the vertical direction and the horizontal direction of the aperture correction signal processed by the wideband low-pass filter in said aperture correction signal generation step in a predetermined ratio and a synthesis step that generates ,
The synthesis step, the to be estimated and said to be estimated to constitute an oblique direction of the contour by the contour estimation step outputs a narrowband aperture correction signal, does not constitute a diagonal contour by the contour estimation step Wideband An image processing method characterized by outputting an aperture correction signal . Generate vertical and horizontal aperture correction signals by emphasizing vertical and horizontal contours based on the input image using vertical and horizontal bandpass filters and vertical and horizontal lowpass filters, respectively. Aperture correction signal generation step to perform,
A contour estimation step for estimating the degree to which the target pixel of the input image and its peripheral pixels form a diagonal contour;
A contour direction estimation step for estimating a degree of whether the direction of the contour of the target pixel is a vertical direction or a horizontal direction based on signals of the target pixel of the input image and its surrounding pixels;
In the aperture correction signal generating step, the vertical and horizontal aperture correction signals processed by the narrow band low-pass filter are weighted and synthesized corresponding to the degree of the vertical direction or the horizontal direction. Generating a correction signal, and combining the vertical and horizontal aperture correction signals processed by the wideband low-pass filter in the aperture correction signal generation step at a predetermined ratio to generate a wideband aperture correction signal;
In the synthesis step, the greater the degree of constituting the oblique contour estimated in the contour estimating step, the greater the weighting of the narrowband aperture correction signal, and the smaller the degree of constituting the oblique outline, the wider band aperture correction. An image processing method comprising: synthesizing and outputting the narrowband aperture correction signal and the wideband aperture correction signal so that signal weighting is increased . The program for making a computer perform each step of the image processing method of Claim 4 or 5 . A computer-readable recording medium on which the program according to claim 6 is recorded.

Images