JP2005295302A - Camera image processing device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a camera image processing device capable of noise reduction with small image quality impairment due to camera shake by a simple configuration and processing, especially when photographing in low illuminance. <P>SOLUTION: Image data outputted from an imaging circuit 101 is input into a camera image processing device 100, and stored in a memory device 105 through a memory control circuit 102. Moreover, edge intensity is outputted from a differentiating circuit 106, and pixel information in which the edge intensity becomes maximum is outputted from a feature pixel extraction circuit 107. The information of the feature pixel is stored in a feature pixel information memory circuit 108 as feature pixel information 1 frame before. Output of the feature pixel extraction circuit 107 and the output of the feature pixel information holding circuit 108 are processed in a feature pixel determination circuit 109, and it is determined whether the same feature point is represented or not. If they are the same feature point, positions of the image data one frame before or after are corrected, and noise reduction is performed in a noise reduction filter 110. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to image quality correction of images continuously captured by a camera, and more particularly to noise reduction of images captured under low illumination conditions.

  A conventional camera image processing apparatus obtains image data by performing image processing on an electrical signal photoelectrically converted by an image sensor such as a CCD or C-MOS, but in general, it is random due to fluctuations in supply voltage or the influence of peripheral circuits. Noise is generated, and particularly in the case of image data taken under low illumination conditions, the image quality deterioration is conspicuous. This random noise can be reduced by averaging using pixel data around the pixel of interest. However, when this averaging process is performed, the image becomes smooth, but high-frequency signal components such as image edge information are lost. . In addition, when the target image data is a moving image, the same averaging process can be performed by using image data captured continuously in time, but even if the subject is stationary, camera shake, etc. As a result of the pixel shift, the edge portion of the subject will be blurred if filter processing such as averaging is performed as it is.

As a measure to improve this, pixel displacement is corrected using a plurality of frame image data. If the input image has a displacement at the pixel level, the image is aligned by function fitting or template matching. After performing this, there is one that reduces random noise by an averaging process (see, for example, Patent Document 1).
Japanese Patent No. 289772 (paragraph 0019, paragraph 0021, FIG. 1)

  In a conventional camera image processing apparatus, if a positional deviation occurs at the pixel level in an input image, the image is aligned by function fitting or template matching, and then subjected to filter processing such as averaging to generate random noise. The image quality degradation at the edge of the image has been suppressed by reducing the image quality, but it is difficult to increase the speed due to the complexity of these processes, making it unsuitable for continuously input image data. there were.

  The present invention has been made to solve the above-described problems. A camera image processing apparatus capable of reducing noise with little image quality degradation due to camera shake, particularly in low-light shooting, with a simple configuration and processing. It is to provide.

  According to a first aspect of the present invention, there is provided a camera image processing apparatus comprising: storage means for storing at least input first image data and second image data; and differentiation means for performing a differentiation operation on each image data. , Calculating the intensity and direction of the edge of each image from the output of the differentiating means, and extracting a pixel having a maximum edge intensity as a characteristic pixel; and a first corresponding to the first image Feature pixel information holding means for holding feature pixel information until at least the detection process of the second feature pixel information corresponding to the second image in the feature pixel extraction means is completed; and first feature pixel information; Determining whether or not the feature pixel indicated by the second feature pixel information represents a feature pixel related to the same image, and outputting the feature pixel position information; Storage control means for controlling writing to and reading from the position, position information generation means for calculating a shift amount of the second image position with respect to the first image position based on the characteristic pixel position information, and the storage means The second image data is superimposed on the read first image data after correcting the positional deviation amount, and provided with noise reduction processing means for obtaining noise reduced image data by averaging calculation processing. Features.

  According to a second aspect of the present invention, there is provided the camera image processing apparatus according to the first aspect, further comprising characteristic pixel information limiting means for limiting the number of characteristic pixels output from the characteristic pixel extracting means according to a predetermined condition. It is characterized by.

  According to a third aspect of the present invention, there is provided the camera image processing apparatus according to the first aspect, wherein the second image position shift amount with respect to the first image position calculated by the position information generating means is A positional deviation corresponding to the number of pixels set by the positional deviation amount is added.

  The camera image processing apparatus according to the present invention extracts a pixel having a maximum edge strength obtained by differentiating input image data in the horizontal and vertical directions as a feature pixel, and detects an image shift amount between frames using the feature pixel. In addition, since noise reduction processing using image data of a plurality of frames is performed by performing pixel alignment, deterioration of image quality due to camera shake can be reduced by a simple configuration and processing.

Embodiment 1 FIG.
FIG. 1 is a block diagram showing a configuration of a camera image processing apparatus according to Embodiment 1 of the present invention. In the figure, a camera image processing apparatus 100 corrects misalignment that occurs between frames using edge information of image data that is sequentially input from the imaging circuit 101 in units of frames, and performs filter processing. A memory 105 that sequentially stores image data to be stored, a memory control circuit 102 that includes a memory control signal generation unit 103 and a position information generation unit 104, and that controls writing and reading operations to the memory 105, and a differential value of input image data , A feature pixel extraction circuit 107 that determines pixels having characteristic edge information in the frame from the result of the differentiation circuit 106, and feature pixel information holding that holds the result of the feature pixel extraction circuit 107 Circuit 108, the output of the feature pixel extraction circuit 107, and the feature pixel information holding circuit 108 A feature pixel that determines whether or not the current frame is subjected to filter processing by determining whether or not each feature pixel of two consecutive frames represents the same feature point from information on the previous frame image data A selection circuit 109, a noise reduction filter 110 that performs filtering using image data of a plurality of frames, and a selection that selects image data of the current frame and image data output of the noise reduction filter 110 from the result of the feature pixel determination circuit 109 The circuit 111 is configured.

  Next, the operation of the camera image processing apparatus 100 will be described with reference to the drawings.

  First, digital image data is sequentially input to the differentiation circuit 106 in frame units from the imaging circuit 101 outside the camera image processing apparatus 100, and the image is output in accordance with the write control signal and write address output from the memory control circuit 102. Data is stored in the memory 105.

  The differentiation circuit 106 applies primary differentiation to the input image data to perform differentiation in the vertical direction and the horizontal direction, and outputs a vertical differential value and a horizontal differential value for each pixel. . Here, the vertical direction and the horizontal direction correspond to the directions of the y-axis and the x-axis when the position of each pixel of the frame image data is expressed in a two-dimensional orthogonal coordinate system.

  Next, the feature pixel extraction circuit 107 generates edge strength by calculating the square root of the square sum of the vertical differential value and the horizontal differential value or the sum of the absolute values of all the pixels in the frame. Then, a feature pixel having a maximum edge strength is extracted, and feature pixel information including the position, edge strength, and direction of the feature pixel is generated. Here, the pixel position is an x coordinate and ay coordinate when the position of each pixel of the frame image data is expressed in a two-dimensional orthogonal coordinate system. Although there may be a case where there are a plurality of feature pixels having the maximum edge strength, in this embodiment, description will be made assuming that there is only one feature pixel in the frame.

  The feature pixel information generated by the feature pixel extraction circuit 107 is stored in the feature pixel information holding circuit 108, and when the processing on the image data of the next frame in the feature pixel extraction circuit 107 is completed, the output of the feature pixel extraction circuit 107 is output. And output to the feature pixel determination circuit 109. That is, with respect to the image data for two frames, the feature pixel information in each frame is output to the feature pixel determination circuit 109, and the feature pixel determination circuit 109 determines each of the edge strength and direction as the feature pixel information. The difference between frames is calculated to determine whether or not the feature pixels extracted in each frame represent the same feature pixel. When the feature pixels are determined to be the same, the amount of misalignment and the direction between the feature pixels are calculated. Output to the position information generation unit 104 of the memory control circuit 102.

  For example, image data as shown in FIG. 2A is input as the first frame, and the pixel Pa in the lower right corner surrounded by a dotted line becomes a pixel having characteristic edge information in this frame. FIG. 2C is an enlarged view of the periphery of the pixel Pa in FIG. The differentiation circuit 106 performs a differentiation operation on each pixel data of the first frame to obtain a vertical differential value and a horizontal differential value, and a feature pixel extraction circuit 107 calculates edge strength and direction at each pixel. As a result, the pixel Pa is extracted as a feature pixel. The vertical differential value and the horizontal differential value of the pixel Pa are respectively the arrow C2 and the arrow C1 in FIG. 2C, the edge strength is the size | C3 | of the arrow C3, and the direction of the arrow C3 is the arrow It is expressed as a result of vector addition of C1 and arrow C2. Information about the pixel Pa obtained in this way is output to the feature pixel determination circuit 109 in the next stage and to the feature pixel information holding circuit 108, and is output from the feature pixel extraction circuit 107 for the next second frame. It is held until the similar operation is completed.

  For the first frame, since the effective pixel determination circuit 109 does not hold valid information, it is once written in the memory 105, and the read data is output as it is without being processed by the noise reduction filter 110. Alternatively, special processing is performed such that the first frame data is not output.

  Image data as shown in FIG. 2B is input as the second frame, and the pixel Pb in the lower right corner becomes a pixel having characteristic edge information in the second frame, and FIG. 2 (B) is an enlarged display around the pixel Pb. The position of the image in FIG. 2A corresponds to the position of the broken line in FIG. 2B, and the image position is shifted by the arrow B1. The pixel Pb is extracted as a feature pixel by executing the same process as that of the first frame for the image data of the second frame. The vertical differential value and the horizontal differential value of the pixel Pb are respectively an arrow D2 and an arrow D1 in FIG. 2D, and the edge intensity in the feature pixel extraction circuit 107 is the magnitude | D3 | Is expressed as a result of vector addition of the arrow D1 and the arrow D2.

  The pixel position, edge strength, and direction information of the pixel Pa from the feature pixel information holding circuit 108 and the pixel Pb from the feature pixel extraction circuit 107 are input to the feature pixel determination circuit 109 during the processing of the second frame. For these two pixels, the difference in edge strength (| D3 | − | C3 |) and the difference in edge direction {(D1 + D2) − (C1 + C2)} are calculated, for example, each within a preset threshold range The pixels Pa and Pb are pixels having the same characteristics and represent the same pixel in each frame, and when the edge intensity difference and the direction difference exceed the threshold range, the pixels Pa and Pb Are determined to be uncorrelated pixels having different characteristics.

  The criterion for determining the correlation between the pixels may be set in advance or may be set from the outside.

  When it is determined that the pixels Pa and Pb are the same pixel, it is considered that the image data is shifted in the number and direction of pixels represented by the arrow B1 in FIG. The shift amount of the number of pixels in each of the x direction and the y direction, that is, the x component Bx and the y component By of the arrow B 1 is calculated from the position information of the pixels Pa and Pb, and the result is output to the memory control circuit 102. In the memory control circuit 102, when reading the image data of the second frame from the memory 105, the position information is set so that the reading start address is offset by Bx pixels in the x direction and by By pixels in the y direction with respect to the first frame. The generation unit 104 generates an address for memory control, and the control signal generation unit 103 generates a signal for actually controlling the memory such as a chip select signal and an output enable signal in synchronization with the output of the position information generation unit 104. The memory 105 is controlled to correct a pixel position shift between frames.

  That is, if only the data in the area Ra surrounded by the thick line in FIG. 2E is read and output from the image data of the first frame stored in the memory 105, the image data of the second frame is shown in FIG. The data of the region Rb surrounded by the thick line in FIG. 2F, which is translated from the region Ra of 2 (E) by the arrow B1, is read out.

  Therefore, the range in which the pixel position deviation can be corrected is limited by the number of pixels of image data that can be stored in the memory 105, the number of frame pixels of image data output from the camera image processing apparatus 100, and the like.

  Next, the noise reduction filter 110 reads out the image data stored in the memory 105 and performs a process of reducing noise, for example, taking an average of two consecutive frames of image data.

  When the selection circuit 111 determines that there is no correlation between the pixels Pa and Pb according to the determination result of the feature pixel determination circuit 109, the image data itself read from the memory 105 is the pixel Pa and Pb are the same feature pixels. When the determination is made, the output image data of the noise reduction filter 110 obtained by performing arithmetic processing on the image data read from the memory 105 is selected and output.

  The above process is repeated every time frame image data is input. However, for the data of the third and subsequent frames, the feature pixel determination circuit 109 determines that the feature pixels are the same image constituent elements during the processing for the previous frame, and the image data from the memory 105 is stored. When the readout address is corrected, the feature pixel position information held by the feature pixel information holding circuit 108 includes a deviation corresponding to the correction of the readout address in the previous frame. Therefore, it is necessary to replace the corrected position information and use it for processing in the feature pixel determination circuit 109.

  Although the present embodiment has been described as processing for image data for two consecutive frames, if the capacity of the memory 105 and the time required for writing / reading are not limited, image data for three frames or more is processed. Can also be configured to execute similar processing.

  With the processing described above, the pixel calculation for the sequentially input image data is corrected, and the filter operation is executed, so that the blur around the edge of the image can be minimized.

Embodiment 2. FIG.
FIG. 3 is a block diagram showing the configuration of Embodiment 2 of the inventive device. In this camera image processing device 200, the same parts as those in the camera image processing device 100 shown in FIG. 1 are denoted by the same reference numerals, and feature pixels are interposed between the feature pixel extraction circuit 107 and the feature pixel determination circuit 109. The difference is that a filter 201 is provided. The feature pixel filter 201 limits the number of feature pixels output to a subsequent circuit according to a set specific condition when a plurality of feature pixels are extracted by the feature pixel extraction circuit 107. A specific operation of the camera image processing apparatus 200 will be described below.

  When a plurality of pixels are extracted by the feature pixel extraction circuit 107, when all the information on the feature pixels is output to the next stage circuit, the feature pixel information holding circuit 108 holds information for the number of extracted feature pixels. A circuit is required, and the feature pixel determination circuit 109 needs to execute determination processing as many times as the number of extracted feature pixels. For this purpose, an execution time or a circuit for parallel processing is required. However, the circuit scale and the processing time increase as the total number of pixels of the input frame image data increases.

  In the present embodiment, in the feature pixel filter 201, for example, information further limited from a plurality of feature pixels extracted by setting a condition that only the last feature pixel in the frame is used is used for processing in the subsequent circuit. Thus, an increase in circuit scale and processing time can be suppressed, and a circuit independent of the total number of pixels of the input frame image data can be configured.

  Note that the filter conditions set in the feature pixel filter 201 may be set in advance, or may be configured to be set from the outside.

Embodiment 3 FIG.
FIG. 4 is a block diagram showing the configuration of Embodiment 3 of the inventive device. In this camera image processing apparatus 300, the same parts as those in the camera image processing apparatus 100 shown in FIG. 1 are denoted by the same reference numerals, and edge enhancement processing is performed instead of the position information generation unit 104 of the memory control circuit 102. The difference is that the position information generating unit 301 is used. A specific operation of the camera image processing apparatus 300 will be described below.

  In the position information generation unit 104 of the camera image processing apparatus 100, the address generation is controlled so that the edges of continuously input image data coincide with each other, but the noise reduction filter 110 does not depend on the content of the image data. When noise reduction processing is performed in this way, the edge portion is blurred. Therefore, when the edge enhancement processing position information generation unit 301 is used to superimpose image data of a plurality of frames to be processed by the noise reduction filter 110, the contour of the image is shifted by about one pixel in either the vertical or horizontal direction. By controlling the image data read address from the memory 105, an effect similar to the enhancement processing of the edge portion can be obtained, and blurring of the edge peripheral portion of the image in the noise reduction filter 110 can be suppressed.

  Note that the number and direction of pixels shifted by the edge enhancement processing position information generation unit 301 may be fixed in advance or may be set from the outside.

It is a block diagram which shows the structure of the camera image processing apparatus of Embodiment 1 of this invention. It is a figure for demonstrating the image shift detection operation | movement of the camera image processing apparatus of Embodiment 1 of this invention. It is a block diagram which shows the structure of the camera image processing apparatus of Embodiment 2 of this invention. It is a block diagram which shows the structure of the camera image processing apparatus of Embodiment 3 of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Camera image processing apparatus, 101 Imaging circuit, 102 Memory control circuit, 103 Memory control signal generation part, 104 Position information generation part, 105 Memory, 106 Differentiation circuit,
107 feature pixel extraction circuit, 108 feature pixel information holding circuit, 109 feature pixel determination circuit, 110 noise reduction filter, 111 selection circuit.

Claims (3)

In a camera image processing apparatus that performs image processing on digital image data obtained by camera photography,
Storage means for storing at least input first image data and second image data;
Differentiating means for performing a differentiation operation on each image data,
Calculating the intensity and direction of the edge of each image from the output of the differentiating means, and extracting a pixel having a maximum edge intensity as a characteristic pixel;
Feature pixel information for holding first feature pixel information corresponding to the first image until at least the detection processing of the second feature pixel information corresponding to the second image in the feature pixel extraction unit is completed Holding means;
Feature pixel determination means for determining whether or not the feature pixels indicated by the first feature pixel information and the second feature pixel information represent feature pixels related to the same image, and outputting the feature pixel position information;
Storage control means for controlling writing and reading of each image to the storage means;
Position information generating means for calculating a shift amount of the second image position with respect to the first image position based on the characteristic pixel position information;
Noise reduction processing means for correcting the positional deviation amount and superimposing the second image data on the first image data read from the storage means and obtaining noise reduced image data by averaging calculation processing; A camera image processing apparatus characterized by being provided.   The camera image processing apparatus according to claim 1, further comprising a feature pixel information limiting unit that limits the number of feature pixels output from the feature pixel extracting unit according to a predetermined condition. The positional deviation corresponding to the number of pixels set by the positional deviation amount is added to the deviation amount of the second image position with respect to the first image position calculated by the positional information generation unit. Item 2. The camera image processing device according to Item 1.

Images