JP2007110445A - Image processor and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processor effective for removing the impulsive noise among the noises. <P>SOLUTION: The image processor comprises n (n is 3 or greater) frame memories 14-1 to 14-n for storing frame data forming one- or two-dimensional images, an A/D converter 12 for converting outputs signals of solid state imaging devices to pixel data, a distribution circuit 13 for distributing the converted pixel data by the A/D converter frame by frame among the n frame memories, a rank processing circuit 15 for outputting one pixel data by rank-processing for removing the impulsive noise and the random noise after reading n pixel data at the same pixel position from the n frame memories, and a frame memory 16 for storing pixel data outputted from a data processing means as pixel data of new frames. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image processing apparatus that removes impulsive noise from an image captured by an image sensor.

  In recent years, a camera using an image sensor includes an image processing device that removes noise in order to meet the demand for higher image quality.

  An example of the image processing apparatus is a method of reducing noise by performing an operation such as addition or arithmetic averaging after an image signal is stored in a frame memory, as disclosed in Patent Document 1.

  FIG. 9 is a block diagram illustrating a configuration of an apparatus that performs image processing, for example, disclosed in Patent Document 1. In FIG. This apparatus includes an A / D unit 102 that converts an output of an image sensor into a digital value, an adder 103 that adds an output value of the A / D unit 102 and an output value of a subsequent frame memory 104, and an imaging A frame memory 104 having a storage area for one screen of elements, a divider 105 for dividing the addition data stored in the frame memory by the number of additions, and an image processing circuit 106 for performing image processing of output data are provided. ing.

  This image processing method will be described. First, when an output image is captured in the inspection process of the image sensor, the output signal of the image sensor 101 is output from the A / D unit 102 via a preprocessing circuit such as a double correlation sampling circuit for the purpose of removing reset noise of the image sensor. Is converted into frame data and output.

Next, each output frame data is cumulatively added for a desired number of times, for example, three times by the configuration of 103 adders and 104 frame memories.
Next, the frame data accumulated and added in the 103 adder and the 104 frame memory is divided by the divider 105 by the number of added frames. Data subjected to such an averaging process is sent to the image processing circuit.
Japanese Patent Application No. 62-133573

  However, in the image processing device shown in the prior art, random noise caused by the device structure of the image sensor can be smoothed to a certain level, but when high pixels and high image quality are required, The noise smoothing characteristic is insufficient.

  In other words, the impulsive noise generated by an external factor of the image sensor can be diluted by the image processing apparatus shown in the prior art, but cannot be completely removed.

  In view of the above problems, an object of the present invention is to provide an image processing apparatus and method for removing external impulsive noise.

  In order to achieve the above object, an image processing apparatus according to the present invention includes n frame memories (n is 3 or more) for storing frame data for forming a one-dimensional or two-dimensional image, and an output signal of a solid-state image sensor. Conversion means for converting the pixel data into pixel data, distribution means for distributing the pixel data converted by the conversion means to n frame memories in units of frames, and n pixel data at the same pixel position from the n frame memories Read out means, data processing means for outputting one pixel data by rank processing for removing impulsive noise and random noise using n pieces of pixel data, and pixel data output from the data processing means Storage means for storing the data as pixel data of a new frame.

  According to this configuration, it is possible to completely remove the impulse external noise and the internal random noise, not diluting. In addition, it is possible to prevent pixel defects of the solid-state imaging device itself, or point and line boundary components of the subject from being erroneously removed. This is because pixel defects and boundary components appear continuously over a plurality of frames, but impulsive noise and random noise do not appear over a plurality of frames.

  Here, the data processing means may obtain an intermediate value of the n pixel data as the rank processing.

  According to this configuration, since rank processing is simplified to obtain an intermediate value, an increase in circuit scale and an increase in processing time can be suppressed.

  Here, the data processing unit calculates a mean value of (n−2) pixel data excluding the maximum value and the minimum value and a determination unit that determines a maximum value and a minimum value for the n pixel data. Calculating means, and the storage means may be configured to store the calculated average value as pixel data by rank processing.

  According to this configuration, since the average value of (n−2) pixel data excluding the maximum value and the minimum value is obtained in the rank processing, the pixel defect included in the solid-state imaging device itself, the boundary component of the point or line of the subject Can be removed with higher accuracy without being removed by mistake.

  Here, the calculation means includes an addition means for adding the n pixel data, a subtraction means for subtracting the maximum value and the minimum value from the addition result by the addition means, and a subtraction result as (n−2). Dividing means for dividing by may be provided.

  Here, the data processing means further includes an accumulation frame memory for accumulating frame data, and the addition means is configured to output an output signal constituting n screens from the solid-state imaging device. The pixel data converted by the conversion means and the corresponding pixel data in the accumulation frame memory are added to accumulate the n pixel data in the accumulation frame memory, and the division means The maximum value and the minimum value may be subtracted from the accumulated value read from the calculation frame memory.

  According to this configuration, since the determination of the maximum value and the minimum value and the accumulation are performed in parallel in the calculation of the average value, an increase in time required for rank processing can be suppressed.

  Here, the solid-state image pickup device picks up an image of a stationary subject, and the image processing apparatus further includes a determination unit that determines whether the solid-state image pickup device is good or not based on an image composed of pixel data stored in a storage unit. You may make it prepare.

  According to this configuration, the pixel defect of the solid-state imaging device itself can be found with high accuracy.

  According to the present invention, it is possible to remove impulsive external factor noise and dilute random random noise as an internal factor without erroneously converting pixel defects of the image sensor itself or boundary components of points and lines of the subject. But can be completely removed. In addition, the image sensor can be inspected with high accuracy.

(First embodiment)
Hereinafter, an image processing apparatus according to a first embodiment of the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of the image processing apparatus according to this embodiment.
First, an image processing apparatus according to the first embodiment includes an image sensor 11, an A / D converter (hereinafter referred to as “A / D unit”) 12, a distribution circuit 13, and n frame memories 14-1 to 14. -N, a rank processing circuit 15, a frame memory 16, and an image processing circuit 17.

  The imaging device 11 is a CCD or MOS type line sensor or image sensor that captures a one-dimensional or two-dimensional image.

  The A / D unit 12 converts the analog output signal of the image sensor into a digital value (pixel data).

  The distribution circuit 13 distributes the pixel data of the A / D unit 12 to the frame memory in units of frames.

  Each of the n frame memories 14-1 to 14-n has a storage area for one screen of the image sensor, and temporarily stores frame data including pixel data for one screen. When the individual frame memories are not particularly distinguished, they are simply referred to as the frame memory 14.

  The rank processing circuit 15 reads pixel data at the same pixel position from the plurality of frame memories 14-1 to 14-n, performs rank processing for removing impulsive noise and random noise, and obtains pixel data after noise removal. Output. Here, the n pixel data at the same pixel position are compared, and an intermediate value thereof is output.

  The frame memory 16 stores the intermediate value output from the rank processing circuit 15 as pixel data of a new frame.

  The image processing circuit 17 performs desired image processing on the frame data in the frame memory 16. This image processing is, for example, three-dimensional processing as shown in FIG.

Furthermore, the operation of the image processing apparatus according to the first embodiment of the present invention will be described.
First, a signal output from the image sensor 11 is converted into a digital signal in the A / D unit 12 through a preprocessing circuit such as a double correlation sampling circuit for the purpose of removing reset noise of the image sensor 11.

  Next, the output signal from the A / D unit 12 is distributed and written to the frame memory 14 in accordance with the number of processing frames n designated by the distribution circuit 13 from the image processing apparatus.

  Specifically, the first frame data corresponding to continuous imaging by the imaging device 11 is stored in the frame memory 14-1, the second frame data is stored in the frame memory 14-2, and the nth frame data is stored in the frame memory 14-. distributed to n.

  Specific examples of the frame data before rank processing stored in the frame memory 14 are shown in FIGS. FIG. 2A to FIG. 4A show pixel data values for each (X, Y) coordinate of the frame data. 2B to 4B are diagrams showing three-dimensional graphics in which the value of each pixel data is set to the height in addition to the X coordinate and the Y coordinate as an example of the image processing by the image processing circuit 17. . Here, for convenience of explanation, an image of 5 pixels in the vertical and horizontal directions is shown. Further, the pixel at the center in the example of FIG. 2 and the pixel at the upper right end in the example of FIG. 4 indicate the presence of impulsive noise.

  Next, pixel data at the same coordinate position of the X coordinate and the Y coordinate in the n frame data written in the frame memory 14 is output to the rank processing circuit 15. The rank processing circuit 15 derives an intermediate value of n pixel data having the same X coordinate and Y coordinate. For example, when (X, Y) = (1, 1), the pixel data value in FIG. 2 is “10”, the pixel data in FIG. 3 is “9”, and the pixel data in FIG. 4 is “20”. Since the magnitudes of the data values are in the order of “9”, “10”, “20”, the rank processing circuit 15 outputs “10”.

  The rank processing circuit 15 derives intermediate values for all (X, Y) coordinates as described above. Each derived pixel data is written in the frame memory 16 to constitute frame data.

FIG. 5 shows frame data obtained by rank processing from the frame data before rank processing shown in FIG. 2, FIG. 3, and FIG.
Is the impulsive extrinsic noise seen in the central part of the frame data in FIG. 2 and not seen in the same part in FIGS. 3 and 4 significantly larger than the original data of the image sensor or vice versa? Therefore, by adopting an intermediate value between the same address data between frames, extrinsic or sudden noise can be completely removed, and the output level of the original image sensor can be obtained. It becomes possible.

  As described above, according to the image processing apparatus of the present embodiment, by adopting the intermediate value of the image data of the same coordinate position of n X coordinates and Y coordinates between frames, the extrinsic or Sudden noise can be completely removed.

  In addition, random noise, which is an internal factor of the image sensor, can be smoothed at the same level as in the prior art. Random noise due to internal factors of the image sensor includes dark current shot noise, floating diffusion (hereinafter referred to as “FD”) reset noise, FD amplifier noise, optical shot noise, and the like. Dark current shot noise is a fluctuation of dark current generated in a photodiode and a vertical / horizontal CCD. The FD reset noise is a fluctuation of the reference potential after the FD is reset. The FD amplifier noise is thermal noise of the amplifier circuit of the FD amplifier. Light shot noise is fluctuation of the number of photons per unit time in incident light.

  Furthermore, it is possible to prevent pixel defects of the solid-state imaging device itself, or point and line boundary components of the subject from being erroneously removed. This is because these pixel defects and boundary components appear continuously over a plurality of frames (for example, images as shown in FIG. 2 continue), but impulsive noise and random noise do not appear over a plurality of frames (for example, 2 to 4).

  The intermediate value may be a value of pixel data located at the center of the arrangement when n pieces of pixel data are arranged in order of size. Further, when there are two pieces of pixel data located at the center (when n is an even number), one of the two may be an intermediate value, and the average of the two may be an intermediate value.

(Second Embodiment)
The image processing apparatus according to the second embodiment of the present invention will be described below with reference to the drawings.

  FIG. 6 is a block diagram showing the configuration of the image processing apparatus according to this embodiment. Compared with FIG. 1, the image processing apparatus of the figure includes a rank processing circuit 25 instead of the rank processing circuit 15, and an addition of an adder 18, a frame memory 19, a subtracter 20, and a divider 21. Is different. Hereinafter, description of the same points will be omitted, and different points will be mainly described.

  The rank processing circuit 25 outputs pixel data of the maximum value and pixel data of the minimum value among n pixel data at the same pixel position from the n frame memories.

  The adder 18 adds n pieces of pixel data at the same pixel position. Here, the adder 18 adds the pixel data output from the A / D unit 12 and the corresponding pixel data in the frame memory 19, thereby accumulating the n pieces of pixel data in the accumulation frame memory. Calculate.

  The frame memory 19 stores frame data obtained by accumulating pixel data at the same pixel position in n pieces of frame data.

  The subtracter 20 subtracts the maximum value and the minimum value of the same pixel position output from the rank processing circuit 25 from the pixel data (accumulated value of n pieces of pixel data) in the frame memory 19.

  The divider 21 divides the subtraction result by (n−2). The division result is stored in the frame memory 16 as pixel data.

Further, a noise removal method performed by the image processing apparatus shown in FIG. 6 will be described.
First, a signal output from the image sensor 11 is converted into a digital signal in the A / D unit 12 through a preprocessing circuit such as a double correlation sampling circuit for the purpose of removing reset noise of the image sensor.

  Next, the output signal from the A / D unit 12 is distributed and written to the frame memory 14 in accordance with the number of processing frames n designated by the distribution circuit 13 from the image processing apparatus.

  Specifically, the first frame data is the frame memory 14-1, the second frame data is the frame memory 14-2, and the nth frame data is the frame memory 14-n.

  In the present embodiment, frame data for one screen each as shown in FIGS. 2 to 4 and 7 is written in the frame memory 14.

  Next, n pieces of image data at the same coordinate position in the n pieces of frame data written in the frame memory 14 are read out to the rank processing circuit 25.

  The rank processing circuit 25 derives the maximum value and the minimum value of image data at the same coordinate position of n X coordinates and Y coordinates. For example, when (X, Y) = (1, 1), the output data in FIG. 2 is “10”, the output data in FIG. 3 is “9”, the output data in FIG. 4 is “20”, and the output in FIG. Since the data is “8” and the image data values are in the order of “8”, “9”, “10”, “20”, the rank processing circuit 25 outputs “8” and “20”. Is done.

  Next, the rank processing circuit 25 repeatedly derives the maximum value and the minimum value from the image data at the same coordinate position in n frame data for all (X, Y coordinates).

Next, the derived image data is passed to the subtracter 20.
On the other hand, the output signal from the A / D unit 12 is configured as frame data, and data addition for the desired number of frames is cumulatively performed for each frame data by the configuration of the adder 18 and the frame memory 19 ( In this description, four times as an example).

  At this time, the data addition is performed on the image data at the same coordinate position of the X coordinate and the Y coordinate stored in the frame memory, and the addition result is output to the subtracter 20 at the subsequent stage.

  Next, the subtracter 20 subtracts data having the same address between the frame data cumulatively added in the adder 18 and the frame memory 19 and the maximum and minimum frame data output from the rank processing circuit 25. I do. For example, when (X, Y) = (1, 1), the output data before rank processing shown in FIGS. 2 to 4 and 7 is “10”, “9”, “20”, and “8”. The minimum value “8” and the maximum value “20” are output from the rank processing circuit 25, and the data output from the subtracter 20 is “19”. Further, it is divided by a value obtained by subtracting 2 from the number of added frames by the divider 21 (in this description, twice as an example). For example, in the case of (X, Y) = (1, 1), the data output from the subtracter 20 is “19”, so the divider 21 is ((10 + 9) / (4-2)) = 9. 5) is output. As a result of outputting the division result for all the pixels by the divider 21, frame data as shown in FIG. 8 is formed.

  As described above, noise removal according to the invention of the present embodiment is performed, the maximum value and the minimum value of the same address data between frames are removed, and the average value of the remaining data is adopted, thereby improving the impulsiveness. Noise can be removed. In addition, smoothing of random noise can be performed at least as much as the conventional method.

  Furthermore, according to the configuration of FIG. 6, since the determination of the maximum value and the minimum value and the accumulation are performed in parallel, an increase in time required for rank processing can be suppressed.

(Comparative example)
Hereinafter, an image processing apparatus and a noise removal method according to the related art will be described as comparative examples with reference to the drawings. The configuration of the image processing apparatus in the prior art is as shown in FIG.

  Image processing in the prior art will be described on the premise of the frame data shown in FIGS. In the frame memory 104 of FIG. 9, the frame data shown in FIGS. 2 to 4 are cumulatively added for each pixel at the same pixel position, and divided by the number of added frames (three times as an example in this description) by the divider 105. The As a result, the frame data subjected to addition averaging as shown in FIG. 10 is formed.

  The noise component of the central portion existing in FIG. 2 is diluted in FIG. 10, but as shown in the central portion of FIG. 10, complete noise removal cannot be performed.

  On the other hand, in FIG. 5 and FIG. 8 after image processing in the first and second embodiments, the central noise component existing in FIG. 2 can be completely removed.

  Thus, the impulsive extrinsic noise seen in the central portion of the frame data at the time of capture in FIG. 2 and not seen in the same location in FIGS. 3 and 4 is significantly higher than the original data of the image sensor. Since the image processing apparatus according to the first embodiment employs an intermediate value between the same pixel positions of consecutive frames as shown in FIG. Or sudden noise can be completely removed, and the output level of the original image sensor can be obtained. In the image processing apparatus according to the second embodiment, by excluding the maximum value and the minimum value between the same pixel positions in the continuous frame and then obtaining the average value, as shown in FIG. 8, extrinsic or sudden noise Can be completely removed, and the output level of the original image sensor can be obtained.

  In the image processing apparatus shown in FIG. 6, after adding n pieces of pixel data, the maximum value and the minimum value are subtracted from the addition result. However, the maximum value and the minimum value are subtracted from the n pieces of pixel data. After exclusion, (n-2) pixel data may be added from the exclusion result.

  Note that the image processing apparatus shown in FIG. 6 includes a register for accumulating pixel data instead of the frame memory 19, and reads out pixel data at the same pixel position from the frame memories 14-1 to 14-n and accumulates it in the register. It is good also as a structure to calculate. In this way, a delay time due to accumulation occurs and the processing time becomes long, but the circuit scale can be reduced.

  The image processing apparatus shown in FIGS. 1 and 6 may be used as an imaging device inspection apparatus. In this case, the following configuration may be used. The imaging device to be inspected continuously shoots a stationary subject (captures n images). The frame memory 16 stores an image of the same subject (hereinafter referred to as reference frame data) captured in advance by another normal image sensor. The image processing circuit 17 determines whether the image sensor is normal (normal or abnormal). That is, the image processing circuit 17 compares the frame data after rank processing obtained from the imaging device 11 to be inspected with the reference frame data, and whether the difference between the pixel data at the same pixel position is equal to or less than the threshold value. Determine whether or not. The image processing circuit 17 performs this determination for all the pixels, and when the number of pixels exceeding the threshold value exceeds a predetermined number (for example, 1% of all the pixels), the image sensor 11 is determined to be abnormal, If it is less than the number, it is judged as normal.

  Instead of realizing each function of the block diagrams shown in FIGS. 1 and 5 by hardware, a program describing all or part of the functions of each block is realized by a program in a DSP or a microprocessor. Also good.

  The image processing apparatus according to the present invention eliminates impulsive external factor noise and internal random noise without erroneously converting pixel defects of the image sensor itself or boundary points of points and lines of the subject. Can be smoothed, the accuracy of the image data at the stage of capturing the output image of the image sensor can be increased, and it is useful as an image processing apparatus used for an image pickup apparatus that requires high image quality and an inspection apparatus for an image sensor. is there.

1 is a block diagram illustrating a configuration of an imaging apparatus according to a first embodiment of the present invention. (A) (b) It is a figure which shows the image data before a rank process. (A) (b) It is a figure which shows the image data before a rank process. (A) (b) It is a figure which shows the image data before a rank process. (A) (b) It is a figure which shows the image data after a rank process. It is a block diagram which shows the structure of the imaging device which concerns on the 2nd Embodiment of this invention. (A) (b) It is a figure which shows the image data before a rank process. (A) (b) It is a figure which shows the image data after a rank process. It is a block diagram which shows schematic structure of the image processing apparatus in a prior art. (A) (b) It is a figure which shows the image data after the image processing in a prior art.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 Image sensor 12 A / D part 13 Distribution circuit 14 Frame memory 15 Rank processing circuit 16 Frame memory 17 Image processing circuit 18 Adder 19 Addition frame memory 20 Subtractor 21 Divider 25 Rank processing circuit

Claims (13)

N frame memories (n is 3 or more) for storing frame data forming a one-dimensional or two-dimensional image;
Conversion means for converting the output signal of the solid-state image sensor into pixel data;
Distribution means for distributing pixel data converted by the conversion means to n frame memories in units of frames;
reading means for reading out n pieces of pixel data at the same pixel position from n pieces of frame memories;
data processing means for outputting one pixel data by rank processing for removing impulsive noise and random noise using n pieces of pixel data;
An image processing apparatus comprising: storage means for storing pixel data output from the data processing means as pixel data of a new frame. The image processing apparatus according to claim 1, wherein the data processing unit obtains an intermediate value of the n pieces of pixel data as the rank processing. The data processing means includes
Determination means for determining a maximum value and a minimum value for the n pieces of pixel data;
A calculating means for calculating an average value of (n−2) pixel data excluding the maximum value and the minimum value, and
The image processing apparatus according to claim 1, wherein the storage unit stores the calculated average value as pixel data by rank processing. The calculating means includes an adding means for adding the n pieces of pixel data;
Subtracting means for subtracting the maximum value and the minimum value from the addition result by the adding means;
The image processing apparatus according to claim 3, further comprising a dividing unit that divides the subtraction result by (n−2). The data processing means further includes an accumulation frame memory for accumulating frame data,
The adding means adds the pixel data converted by the converting means and the corresponding pixel data in the accumulated frame memory while output signals constituting n screens are output from the solid-state imaging device. , Accumulating the n pixel data in an accumulating frame memory,
The image processing apparatus according to claim 4, wherein the dividing unit subtracts the maximum value and the minimum value from an accumulated value read from an accumulated frame memory. The solid-state imaging device images a stationary subject,
The said image processing apparatus is further provided with the determination means which determines the quality of the said solid-state image sensor based on the image which consists of pixel data memorize | stored in the memory | storage means. Image processing apparatus. The nth frame data obtained from the solid-state imaging device and the A / D converter is stored in n (n is 3 or more) frame memories for storing frame data for forming a one-dimensional or two-dimensional image. One storage step;
a reading step of reading out n pixel data at the same pixel position from n frame memories;
a data processing step of outputting one pixel data by rank processing for removing impulsive noise and random noise using n pieces of pixel data;
An image processing method comprising: a second storage step of storing the pixel data obtained in the data processing step in a frame buffer as pixel data of a new frame. The image processing method according to claim 7, wherein in the data processing step, an intermediate value of the n pieces of pixel data is obtained as the rank processing. The data processing step includes
A sub-step of determining a maximum value and a minimum value for the n pieces of pixel data;
A sub-step of calculating an average value of (n−2) pixel data excluding the maximum value and the minimum value;
Have
The image processing method according to claim 7, wherein in the second storing step, the average value is stored in a frame buffer as pixel data by rank processing. In the sub-step of calculating the average value, the n pieces of pixel data are added, the maximum value and the minimum value are subtracted from the addition result, and the subtraction result is divided by (n−2). The image processing method according to claim 9. In the addition of the n pixel data,
Using the accumulated frame memory for accumulating the frame data, while the output signals constituting the n screens are output from the solid-state imaging device, the pixel data and the corresponding pixel data in the accumulated frame memory The image processing method according to claim 10, wherein the n pieces of pixel data are accumulated in an accumulated frame memory by adding. In the first storing step, the solid-state imaging device images a stationary subject,
12. The image processing method according to claim 7, further comprising a determination step of determining whether the solid-state imaging device is good or not based on an image made up of pixel data stored in a frame buffer. Image processing method.   A computer-readable program that causes a computer to execute the image processing method according to claim 7.

Images