JP2006339986A - Image processing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing apparatus capable of detecting a white-dot defect and a black-dot defect of pixels of an image sensor without any error while individual nonvolatile memories are not provided. <P>SOLUTION: The image processing apparatus includes a decision section which decides whether a pixel corresponding to pixel data is a defective pixel when the pixel data is output from the image sensor, a storage section which stores the pixel address of a pixel decided as a defective pixel by the decision section and decision count the decision section decides the pixel as the defective pixel, a determination section which determines a pixel whose decision count exceeds a specified value as a defective pixel, and a complementation section which complements pixel data corresponding to the pixel determined as the defective pixel by the determination section by a specified method and then outputs the complemented pixel data. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image processing apparatus used for a digital camera or the like.

  At present, charge coupled devices (hereinafter referred to as “CCD”) and complementary metal oxide semiconductor sensors (hereinafter referred to as “CMOS sensors”) are typical image sensors of digital video cameras and digital still cameras. . In such an image sensor, the brightness of a pixel at a specific position may be always fixed at a constant brightness. Such a fixed brightness of the pixel always being high or low is called a white spot defect and a black spot defect, respectively.

  In general, in order to display an image captured by an image sensor neatly on a display or the like with a VGA size of 640 × 480 pixels, a 300,000 pixel class image sensor is required. However, at present, it is difficult to correctly manufacture all 300,000 pixels, and therefore it is necessary to allow a certain amount of white spot failure and black spot failure. Actually, even when there are white point defective pixels and black point defective pixels (hereinafter, simply referred to as “defective pixels”), the image sensor is defective when the number of defective pixels is equal to or less than a predetermined number. It is sold at a price according to the grade determined by the number of items. For example, an image sensor having a total of more than 30 defective pixels is handled as a defective product, but an image sensor having a total of more than 10 defective pixels and not more than 30 is considered as a low-grade image sensor. Image sensors with a total of 10 or less defective pixels are sold as high-grade image sensors.

  As described above, it is inevitable that white spot defects and black spot defects occur in the image sensor. Therefore, an image captured by the image sensor is transmitted to the host computer via, for example, the USB bus, and the host computer In the case of displaying on the display, there is a problem that a dark point is emphasized and displayed in a bright image, or a brighter point is emphasized and displayed in a dark image.

  As a general method for solving such a problem, the position of a defective pixel in an image sensor is detected in an inspection process before shipment, and the address data of the detected defective pixel is stored in an external nonvolatile memory or the like. Later, when pixel data of a pixel corresponding to the stored address data was output from the image sensor, there was a method of complementing the pixel data.

  In addition, in a conventional image sensor, pixel data is read out from a pixel unit in which pixels are arranged in a matrix, and the difference in brightness from surrounding pixels is calculated with respect to the read pixel data. Some have detected a white point defect without connecting a non-volatile memory and corrected the read pixel data (see, for example, Patent Document 1).

Note that the conventional image reading apparatus detects a peak value by a digital peak hold circuit in which tracking characteristics from high density to low density and low density to high density are set, and a divider based on the detected value. There is one that normalizes input image data and outputs background data with a constant white level value (see, for example, Patent Document 2).
JP 2003-234958 A JP 2003-101777 A

  However, in the conventional method of storing the defective pixel address data in the external nonvolatile memory, it is necessary to store the position of the defective pixel detected in the inspection process before shipment for each image sensor in the nonvolatile memory. There is a problem that processing before shipment becomes complicated, and cost is increased because a non-volatile memory is separately required.

  On the other hand, the conventional image sensor that calculates the difference in brightness from the surrounding pixels with respect to the read pixel data does not require an individual non-volatile memory, but actually has a very thin line or brightness with bright brightness. When there were bright and very small points, they were sometimes mistaken for white spot defects and complemented. If such a misrecognition is made, a very thin line or point is complemented with surrounding pixels and disappears, resulting in an unnatural image. This problem may not be a problem because the area of one pixel is very small in a high-resolution image sensor, but is particularly noticeable in an image sensor with 300,000 pixels or less.

  SUMMARY OF THE INVENTION The present invention has been made to solve the above-described problems, and has no individual nonvolatile memory, but can detect a white point defect and a black point defect of a pixel in an image sensor without error. The purpose is to provide.

  An image processing apparatus according to the present invention is an image processing apparatus that complements and outputs pixel data corresponding to defective pixels of an image sensor by a predetermined method. In the image processing apparatus, when pixel data is output from the image sensor, a determination unit that determines whether a pixel corresponding to the pixel data is a defective pixel, and the determination unit determines that the pixel is a defective pixel. A storage unit that stores a pixel address of a pixel determined to be and a determination number of times that the pixel is determined to be a defective pixel by the determination unit; and a pixel that has the predetermined number of determinations exceeding a predetermined number A determining unit that determines the pixel; and a complementing unit that complements and outputs pixel data corresponding to the pixel determined to be a defective pixel by the determining unit by a predetermined method.

  Preferably, the storage unit includes a data storage unit that stores the pixel address and the number of determinations, and a storage control unit that controls reading and writing of data with respect to the data storage unit. The storage control unit determines that the pixel corresponding to the pixel data output from the image sensor is a defective pixel by the determination unit, and stores the pixel address of the pixel and the number of determinations in the data storage unit Is stored, the pixel address corresponding to the pixel and the number of determinations are written in the data storage unit.

  Preferably, the storage control unit determines that the pixel corresponding to the pixel data output from the image sensor is a defective pixel by the determination unit, and stores the pixel address of the pixel in the data storage unit. When the determination number is stored, the determination number of the pixel stored in the data storage unit is increased by one.

  Preferably, the data storage unit is a FIFO memory.

  Preferably, the predetermined number of times can be set from the outside.

  Preferably, the determination unit has a predetermined relationship between the brightness of the pixel data of the pixel to be determined and the sum of the brightness of the input and output pixel data input before and after the input of the pixel data. Sometimes, it is determined that the pixel to be determined is a defective pixel.

  Preferably, the front and rear input pixel data is pixel data having the same color component as the pixel data of the pixel to be determined.

  Preferably, the determination unit includes a first multiplied pixel data obtained by multiplying a brightness of pixel data of the pixel to be determined by a predetermined first coefficient by a sum of the brightness of the preceding and following input pixel data. Or the brightness of the pixel data of the pixel to be determined is smaller than the second multiplied pixel data obtained by multiplying the sum of the brightness of the preceding and following input pixel data by a predetermined second coefficient. In this case, it is determined that the pixel to be determined is a defective pixel.

  Preferably, each of the first and second coefficients can be set from the outside.

  According to the image processing device of the present invention, when pixel data is output from the image sensor, a determination unit that determines whether or not a pixel corresponding to the pixel data is a defective pixel, and the determination unit determines that the pixel is a defective pixel. A storage unit that stores a pixel address of the pixel determined as a pixel and a determination number of times that the pixel is determined to be a defective pixel by the determination unit, and a pixel that has exceeded the predetermined number of determinations is determined as a defective pixel Since it includes a determination unit and a complement unit that complements and outputs pixel data corresponding to a pixel determined to be a defective pixel by the determination unit using a predetermined method, an image can be obtained without having an individual nonvolatile memory. It is possible to detect a white spot defect and a black spot defect of a pixel in the sensor without error.

Embodiments of the present invention will be described below with reference to the accompanying drawings.
(Embodiment 1)
FIG. 1 is a block diagram showing a configuration example of an image processing apparatus according to the present invention. An image processing apparatus 1 according to the present invention is used when an image captured by an image sensor such as a CCD is transmitted to a host computer 2 via a USB bus. The image processing apparatus 1 determines whether or not the pixel corresponding to the pixel data is a defective pixel with respect to the pixel data input from the image sensor, and determines that the pixel data is a defective pixel. The data is complemented by a predetermined method and output to the host computer 2. As shown in FIG. 1, the image processing apparatus 1 includes an image sensor unit 3, a white point / black point correction device 4, and a USB interface unit 5 (hereinafter referred to as “USB I / F unit 5”). The white point / black point correction device 4 includes a white point / black point storage unit 6 and a white point / black point detection complement unit 7, and the white point / black point detection complement unit 7 includes a pixel address generation unit 8. The image sensor unit 3, white point / black point correction device 4, and USB I / F unit 5, which are components of the image processing device 1, are synchronized with the timing signal TM input from the external timing signal generation unit 9. Each works.

  The image sensor unit 3 includes an image sensor (not shown) composed of a solid-state imaging device such as a CCD or a CMOS sensor. The image sensor includes a pixel array composed of a plurality of pixels arranged in the vertical direction and the horizontal direction, and converts the amount of light irradiated to each pixel into an electrical signal and outputs it. An analog signal corresponding to each pixel output from the image sensor is converted into a digital signal by an A / D converter (not shown) and output. That is, the image sensor unit 3 outputs pixel data Din corresponding to each pixel of the image sensor to the white point / black point detection complementing unit 7, respectively.

  The image sensor unit 3 outputs a vertical blanking interval signal VBlank and a horizontal blanking interval signal HBlank, respectively. FIG. 2 is a timing chart for explaining the vertical blanking period signal VBlank and the horizontal blanking period signal HBlank. These blanking period signals VBlank and HBlank are each active at a Low (L) level. The vertical blanking period signal VBlank is at the L level during the period when valid pixel data is output from the image sensor unit 3. For example, when the number of effective pixels of the image sensor is 640 × 480, the vertical blanking period signal VBlank is at L level while pixel data corresponding to 640 × 480 pixels are being output. The pixel data is generally output for each horizontal line from the top horizontal line to the bottom horizontal line in the pixel array of the image sensor, and for each horizontal line, each pixel from the left end pixel to the right end pixel is output to each pixel. Corresponding pixel data is output in order. Therefore, the vertical blanking period signal VBlank is output from the pixel data of the upper left pixel of the pixel area (hereinafter referred to as “effective pixel area”) actually used for imaging in the image sensor, and then the lower right pixel. It becomes L level during the period until the pixel data is output. On the other hand, the horizontal blanking period signal HBlank becomes L level during a period T in which pixel data of each pixel is output from the leftmost pixel to the rightmost pixel in each horizontal line in the effective pixel region of the image sensor. The pixel address generation unit 8 detects the pixel address of the pixel corresponding to the pixel data Din from the vertical blanking period signal VBlank and the horizontal blanking period signal HBlank output from the image sensor unit 3.

  Further, the vertical blanking period signal VBlank and the horizontal blanking period signal HBlank are also input to the USB I / F unit 5, respectively. This is because the pixel data Din output from the image sensor unit 3 includes pixel data corresponding to pixels other than the effective pixel region of the image sensor, and the USB I / F unit 5 identifies these pixel data. This is to prevent output to the host computer 2. The USB I / F unit 5 converts the pixel data Dout output from the white point / black point detection complement unit 7 into pixel data Dusb so that it can be transmitted via the USB bus, and outputs it to the host computer 2.

  When the pixel data Din is input from the image sensor unit 3, the white point / black point detection complementing unit 7 is a white point defective pixel, a black point defective pixel, or a normal pixel corresponding to the pixel data Din. The determination data Det indicating the determination result is output to the white / black point storage unit 6. In addition, the pixel address generation unit 8 in the white spot / black spot detection complement unit 7 uses the vertical blanking period signal VBlank and the horizontal blanking period signal HBlank input from the image sensor unit 3 to input pixel data Din. The pixel address of the pixel corresponding to is detected, pixel address data Ad indicating the pixel address is generated, and output to the white point / black point storage unit 6. The pixel address data Ad is expressed as (Vad, Had) using, for example, a vertical address Vad and a horizontal address Had in the pixel array of the image sensor. For example, (033, 045), (124, 048) and the like. The white point / black point storage unit 6 uses the pixel address data Ad and the determination data Det output from the white point / black point detection complement unit 7 to perform the pixel data Din of the pixel specified by the pixel address data Ad. It is determined whether or not complementation is necessary, and a complement request signal Req indicating the determination result is output to the white / black spot detection and complement unit 7. The white point / black point detection complementing unit 7 complements the pixel data Din and outputs the pixel data Dout to the USB I / F unit 5 when the white point / black point storage unit 6 determines that complementation is necessary. To do. On the other hand, when the white point / black point storage unit 6 determines that complementation is not necessary, the pixel data Din input from the image sensor unit 3 as the pixel data Dout is output to the USB I / F unit 5 as it is.

  The image sensor of the image sensor unit 3 includes an R pixel that detects only a red component of light, a B pixel that detects only a blue component, and a G pixel that detects only a green component. FIG. 3 is a diagram illustrating an example of a pixel array of the image sensor. The pixel array shown in FIG. 3 is a Bayer array, and this array is currently used in most image sensors. As shown in FIG. 3, in the Bayer array, a first horizontal line in which R pixels and G pixels are alternately arranged in the horizontal direction and a second line in which G pixels and B pixels are alternately arranged in the horizontal direction are used. There are horizontal lines, and the first and second horizontal lines are alternately arranged in the vertical direction.

  FIG. 4 is a block diagram illustrating a configuration example of the white / black spot detection / complementing unit 7. As shown in FIG. 4, the white point / black point detection complement unit 7 includes a pixel address generation unit 8, a white point / black point determination unit 11, five pixel data storage units 12 to 16, a pixel complement unit 17, and a multiplexer ( (Hereinafter referred to as “MUX”) 18. In addition, the white spot / black spot determination unit 11 includes a coefficient storage register 19. The pixel data Din input from the image sensor unit 3 is first stored in the pixel data storage unit 12, and then each time the pixel data Din is input from the image sensor unit 3, the pixel data Din is stored in each of the pixel data storage units 13 to 16. Shifted in order. Hereinafter, in order to simplify the description, the pixel data stored in each of the pixel data storage units 12 to 16 is referred to as pixel data d12 to d16, respectively. For example, when the pixel array of the image sensor is a Bayer array as shown in FIG. 2, the image sensor unit 3 has pixel data Din indicating the brightness of the red component and pixel data Din indicating the brightness of the green component in a certain horizontal line. Are alternately output, and then pixel data Din indicating the brightness of the green component and pixel data Din indicating the brightness of the blue component are alternately output in the horizontal line below. That is, the image sensor unit 3 outputs every other pixel data Din indicating the brightness of the same color component. Therefore, the pixel data d12, d14, and d16 stored in the pixel data storage units 12, 14, and 16 are pixel data indicating the brightness of the same color component, and the pixel data d13 that is stored in the pixel data storage units 13 and 15, respectively. , D15 are pixel data indicating the brightness of the same color component.

In the white / black point detection / complementing unit 7, the pixel data d 12, d 14, and d 16 are input to the white / black point determining unit 11, respectively. The white point / black point determination unit 11 uses the pixel data d14 and the pixel data d12 and d16 indicating the brightness of the same color component adjacent to each other, and the pixels corresponding to the pixel data d14 are white point defective pixels, black point defective pixels, And a normal pixel that is not defective is determined. Specifically, the white / black point determination unit 11 determines whether or not the following expressions (1) and (2) are satisfied for the pixel data d14. D12, D14, and D16 are the brightness values of the pixels indicated by the pixel data d12, d14, and d16, respectively.
D14> (D12 + D16) × k1 (1)
D14 <(D12 + D16) × k2 (2)
Here, the coefficients k1 and k2 are predetermined constants stored in the coefficient storage register 19, respectively.

  The white point / black point determination unit 11 determines that the pixel corresponding to the pixel data d14 is a white point defective pixel when the expression (1) is satisfied, and the pixel data d14 when the expression (2) is satisfied. The pixel corresponding to is determined as a black spot defective pixel, and when both of the expressions (1) and (2) are not satisfied, it is determined that the pixel corresponding to the pixel data d14 is a normal pixel. Then, the white point / black point determination unit 11 outputs determination data Det indicating the determination result to the white point / black point storage unit 6.

  Further, the pixel address generation unit 8 outputs address data Ad indicating the address of the pixel corresponding to the pixel data d14 at the same timing as the timing at which the determination data Det is output.

  If the coefficient k1 for detecting a white spot defect and the coefficient k2 for detecting a black spot defect are set to a very small value, a normal pixel is erroneously determined to be a defective pixel and set to a very high value. On the contrary, there is a possibility that the defective pixel is erroneously determined as a normal pixel. Therefore, the values of the coefficients k1 and k2 may be settable from the outside so that they can be changed according to the image captured by the image sensor unit 3. For example, when a dark image is captured, the coefficient k1 is decreased and the coefficient k2 is increased. When a bright image is captured, the coefficient k1 is increased and the coefficient k2 is increased. May be reduced.

On the other hand, pixel data d12 and d16 are input to the pixel complementing unit 17, respectively. The pixel complementing unit 17 generates complemented pixel data dd14 that complements the pixel data d14 using the input pixel data d12 and d16, and outputs the complemented pixel data dd14 to the MUX 18, as shown in the following formula (3).
DD14 = (D12 + D16) / 2 (3)
Here, DD14 is the brightness of the pixel indicated by the complementary pixel data dd14.

  The MUX 18 receives pixel data d14, complementary pixel data dd14, and a complementary request signal Req. Here, the complement request signal Req is a signal indicating the determination result of whether or not the white spot / black spot storage unit 6 needs to complement the pixel data d14. For example, when the white point / black point storage unit 6 determines that complementation is necessary for the pixel data d14, the H level complement request signal Req is output, and when it is determined that complementation is not necessary, L level complementation is performed. When the request signal Req is output, the MUX 18 outputs the complementary pixel data dd14 as the pixel data Dout when the H level complement request signal Req is input, and when the L level complement request signal Req is input, Pixel data d14 is output as data Dout.

  FIG. 5 is a block diagram illustrating a configuration example of the white spot / black spot storage unit 6. As shown in FIG. 5, the white point / black point storage unit 6 includes a determination control unit 21, a first-in first-out memory (hereinafter referred to as “FIFO memory”) 22, and a determination comparison unit 23. Further, the determination comparison unit 23 includes a number storage register 24. The determination control unit 21 reads and writes data from and to the FIFO memory 22. The determination control unit 21 receives from the white / black point detection complementing unit 7 the pixel address data Ad corresponding to the pixel data d14 and the pixel corresponding to the pixel data d14, which is either a white point defective pixel, a black point defective pixel, or a normal pixel. The determination data Det indicating whether or not is input. The operation of the determination control unit 21 will be described in detail later.

  The FIFO memory 22 stores defective pixel data. FIG. 6 shows an example of the data structure of data stored in the FIFO memory 22. In this data structure, pixel addresses of pixels determined to be defective pixels by the white / black point determination unit 11 are arranged in order from the smallest. Further, as shown in FIG. 6, in this data structure, white point / black point data indicating whether a pixel corresponding to the pixel address is a black point defective pixel or a white point defective pixel, and the pixel is a defective pixel. Is stored in association with each pixel address. The pixel address data is composed of 10-bit horizontal address data and 9-bit vertical address data when the image sensor of the image sensor unit 3 is, for example, a 300,000-pixel VGA class image sensor. The white / black spot data is composed of 1-bit data. The capacity of the FIFO memory 22 is determined by the number of defective pixels assumed for the image sensor of the image sensor unit 3. For example, it may be determined according to the grade of the image sensor determined by the number of defective pixels. In the following description, the pixel address stored in the FIFO memory 22 is referred to as “FIFO pixel address” for the sake of simplicity.

  The determination comparison unit 23 reads data DM related to the pixel address indicated by the input pixel address data Ad (hereinafter referred to as “current pixel address”) from the FIFO memory 22. Then, by comparing the determination number m included in the data DM with the predetermined number s stored in the number storage register 24, it is determined whether or not the pixel corresponding to the current pixel address is a defective pixel. to decide. Here, m and s are each an integer of 0 or more. Specifically, when s <m holds, it is determined that the pixel corresponding to the current pixel address is a defective pixel, and it is determined that complementation is necessary. If s ≧ m holds, it is determined that the pixel corresponding to the current pixel address is a normal pixel, and it is determined that complementation is unnecessary. It is desirable that the predetermined number of times s can be changed according to the image captured by the image sensor unit 3. The judgment comparison unit 23 outputs a complement request signal Req according to the judgment result.

  Next, the operation of the determination control unit 21 will be described in detail. FIG. 7 is a flowchart for explaining the operation of the determination control unit 21. The determination control unit 21 first resets the FIFO memory 22 (step S1). Specifically, the determination control unit 21 sets all address data in the FIFO memory 22 to (maxX, maxY), and sets all determination count data to “0h”, that is, “0”. Here, (maxX, maxY) is the maximum value of the pixel address of the pixel in the effective pixel region of the image sensor. For example, when the number of effective pixels of the image sensor is 640 × 480, (maxX, maxY) is (640, 480). Next, the determination control unit 21 sets a read pointer for designating an address for read access in the FIFO memory 22 to the first address in the FIFO memory 22 (step S2).

  Next, the determination control unit 21 performs different processing depending on whether or not data has been written to the FIFO memory 22 after the FIFO memory 22 is once reset. Therefore, the determination control unit 21 may store the number of writes after the FIFO memory 22 is once reset in an internal register. If the number of times of writing is 0 (YES in step S3), the writing process shown in FIG. 8 is performed.

  FIG. 8 is a flowchart for explaining the operation of the determination control unit 21 when data is first written to the FIFO memory 22 once reset. As shown in FIG. 8, the determination control unit 21 first acquires the current pixel address (Vad, Had) from the input pixel address data Ad (step S21). Then, the vertical address Vad and maxY of the current pixel address are compared (step S22). Here, “maxY” is the maximum vertical address in the effective pixel region. If vertical address Vad of the current pixel address is larger than maxY (YES in step S22), the process returns to step S2. On the other hand, if the vertical address Vad is equal to or less than maxY (NO in step S22), it is determined whether or not the pixel corresponding to the current pixel address is a defective pixel using the determination data Det (step S23). If the pixel corresponding to the current pixel address is not a defective pixel (NO in step S23), the process returns to step S21, and the pixel address indicated by the next input pixel address data Ad is acquired.

  On the other hand, if the pixel corresponding to the current pixel address is a defective pixel (YES in step S23), new defective pixel data is written in the FIFO memory 22. Specifically, the data stored in each address of the FIFO memory 22 is shifted to the next address after that address (step S24). That is, the address currently designated by the read pointer becomes empty. Then, the current pixel address, white / black point data, and the number of determinations are written in the address designated by the read pointer (step S25). Here, the number of determinations is written as “1h”. Thereafter, the read pointer is advanced to the address next to the FIFO pixel address in which the data is written (step S26), and the process returns to step S21. Each time pixel address data Ad indicating the next pixel address is input, steps S21 to S26 are repeated, and when the vertical address Vad of the current pixel address is greater than maxY, the process returns to step S2.

  Referring to FIG. 7 again, if the number of times of writing is not 0 in step S3 (NO in step S3), the current pixel address (Vad, Had) is acquired from the input pixel address data Ad (step S4). Then, the vertical address Vad and maxY of the current pixel address are compared (step S5). If vertical address Vad of the current pixel address is larger than maxY (YES in step S5), the process returns to step S2. On the other hand, if the vertical direction address Vad is equal to or less than maxY (NO in step S5), the data DM at the address specified by the read pointer is read (step S6). Next, the FIFO pixel address included in the read data is compared with the current pixel address (step S7). If the current pixel address is smaller than the FIFO pixel address, it is determined whether or not the pixel corresponding to the current pixel address is a defective pixel using the determination data Det (step S8). (NO in step S8), the process returns to step S4, and the pixel address indicated by the pixel address data Ad input next is acquired. If the pixel corresponding to the current pixel address is a defective pixel (YES in step S8), it is determined whether or not the FIFO memory 22 is in a FULL state (step S9). For this determination, for example, the write count data stored in the internal register may be used. If the write count data is smaller than the number of data that can be written to the FIFO memory 22, it is determined that the FIFO memory 22 is still writable. This number of writable data may be stored in advance in another internal register.

If it is determined in step S9 that the FIFO memory 22 is not in the FULL state (NO in step S9), new defective pixel data is written in the FIFO memory 22.
Specifically, the data stored in each address after the address specified by the current read pointer in the FIFO memory 22 is shifted to the next address after that address (step S10). That is, the address currently designated by the read pointer becomes empty. Then, the current pixel address, white / black point data, and the number of determinations (value 1h) are written in the address designated by the read pointer (step S11). Thereafter, the read pointer is advanced to the address next to the FIFO pixel address in which the data is written (step S12), and the process returns to step S4. On the other hand, if it is determined that FIFO memory 22 is in the FULL state (YES in step S9), the read pointer is advanced to the next address in FIFO memory 22 (step S13), and the process returns to step S4.

  When the FIFO pixel address and the current pixel address match as a result of the comparison between the FIFO pixel address and the current pixel address in step S7 (“=” in step S7), the determination control unit 21 uses the determination data Det. It is determined whether or not the pixel corresponding to the pixel address is a defective pixel (step S14). If it is not a defective pixel (NO in step S14), the read pointer is advanced to the next address (step S15), and the process returns to step S4. If the pixel corresponding to the current pixel address is a defective pixel (YES in step S14), the determination count data stored at the address specified by the read pointer is updated (step S16). For example, if the determination count data stored in the FIFO memory 22 is “3h”, it is set to “4h”. Thereafter, the read pointer is advanced to the next address (step S17), and the process returns to step S4.

  In the above description of the flowchart, the case where the vertical address Vad of the current pixel address is larger than maxY is the case where the processing of all pixel data is completed for one scene captured by the image sensor. Then, the read pointer is set to the first address in the FIFO memory 22 in order to process the pixel data for the next imaged scene. As described above, the image processing apparatus according to the present embodiment detects white point defects and black point defects for all the pixels used for imaging each scene for a plurality of scenes captured by the image sensor. Unlike the case of detecting a defective pixel from only one scene, it is possible to accurately detect a defective pixel of the image sensor. Thereby, it is possible to prevent a very thin line or a very small point having high or low brightness that is not a defective pixel from being erroneously determined as a defective pixel. This is very effective when there is a motion in the image, that is, when the brightness of each pixel differs greatly from scene to scene.

  Further, in the image processing apparatus according to the present embodiment, if the coefficients k1 and k2 stored in the coefficient storage register 19 and the predetermined number s stored in the number storage register 24 can be set from the outside, the entire image is dark. Alternatively, by changing the coefficients k1 and k2 and the number of times s according to the photographing environment such as the overall brightness, white spot defects and black spot defects can be detected more accurately. For example, if the coefficients k1 and k2 can be changed from the outside to a large value and a small value, respectively, so that it is easy to detect a white spot defect when the entire shooting environment is dark, the defective pixel is determined more accurately. be able to.

  Furthermore, since the image processing apparatus according to the present embodiment does not need to store the position of the defective pixel of the image sensor in advance, an individual non-volatile memory is not required, and the cost can be reduced. It is not necessary to perform complicated processing such as storing the position of the position in the nonvolatile memory. In addition, by changing the capacity of the FIFO memory 22, an image sensor with many defective pixels can be used, so that there is no restriction on the image sensor to be used. Thereby, an image sensor with a lower price can be used, and the cost of the entire image processing apparatus can be reduced.

It is a block diagram which shows the structural example of the image processing apparatus by this invention. It is a timing chart for demonstrating the vertical blanking period signal VBlank and the horizontal blanking period signal HBlank. 3 is a diagram illustrating an example of a pixel array of an image sensor in an image sensor unit 3. FIG. It is a block diagram which shows the structural example of the white spot / black spot detection complement part. 3 is a block diagram illustrating a configuration example of a white point / black point storage unit 6. FIG. 3 is a diagram illustrating an example of a data structure of data stored in a FIFO memory 22. FIG. 4 is a flowchart for explaining the operation of a determination control unit 21. 6 is a flowchart for explaining the operation of the determination control unit 21 when data is first written to the FIFO memory 22 once reset.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image processing apparatus 2 Host computer 3 Image sensor part 4 White point / black point correction apparatus 5 USB I / F part 6 White point / black point memory | storage part 7 White point / black point detection complement part 8 Pixel address generation part 9 Timing signal generation part 11 White Point / black point determination unit 12 to 16 Pixel data storage unit 17 Pixel complementation unit 18 MUX
19 Coefficient storage register 21 Judgment control unit 22 FIFO memory 23 Judgment comparison unit 24 Count storage register Ad Address data Din, Dout, Dusb Pixel data Det Judgment data Req Complement request signal VBlank Vertical blanking period signal HBlank Horizontal blanking period signal TM Timing signal

Claims (9)

An image processing apparatus that complements and outputs pixel data corresponding to defective pixels of an image sensor by a predetermined method,
When pixel data is output from the image sensor, a determination unit that determines whether a pixel corresponding to the pixel data is a defective pixel;
A storage unit that stores a pixel address of a pixel determined to be a defective pixel by the determination unit, and a determination number of times that the pixel is determined to be a defective pixel by the determination unit;
A determination unit that determines, as a defective pixel, a pixel whose number of determinations exceeds a predetermined number;
An image processing apparatus comprising: a complementing unit that complements and outputs pixel data corresponding to a pixel determined to be a defective pixel by the determination unit by a predetermined method. The storage unit
A data storage unit for storing the pixel address and the number of determination times;
A storage control unit that controls reading and writing of data with respect to the data storage unit,
The storage control unit determines that the pixel corresponding to the pixel data output from the image sensor is a defective pixel by the determination unit, and the pixel address and the number of determinations of the pixel are stored in the data storage unit. 2. The image processing apparatus according to claim 1, wherein if there is no pixel address, the pixel address corresponding to the pixel and the number of times of determination are written in the data storage unit.   The storage control unit determines that the pixel corresponding to the pixel data output from the image sensor is a defective pixel by the determination unit, and stores the pixel address and the number of determinations of the pixel in the data storage unit. 3. The image processing apparatus according to claim 2, wherein the number of times of determination of the pixel stored in the data storage unit is increased by one when there is.   The image processing apparatus according to claim 1, wherein the data storage unit is a FIFO memory.   The image processing apparatus according to claim 1, wherein the predetermined number of times can be set from outside.   The determination unit, when the brightness of the pixel data of the pixel to be determined and the sum of the brightness of the input and output pixel data input before and after the input of the pixel data are in a predetermined relationship, The image processing apparatus according to claim 1, wherein the pixel to be determined is determined to be a defective pixel.   The image processing apparatus according to claim 6, wherein the front and rear input pixel data is pixel data having the same color component as pixel data of the pixel to be determined.   The determination unit, when the brightness of the pixel data of the pixel to be determined is larger than the first multiplied pixel data obtained by multiplying the sum of the brightness of the front and rear input pixel data by a predetermined first coefficient, or When the brightness of the pixel data of the pixel to be determined is smaller than the second multiplied pixel data obtained by multiplying the sum of the brightness of the front and rear input pixel data by a predetermined second coefficient, The image processing apparatus according to claim 7, wherein a certain pixel is determined to be a defective pixel. The image processing apparatus according to claim 8, wherein each of the first and second coefficients can be set from outside.

Images