JP6701025B2 - Data processing device, data processing method, and program - Google Patents

Description

  The present invention relates to a data processing device, a data processing method, and a program.

  Among a plurality of pixels arranged in an image sensor such as a digital camera, there are some pixels which have become defective pixels (hereinafter referred to as defective pixels) for which correct pixel data cannot be generated for various reasons. The scratched pixels are initial scratched pixels (hereinafter referred to as process scratched pixels) generated in a manufacturing process and the like, and subsequent scratched pixels generated after product shipment (hereinafter referred to as post-scratched pixels). And, When correcting the image data of these defective pixels, image processing is performed using the defective data including the information indicating the positions of the defective pixels. The position of the defective pixel included in the defective data can be represented by the relative distance between the defective pixels. However, the number of bits representing the distance value is limited. Therefore, when a defective pixel does not exist within the maximum relative distance that can be represented by the number of bits, a pseudo defective pixel (hereinafter referred to as a dummy defective pixel) is sandwiched between the dummy defective pixel and the defective pixel. The position of the defective pixel is represented by the relative distance of.

  Further, the two defect data of the process defect pixel and the subsequent defect pixel may be integrated and treated as one defect data. When two flaw data are integrated into one flaw data, even if the flaw pixel position information (relative distance) included in the flaw data after integration is within the maximum relative distance described above, the flaw data after integration In some cases, scratch data of dummy scratch pixels before integration may remain. To solve this problem, for example, in the technique described in Patent Document 1, unnecessary dummy scratches are generated by converting the position information based on the relative distance to the absolute position (coordinate position), superimposing the position information, and then converting the position information based on the relative distance. Is not left after integration.

JP, 2011-82634, A

  However, in the method described in Patent Document 1, in order to prevent unnecessary dummy scratches from being left after integration, the position information represented by the relative distance is temporarily displayed on the pixel array in the image sensor when integrating the scratch data. It is necessary to convert to the absolute position (coordinate position) information of. For this reason, a RAM is required to store the information when the position information is converted and to store the expanded data.

  Therefore, an object of the present invention is to enable integration of defect data without leaving unnecessary dummy defect pixels without the need for processing such as expanding position information based on relative distances to absolute positions (coordinate positions). ..

  According to the present invention, integrated processing means for integrating the first and second flaw data representing the position of the defective pixel with a finite relative distance between the defective pixels as the third flaw data, and the third flaw. When the data is dummy flaw data inserted to relay between defective pixels, the dummy flaw data is inserted based on the relative distance of the dummy flaw data and the relative distance of the third flaw data integrated immediately before. Determining whether or not to integrate the dummy scratch data as the third scratch data, and excluding the dummy scratch data from the third scratch data when it is determined not to integrate the dummy scratch data, And

  According to the present invention, it is possible to integrate defect data without leaving unnecessary dummy defect pixels without the need to perform processing for expanding position information based on relative distances to absolute positions (coordinate positions).

It is a figure which shows the schematic structural example of the imaging device of this embodiment. It is a figure which shows the structural example of the flaw data integration part which concerns on 1st Embodiment. It is a flowchart of the flaw data integration process which concerns on 1st Embodiment. It is a figure which shows the flaw arrangement|positioning before and after integration and the flaw data example which concern on 1st Embodiment. It is a figure used for explanation of the crack data before and behind integration concerning a 1st embodiment. It is a figure which shows the structural example of the flaw data integration part which concerns on 2nd Embodiment. It is a flowchart of the flaw data integration process which concerns on 2nd Embodiment. It is a figure which shows the flaw arrangement|positioning before and after integration and the flaw data example which concern on 2nd Embodiment. It is a figure used for explaining the crack data before and behind integration concerning a 2nd embodiment.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.
<First Embodiment>
FIG. 1 is a diagram showing a schematic internal configuration example of an image pickup apparatus as an example to which the data processing apparatus of the present invention is applied. The image pickup apparatus of the present embodiment is applicable to digital cameras, digital video cameras, various mobile terminals such as smartphones and tablet terminals having a camera function, industrial cameras, vehicle-mounted cameras, medical cameras, and the like.

  1, the image pickup apparatus according to the present embodiment includes a lens 101, an image sensor 102, an A/D converter 103, a flaw data generation unit 105, a process flaw data holding unit 111, a flaw data integration unit 108, and an integrated flaw data holding unit. 112 and an image processing unit 110.

  The lens 101 is a lens of an image pickup optical system and forms an optical image of a subject or the like on an image pickup sensor of the image pickup element 102. The image sensor 102 is configured to have an image sensor such as a CCD or CMOS and a peripheral circuit thereof, and converts an optical image formed on the image sensor by the lens 101 into an electric signal (image signal). Output to the A/D converter 103. The A/D converter 103 converts the image pickup signal acquired and supplied by the image pickup element 102 into digital image data (hereinafter, referred to as image data 104). The image data 104 output from the A/D converter 103 is sent to the image processing unit 110 and the flaw data generation unit 105.

  Here, a plurality of pixels are two-dimensionally arranged in the image sensor of the image sensor 102, and some of the plurality of pixels have become defective pixels (scratch pixels) that cannot correctly generate pixel data. is there. The scratched pixels include, for example, a first scratched pixel (post-scratched pixel) that is generated after the product is shipped due to external factors such as cosmic rays and electrostatic breakdown, and changes over time, and the manufacturing process of the image sensor. There is a second defective pixel (process defective pixel) generated in the initial stage. Further, since the process-defective pixel is a defective pixel generated in the manufacturing process or the like, the position of the defective pixel, the state or type of the defective pixel, etc. are known in advance by inspection before shipping the product.

  The defect data generation unit 105 generates, from the image data 104, first defect data (hereinafter, referred to as rear defect data 106) which is defect data of a rear defect pixel. Specifically, the defect data generation unit 105 includes pixels in the pixel data of the image data 104 that have significantly different signal levels from the surrounding pixels (for example, the signal level difference between the surrounding pixels exceeds the threshold). Is detected as a post-scratch pixel. Then, the flaw data generation unit 105 generates the post flaw data 106 corresponding to the detected post flaw pixels.

  The post-scratch data 106 is data including position information of a defective pixel in the image sensor, information on the state and type of the defective pixel, parameters such as ISO sensitivity, shutter speed, and temperature. In the present embodiment, the position information of the defective pixel is represented by the relative distance between defective pixels (between defective pixels). Further, the relative distance between the defective pixels is information corresponding to the number of read pixels when each pixel data is read from the image sensor in a predetermined direction and order such as a raster scan order. That is, the number of pixels between a certain defective pixel and the next defective pixel among the pixels sequentially read from the image sensor in the raster scan order is used as information indicating the relative distance between the defective pixels. However, in reality, the number of bits used to represent the value of the relative distance (the number of pixels) is limited, and therefore the relative distance between the defective pixels can only represent a finite distance. For example, when the distance is represented by 16-bit data, the range represented by the 16-bit is a finite range of "0 to 65535", and the relative distance can be represented only within the finite range. Therefore, when the relative distance between the defective pixels exceeds the finite range, for example, a dummy defective pixel serving as a relay point is inserted between the defective pixels so that the relative distance between the defective pixels exceeds the finite range. I try not to. Hereinafter, the flaw data of the dummy flaw pixel will be referred to as dummy flaw data. The flaw data generation unit 105 sends the flaw data 106 (including dummy flaw data when the dummy flaw data is inserted) generated as described above to the flaw data integration unit 108.

  The process flaw data holding unit 111 is composed of, for example, a non-volatile memory, and holds second flaw data (hereinafter, referred to as process flaw data 107) which is flaw data of known process flaw pixels generated in a manufacturing process or the like. is doing. In the process flaw data 107, the position of the flaw pixel is represented as a relative distance corresponding to the number of pixels between the flaw pixels, as in the case of the post flaw data 106 described above. Also in the process flaw data 107, as in the case of the post flaw data 106 described above, when the relative distance between flaw pixels exceeds a finite range, a dummy flaw pixel serving as a relay point is inserted between the flaw pixels, The relative distance between the defective pixels is set within a finite range. The process flaw data holding unit 111 holds the process flaw data 107 (including dummy flaw data when the dummy flaw data is inserted) including the positions (relative distance information) of these process flaw pixels.

  The flaw data integration unit 108 is supplied with the post-scratch data 106 output from the flaw data generation unit 105 and the process flaw data 107 read from the process flaw data holding unit 111. Then, the defect data integration unit 108 integrates (merges) the subsequent defect data 106 and the process defect data 107 to generate third defect data (hereinafter, referred to as integrated defect data 109). In addition, the integration (merging) of the flaw data in the present embodiment is to generate flaw data in which both the post flaw data 106 and the process flaw data 107 are combined. However, in the case of the present embodiment, as will be described in detail later, if the flaw data to be integrated is dummy flaw data, the flaw data integrating unit 108 does not need the dummy flaw data. Or not. Then, when it is determined that the dummy scratch data is unnecessary dummy scratch data, the scratch data integrating unit 108 excludes the dummy scratch data as the integrated scratch data 109 from being integrated. A more detailed configuration and operation of the flaw data integration unit 108 will be described later. The integrated flaw data 109 thus generated by the flaw data integrating unit 108 is sent to and held in the integrated flaw data holding unit 112.

The integrated flaw data holding unit 112 reads the held integrated flaw data 109 and supplies it to the image processing unit 110 in response to a request from the image processing unit 110.
The image processing unit 110 uses the integrated defect data 109 read from the integrated defect data holding unit 112 to perform defect correction processing on the defect pixel of the image data 104. Specifically, the image processing unit 110 determines the position of the defective pixel (pixel data of the defective pixel in the image data 104) based on the positional information (information of the relative distance) included in the integrated defective data 109. Identify. Furthermore, the image processing unit 110 uses the parameter information such as the state and type of the defective pixel, the ISO sensitivity, the shutter speed, and the temperature, which are included in the integrated defective data 109, to perform the defective interpolation process on the defective pixel. Determine whether to do it. Then, when the image processing unit 110 determines to perform the flaw interpolation processing on the identified flaw pixel, the image processing unit 110 performs flaw correction processing on the pixel data of the flaw pixel. Specifically, the image processing unit 110 uses the pixel data of the peripheral pixels with respect to the defective pixel (pixel data of normal pixels around the defective pixel) among the pixels two-dimensionally arranged in the image sensor. Then, a defect correction process for performing interpolation or the like on the defective pixel is performed. The image data subjected to the image processing such as the defect correction by the image processing unit 110 in this manner is sent to a display unit, an encoding unit, a recording unit and the like (not shown). The description of the display unit, the encoding unit, the recording unit, etc. will be omitted.

FIG. 2 is a diagram showing a detailed configuration example of the flaw data integration unit 108 of FIG.
The flaw data integration unit 108 includes a distance update unit 201, a distance comparison unit 202, an integration target selection unit 205, and an unnecessary dummy flaw determination unit 207.

  The post-scratch data 106 generated by the flaw data generation unit 105 and the process flaw data 107 read from the process flaw data holding unit 111 are input to the distance update unit 201. Although the details will be described later, the distance update unit 201 determines whether the relative distance included in the post flaw data 106 or the relative distance included in the process flaw data 107 is based on the comparison result of the relative distance by the distance comparison unit 202. Update processing is performed on the other hand. Details of the relative distance information updating process in the distance updating unit 201 will be described later. After the distance update unit 201 updates one of the relative distances, the flaw data 203 and the process flaw data 204 are sent to the integration target selection unit 205 and the distance comparison unit 202.

  The distance comparison unit 202 compares the relative distance included in the post-scratch defect data 203 supplied from the distance update unit 201 with the relative distance included in the process defect data 204. Details of the relative distance comparison processing in the distance comparison unit 202 will be described later. The distance comparison unit 202 sends the information of the comparison result of the relative distances to the integration target selection unit 205 and the unnecessary dummy flaw determination unit 207, and also feeds it back to the distance update unit 201.

  The integration target selecting unit 205 appropriately selects the post-scratch data 203 and the process scratch data 204 supplied from the distance updating unit 201 based on the information on the comparison result from the distance comparing unit 202, and outputs the integrated target scratch data 206. To do. In the case of the present embodiment, the integration target selection unit 205 corresponds to integration processing means, and the integration defect data 109 output from the defect data integration unit 108 is generated by the integration target defect data 206 selected by the integration target selection unit 205. Will be done. However, in the case of the present embodiment, when the integration target flaw data 206 selected from the integration target selecting unit 205 is dummy flaw data, a dummy flaw determining unit 207 described later does not require the dummy flaw data to be a dummy flaw. It is determined whether or not it is data. Details of the selection processing of the integration target flaw data 206 in the integration target selection unit 205 will be described later. The integration target flaw data 206 selected by the integration target selecting unit 205 is sent to the unnecessary dummy flaw determining unit 207.

  The unnecessary dummy flaw determination unit 207 determines whether the integration target flaw data 206 is integrated based on the information of the comparison result by the distance comparison unit 202 and the information included in the integration target flaw data 206 selected by the integration target selection unit 205. It is determined whether it is unnecessary dummy flaw data. Then, when the unnecessary dummy flaw determination unit 207 determines that the integration target flaw data 206 is unnecessary dummy flaw data, the unnecessary dummy flaw data 206 excludes the integration target flaw data 206 (dummy flaw data) from the integration target flaw data. Details of the determination process and the unnecessary dummy defect data exclusion process in the unnecessary dummy defect determination unit 207 will be described later. All of the integration target flaw data that has not been excluded by the unnecessary dummy flaw determination unit 207 is output as the integrated flaw data 109 from the flaw data integration unit 108 in FIG. 2 and is held in the integrated flaw data holding unit 112 in FIG. ..

  FIG. 3 is a flowchart of the processing performed by the flaw data integration unit 108 shown in FIG. The process shown in the flowchart of FIG. 3 may be realized by, for example, the CPU executing the program according to the present embodiment. The program according to the present embodiment may be prepared in advance in, for example, a ROM (not shown), read from an external storage medium (not shown), or downloaded from a network such as the internet (not shown), It may be loaded in a RAM or the like (not shown). The same applies to other flowcharts described later. Note that steps S301 to S314 of each process in FIG. 3 are abbreviated as S301 to S314. In the following description, the post-scratch data 106 (203) is the scratch data A, the relative distance is the relative distance a, the process scratch data 107 (204) is the scratch data B, the distance information is the relative distance b, and the integrated scratch is the integrated scratch. Let the data 109 be flaw data M.

  When the defect data A (post-defect data 106) and the defect data B (process defect data 107) are input in S301, the defect data integration unit 108 outputs the defect data A and B via the distance update unit 201. It is sent to the distance comparison unit 202. After S301, the flaw data integration unit 108 advances the process to S302.

  In S302, the distance comparison unit 202 compares the relative distance a of the flaw data A with the relative distance b of the flaw data B, and determines whether the relative distance a is less than or equal to the relative distance b. When the distance comparison unit 202 determines that the relative distance a is less than or equal to the relative distance b (Yes), the process of the flaw data integration unit 108 proceeds to S303. On the other hand, when the distance comparison unit 202 determines that the relative distance a is larger than the relative distance b (No), the process of the flaw data integration unit 108 proceeds to S304. S303 and S304 are processes performed by the integration target selection unit 205.

  In S303, the integration target selection unit 205 selects the flaw data A as integration target flaw data, while in S304, the integration target selection unit 205 selects the flaw data B as integration target flaw data. After these S303 and S304, the flaw data integration unit 108 advances the processing to S305. S305 is a process performed by the unnecessary dummy flaw determination unit 207.

  In step S305, the unnecessary dummy flaw determination unit 207 determines whether or not the comparison result by the distance comparison unit 202 is the same distance (the same distance) as the relative distance a and the relative distance b. If the relative distance a and the relative distance b are equal (Yes), the unnecessary dummy flaw determination unit 207 advances the process to S308. On the other hand, when the relative distance a and the relative distance b are different (No), the unnecessary dummy flaw determination unit 207 advances the process to S306.

  In S306, the unnecessary dummy flaw determination unit 207 determines whether the integration-target flaw data selected in S303 or S304 is dummy flaw data. The unnecessary dummy flaw determination unit 207 advances the processing to S307 if the integration-target flaw data is dummy flaw data (Yes), and advances the processing to S308 if it is not the dummy flaw data (No).

  In step S307, the unnecessary dummy flaw determination unit 207 discards the integration target flaw data (that is, the dummy flaw data) selected in step S303 or S304 without outputting the integration target flaw data. After S307, the flaw data integration unit 108 advances the processing to S309.

  In step S308, the unnecessary dummy flaw determination unit 207 outputs the integration target flaw data selected in step S303 or S304 (that is, the flaw data that is not the dummy flaw data), leaving the integration target flaw data. After S308, the flaw data integration unit 108 advances the processing to S309.

  In S309, the flaw data integration unit 108 determines whether the integration process for all the flaw data A and B has been completed. If the flaw data integration unit 108 determines that the integration processing for all the flaw data A and B has been completed (Yes), the processing of the flowchart of FIG. 3 is terminated. On the other hand, when the flaw data integration unit 108 determines that the integration processing for all the flaw data A and B is not completed (the flaw data still remains) (No), the processing proceeds to S310. S310 is a process performed by the distance update unit 201.

  In S310, the distance updating unit 201 determines whether the comparison result by the distance comparing unit 202 indicates that the relative distance a is less than or equal to the relative distance b. When the relative distance a is less than or equal to the relative distance b (Yes), the distance update unit 201 advances the process to S311. On the other hand, when the relative distance a is larger than the relative distance b (No), the distance update unit 201 advances the process to S313.

  In S311, the distance updating unit 201 updates the relative distance of the flaw data that has not been selected as an integration target in S302 to S304 by a value obtained by subtracting the relative distance of the flaw data selected as an integration target. That is, in the case of proceeding to S311, the flaw data which is not selected as the integration target because the relative distance is large is the flaw data B, while the integration target flaw data which is selected because the relative distance is small is the flaw data A. Is. Therefore, the distance updating unit 201 in this case updates the value of the relative distance b of the flaw data B by a value obtained by subtracting the value of the relative distance a of the flaw data A. After S311, the flaw data integration unit 108 advances the process to S312. In step S<b>312, the flaw data integration unit 108 receives the next flaw data (in this case, the next flaw data A), and sends the flaw data A to the distance comparison unit 202 via the distance update unit 201. After S312, the flaw data integration unit 108 returns to the processing of S302.

  In S313, the distance update unit 201 updates the relative distance of the flaw data that is not selected as the integration target in S302 to S304 by a value obtained by subtracting the relative distance of the flaw data selected as the integration target. That is, in the case of proceeding to S313, it is the flaw data A that has not been selected as the integration target, while the selected integration target flaw data is the flaw data B. Therefore, the distance updating unit 201 in this case updates the value of the relative distance a of the flaw data A by a value obtained by subtracting the value of the relative distance b of the flaw data B. After S313, the flaw data integration unit 108 advances the processing to S314. In S<b>314, the flaw data integration unit 108 receives the next flaw data (in this case, the next flaw data B) and sends the flaw data B to the distance comparison unit 202 via the distance update unit 201. After S314, the flaw data integration unit 108 returns to the processing of S302.

  Note that the processes of S305 and S306 may be interchanged. In this case, in S305, it is determined whether or not the integration target flaw data is dummy flaw data, and in S306, it is determined whether the values of the relative distance a and the relative distance b are equal. Then, in S305, if it is determined that the integration target flaw data is dummy flaw data, the process proceeds to S306, and if it is determined that it is not the dummy flaw data, the process proceeds to S308. In S306, if the values of the relative distance a and the relative distance b are equal, the process proceeds to S308, and if they are not equal (different), the process proceeds to S307.

FIG. 4A and FIG. 4B are diagrams showing an example of states of flaw data before and after the integration process performed by the flaw data integration unit 108.
FIG. 4A shows the position of the defective pixel xA of the defective data A, the position of the defective pixel xB of the defective data B, and the integrated defect of the integrated defective data M in a state corresponding to the two-dimensional pixel arrangement of the image sensor. An example of the positions of the pixels xA and xB is shown. In addition, the position of the dummy defect pixel d of the dummy defect data is also included in the example of FIG.

  FIG. 4B shows an example of flaw data corresponding to each of the flaw pixels xA, xB, and d of the flaw data A, the flaw data B, and the integrated flaw data M. For example, in the flaw data “03_A” of FIG. 4B, the type of the flaw pixel is the post-scratch pixel, and the relative distance a is the number of pixels between the flaw pixel read immediately before in the raster scan order described above. It indicates that the distance is equivalent to 3". Similarly, for example, in the flaw data “01_B” of FIG. 4B, the type of the flaw pixel is the process flaw pixel, and the relative distance b is between the flaw pixel read immediately before in the raster scan order described above. It indicates that the distance is equivalent to the number of pixels "1". Further, for example, in the scratch data “15_D” of FIG. 4B, the type of the scratch pixel is a dummy scratch pixel, and the relative distance is the pixel number “15” between the scratch pixel read immediately before in the raster scan order. Indicates that the distance is equivalent to. The same applies to other scratch data. The relative distance of the first scratched pixel in the raster scan order described above is represented by the distance from the first pixel (pixel at the start point coordinate) read in the raster scan order.

  As shown in FIGS. 4A and 4B, the flaw pixels xA, xB, and d of the integrated flaw data M are the flaw pixel xA of the flaw data A, the flaw pixel xB of the flaw data B, and the dummy. The dummy defect pixel d of the defect data is integrated.

FIG. 5A to FIG. 5C show the relative distance information updating process performed by the flaw data integration unit 108, taking the example of the flaw data shown in FIGS. 4A and 4B. It is a figure used for description of an example.
5A is an example of flaw data corresponding to each flaw pixel of the flaw data A, FIG. 5B is an example of flaw data corresponding to each process flaw pixel of the flaw data B, and FIG. 5C is integrated. The example of the flaw data corresponding to each integrated flaw pixel of the flaw data M is shown. Each of these flaw data also includes flaw data corresponding to the dummy flaw pixels of the dummy flaw data. The arrangement order of the defect data in FIGS. 5A to 5C corresponds to the raster scan order described above. Further, the flaw data “03_A”, “01_B”, “15_D” and the like in FIGS. 5A to 5C are the same as those described in FIG. 4B.

  In the flaw data integration unit 108, the distance comparison unit 202 first compares the relative distance of the leading flaw data 501 of the flaw data A with the relative distance of the leading flaw data 502 of the flaw data B. Then, as a result of the comparison by the distance comparison unit 202, the flaw data integration unit 108 selects the flaw data having a smaller (shorter) relative distance as the integration target flaw data by the integration target selection unit 205. In the example of FIGS. 4A, 4B, and 5A to 5C, the scratch data 501 (03_A) at the beginning of the scratch data A has a relative distance value of “03” ( It is a distance corresponding to the number of pixels "3"). On the other hand, the scratch data 502 (01_B) at the beginning of the scratch data B has a relative distance value of “01” (distance corresponding to the number of pixels “1”). Then, comparing these relative distances, the flaw data 502 (01_B) has the smaller relative distance. Therefore, in the defect data integration unit 108, the integration target selection unit 205 selects the defect data 502 (01_B) of the defect data B as integration target defect data. Further, the flaw data 502 selected as the integration target flaw data is the flaw data B and is not the dummy flaw data. Therefore, the unnecessary dummy flaw determination unit 207 outputs the flaw data 502 as the integrated flaw data M. As a result, the leading flaw data of the integrated flaw data M becomes the flaw data 502 (01_B).

  Next, the flaw data integration unit 108 reads the next flaw data in the raster scan order for the side selected as the integration target (the flaw data B side in this case). That is, in the example of FIGS. 4A, 4B, and 5B, the next flaw data 504 in the raster scan order in the flaw data B selected as the integration target is the flaw data “05_B”. Become. On the other hand, the flaw data integration unit 108 sets the relative distance of the flaw data on the side not selected as the integration target (in this case, the flaw data A side) to the difference value from the relative distance of the flaw data selected as the integration target. To update. In this example, the flaw data 501 (03_A) of the flaw data A not selected as the integration target has a relative distance value of “03”, and the flaw data 502 (01_B) of the flaw data B selected as the integration target is relative. The distance value is "01". Then, the difference between the relative distance value “03” of the scratch data 501 (03_A) and the relative distance value “01” of the scratch data 502 (01_B) is “03”−“01”=“02”. .. Therefore, in the defect data A on the side not selected as the integration target, the next defect data 503 in the raster scan order is updated to the defect data “02_A”.

  The same process is repeated thereafter, for example, the process proceeds to the stage where the relative distance of the flaw data 505 (13_A) of the flaw data A and the relative distance of the flaw data 506 (00_D) of the flaw data B are compared by the distance comparing unit 202. Suppose In the example of FIGS. 5A and 5B, the value of the relative distance of the flaw data 505 (13_A) is “13”, and the value of the relative distance of the flaw data 506 (00_D) is “00”. Therefore, the integration target selection unit 205 selects the defect data 506 (00_D) as the integration target defect data. Further, the values of the relative distances between the scratch data 505 (13_A) and the scratch data 506 (00_D) are “13” and “00” and are not equal. However, the scratch data 506 is dummy scratch data “00_D”, although it is selected as the integration target scratch data by the integration target selection unit 205. Therefore, in this case, the unnecessary dummy flaw determination unit 207 discards the dummy flaw data “00_D” without integrating and sets no output 507.

  After that, as described above, the flaw data integration unit 108 reads the next flaw data in the raster scan order for the side not selected for integration (the flaw data B side in this case). That is, in the defect data B on the side selected as the integration target, the next defect data 509 in the raster scan order becomes the defect data “15_B”. On the other hand, the flaw data integration unit 108 determines the difference between the flaw data 508 of the flaw data A not selected as the integration target and the relative distance value “00” of the flaw data 506 (00_D) on the side selected as the integration target. Data (13-00=13). In the case of this example, the value of the relative distance of the flaw data 505 of the flaw data A not selected as the integration target is "13", and the value of the relative distance of the flaw data 506 of the flaw data B selected as the integration target is " 00". Therefore, in the defect data A on the side not selected as the integration target, the next defect data 508 in the raster scan order becomes the defect data “13_A”. Since the subsequent steps are the same as the above, the description thereof will be omitted.

  As described above, in the first embodiment, the defective pixel position of the image sensor is represented by the relative distance between the defective pixels, and when the relative distance between the defective pixels exceeds a finite range that can be represented by a predetermined number of bits, the Dummy scratch pixels that can be represented by a relative distance within a finite range are inserted. Further, the imaging apparatus of the present embodiment selects the integration target flaw data based on the comparison result of the relative distance between the flaw data A and the flaw data B, and further determines whether the integration target flaw data is the dummy flaw data. Then, when the relative distance between the flaw data A and the flaw data B is not equal and the target flaw data to be integrated is dummy flaw data, the imaging apparatus of the present embodiment does not integrate the dummy flaw data as the integrated flaw data M. I am trying to discard it. As a result, in the image pickup apparatus according to the present embodiment, it is possible to integrate defect data without leaving unnecessary dummy defect pixels, for example, without the need to perform processing of expanding position information based on relative distances to absolute positions (coordinate positions). Has become.

<Second Embodiment>
The second embodiment will be described below. Note that, in the second embodiment, the schematic configuration of the image pickup apparatus is the same as that in FIG. 1, and a description thereof will be omitted.
In the second embodiment, when the dummy defect data is discarded, the relative distance information is updated based on the defective pixel position immediately before the dummy defect data is discarded, which is different from the first embodiment. .

  FIG. 6 is a diagram showing a detailed configuration example of the flaw data integration unit 108 in the case of the second embodiment. Note that, in FIG. 6, components that are substantially the same as the configuration in FIG. 2 are assigned the same reference symbols as the reference symbols in FIG. For example, the distance update unit 201, the distance comparison unit 202, the integration target selection unit 205, the unnecessary dummy flaw determination unit 207, the post flaw data 106 and 203, the process flaw data 107 and 204, the integration target flaw data 206, and the integrated flaw data 109 are This is similar to the example of FIG.

  In the example of FIG. 6, when the unnecessary dummy flaw determination unit 207 determines that the integration target flaw data 206 is unnecessary dummy flaw data and discards it, the distance information storage unit 601 discards the integration target flaw data 206 (dummy kiss data). Information) of the relative distance included in (data).

  The distance adjusting unit 602 uses the relative distance information (relative distance information of the dummy flaw data discarded by the unnecessary dummy flaw determining unit 207) held in the distance information holding unit 601 to determine the relative distance of the integration target flaw data 206. Adjust the distance. Specifically, the distance adjusting unit 602 compares the value of the relative distance of the integration target flaw data 206 with the value of the relative distance stored in the distance information storage unit 601 during the immediately preceding integration processing (that is, the value of the dummy flaw data). Adjustment processing is performed such that the value of the relative distance) is added. Note that, here, the adjustment process of adding the value of the relative distance of the scratch data discarded immediately before is given as an example, but other adjustment methods may be used. In the case of the second embodiment, the integration target flaw data 603 after the relative distance is adjusted by the distance adjusting unit 602 is supplied to the unnecessary dummy flaw determining unit 207.

  FIG. 7 is a flowchart of processing performed by the flaw data integration unit 108 according to the second embodiment. In the flowchart of FIG. 7, each processing of S301 to S314 is the same as the processing of S301 to S314 of the flowchart of FIG. 3 described above, and thus the description thereof will be omitted.

  In the case of the second embodiment, the unnecessary dummy flaw determination unit 207 determines in S306 that the integration subject flaw data is dummy flaw data, and when the dummy flaw data is discarded in S307, the process proceeds to S701. In step S<b>701, the unnecessary dummy flaw determination unit 207 causes the distance information holding unit 601 to hold information on the relative distance of the dummy flaw data.

  Further, in the defect data integration unit 108 of the second embodiment, after S303 and S304, the process proceeds to S702. The process of S702 is a process performed by the distance adjusting unit 602. In step S<b>702, the distance adjusting unit 602 determines whether or not output has been performed (no output 507 as illustrated in FIG. 5C described above) in the immediately preceding integration process based on the information stored in the distance information storage unit 601. Determine whether or not. That is, if the distance information holding unit 601 holds the information on the relative distance, it can be determined that the previous integration process has not output. Then, the distance adjusting unit 602 advances the processing to S703 when the immediately preceding integration processing has not output (Yes). On the other hand, if there is an output (No), the process of the flaw data integration unit 108 proceeds to the process of S305 described above.

  In S703, the distance adjustment unit 602 adds the value of the relative distance held in the distance information holding unit 601 to the relative distance of the integration target flaw data, and then advances the processing to S308. Subsequent processing is the same as the above-described flowchart of FIG.

  FIG. 8A and FIG. 8B are diagrams showing an example of states of flaw data before and after the integration process performed by the flaw data integration unit 108 in the second embodiment. Similar to FIG. 4A described above, FIG. 8A shows the defective pixel xA of the defective data A, the defective pixel xB of the defective data B, the integrated defective pixels xA and xB of the integrated defective data M, and the dummy defective data. An example of the position of the dummy defect pixel d is shown. 8B shows an example of the defect data corresponding to each of the defect pixels xA, xB, and d of the defect data A, the defect data B, and the integrated defect data M, similarly to FIG. 4B described above. There is.

  Also in the second embodiment, as described above, the flaw pixels xA, xB, and d of the integrated flaw data M shown in FIGS. 8A and 8B are the flaw pixels xA and flaw data of the flaw data A. The defective pixel xB of B and the dummy defective pixel d of the dummy defective data are integrated. The flaw pixels xA, xB, d and the flaw data are the same as those described with reference to FIGS. 4A and 4B, and thus detailed description thereof will be omitted.

  FIGS. 9A to 9C show the relative distances performed by the defect data integration unit 108 of the second embodiment in the example of the defect data shown in FIGS. 8A and 8B. It is a figure used for explaining an example of adjustment processing and update processing. FIG. 9A shows an example of flaw data corresponding to each flaw pixel of the flaw data A, similarly to FIG. 5A described above. Similarly to FIG. 5B, FIG. 9B illustrates an example of defect data corresponding to each process defect pixel of the defect data B, and FIG. 9C illustrates the same as FIG. 5C. The example of the flaw data corresponding to each integrated flaw pixel of the integrated flaw data M is shown. Each of these flaw data also includes flaw data corresponding to the dummy flaw pixels of the dummy flaw data.

  In the example of FIGS. 9A to 9C, for example, the relative distance of the defect data 901 (13_D) of the defect data A and the relative distance of the defect data 902 (15_D) of the defect data B by the distance comparison unit 202. Let's say that we have advanced to the stage where the comparison is made. In the case of this example, the flaw data 901 (13_D) of the flaw data A and the flaw data 902 (15_D) of the flaw data B are both dummy flaw data. In the case of this example, since the relative distance value of the flaw data 901 (13_D) is “13” and the relative distance value of the flaw data 902 (15_D) is “15”, the integration target selection unit 205 The data 901 (13_D) is selected as the integration target flaw data. However, the flaw data 901 (13_D) is dummy flaw data, and the relative distance of the flaw data 901 (13_D) and the relative distance of the flaw data 902 (15_D) are different. Therefore, in the unnecessary dummy flaw determination unit 207, the flaw data 901 (13_D) is discarded and no output is given 903. Here, in the case of the second embodiment, the unnecessary dummy flaw determination unit 207 stores the information of the relative distance value “13” of the discarded dummy flaw data (the flaw data 901 (13_D)) in the distance information holding unit 601. Hold it.

  As the next integration processing, in the flaw data integration unit 108, the distance comparison unit 202 compares the relative distance of the flaw data 904 (04_A) of the flaw data A with the relative distance of the flaw data 905 (02_D) of the flaw data B. Done. In this case, since the relative distance value of the flaw data 904 (04_A) is “04” and the relative distance value of the flaw data 905 (02_D) is “02”, the integration target selection unit 205 causes the flaw data 905( 02_D) is selected as the integration target flaw data. However, in this case, the defect data is discarded in the immediately preceding integration process, and the output is 903. Therefore, the distance adjusting unit 602 compares the relative distance value “02” of the flaw data 905 (02_D) with the relative distance value “13” of the flaw data 901 (13_D) held in the distance information holding unit 601 immediately before. The value "15" obtained by adding "is" is used as information on the relative distance. As a result, the distance adjusting unit 602 sends new flaw data “15_D” in which the relative distance of the flaw data 905 (02_D) is adjusted to the value “15” to the unnecessary dummy flaw determining unit 207 as the flaw data to be integrated. .. The unnecessary dummy flaw determination unit 207 at this time outputs the integration target flaw data 906 (15_D) after the relative distance is adjusted by the distance adjustment unit 602 as the integrated flaw data M.

  After that, it is assumed that the distance comparison unit 202 has advanced to a stage where the relative distance between the flaw data 907 (12_A) of the flaw data A and the relative distance between the flaw data 908 (13_D) of the flaw data B are compared. In the immediately preceding integration process, the flaw data is discarded and no output is performed, and the distance information holding unit 601 holds, for example, the value “02” of the flaw data 909 (02_D) as the immediately preceding distance information. And In the case of this example, since the relative distance of the flaw data 907 (12_A) is “12” and the relative distance of the flaw data 908 (13_D) is “13”, the integration target selection unit 205 causes the flaw data 907 (12_A). Is selected as the scratch data to be integrated. In the case of this example, in the immediately preceding integration process, the flaw data is discarded and no output is made. Therefore, the distance adjusting unit 602 compares the relative distance value “12” of the flaw data 907 (12_A) with the relative distance value “02” of the flaw data 909 (02_D) held in the distance information holding unit 601 immediately before. The value "14" obtained by adding "is used as information on the relative distance. As a result, from the distance adjusting unit 602, new scratch data “14_A” in which the relative distance of the scratch data 907 (12_A) is adjusted to the value “14” is sent to the unnecessary dummy scratch determining unit 207 as integration target scratch data. .. The unnecessary dummy flaw determination unit 207 at this time outputs the integrated flaw data 910 (14_A) after the relative distance is adjusted by the distance adjustment unit 602 as the integrated flaw data M. The subsequent description will be omitted.

  As described above, in the second embodiment, when the dummy scratch data is discarded during the integration process, the information on the relative distance of the dummy kiss data is held, and the held information is used to Next, the information on the relative distance of the selected integration target flaw data is adjusted. As a result, according to the second embodiment, not only is it possible to integrate defect data without leaving unnecessary dummy defect pixels, but it is also possible to generate integrated defect data in consideration of the relative distance of discarded dummy defect data. It will be possible.

<Other embodiments>
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program. It can also be realized by the processing. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

  The above-described embodiments are merely examples of specific embodiments for carrying out the present invention, and the technical scope of the present invention should not be limitedly interpreted by these. That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

  102 image sensor, 105 flaw data generation unit, 108 flaw data integration unit, 110 image processing unit, 201 distance update unit, 202 distance comparison unit, 205 integration target selection unit, 207 unnecessary dummy flaw determination unit, 601 distance information holding unit, 602 Distance adjuster

Claims (13)

Integrated processing means for integrating the first and second flaw data representing the position of the defective pixel with a finite relative distance between the defective pixels as the third flaw data,
When the third flaw data is dummy flaw data inserted to relay between defective pixels, the relative distance of the dummy flaw data and the relative distance of the third flaw data integrated immediately before. Based on, it is determined whether or not to integrate the dummy flaw data as the third flaw data, and when it is determined that the dummy flaw data is not integrated, an exclusion unit that excludes the dummy flaw data from the third flaw data.
A data processing device comprising: The first flaw data is flaw data in which the position of the defective pixel detected from the image input from the image sensor is represented by the relative distance,
The data processing device according to claim 1, wherein the second flaw data is flaw data in which a position of a known defective pixel is represented by the relative distance. A relative distance of the first flaw data and a relative distance of the second flaw data, and a comparison unit for comparing the relative distance,
Based on the result of the comparison, the integration processing means selects one of the first flaw data and the second flaw data having a smaller value of the relative distance to obtain the third flaw data. The data processing apparatus according to claim 1, wherein the data processing apparatus comprises:   When the relative distance of the first flaw data and the relative distance of the second flaw data are equal to each other as a result of the comparison, the exclusion means determines that the selected flaw data is the dummy flaw data. The data processing device according to claim 3, wherein the data processing device is not excluded from the third flaw data.   When the relative distance of the first flaw data is different from the relative distance of the second flaw data, the excluding means determines the third flaw if the selected flaw data is the dummy flaw data. The data processing device according to claim 3, wherein the data processing device is excluded from the data.   Of the first flaw data and the second flaw data, the value of the relative distance of the flaw data that is not integrated as the third flaw data is the value of the relative distance that is integrated as the third flaw data. 6. The data processing apparatus according to claim 3, further comprising an updating unit that updates based on the value of the relative distance of the scratch data.   The comparison means compares the relative distance of the next flaw data of the flaw data integrated as the third flaw data with the relative distance of the flaw data of which the value is updated. The data processing device according to claim 6.   The updating means uses the relative distance value of the flaw data that is not integrated as the third flaw data from the value of the relative distance of the flaw data that is integrated as the third flaw data as the update. The data processing device according to claim 6 or 7, wherein the value of is subtracted.   Adjusting means for adjusting the relative distance of the flaw data next to the flaw data integrated as the third flaw data, based on the value of the relative distance of the dummy flaw data excluded from the third flaw data. The data processing device according to claim 1, further comprising:   The adjustment means, as the adjustment, with respect to the value of the relative distance of the flaw data next to the flaw data integrated as the third flaw data, the dummy flaw excluded from the third flaw data. 10. The data processing device according to claim 9, wherein the value of the relative distance of the data is added.   7. An image processing means for specifying the position of the defective pixel of the image input from the image sensor based on the third flaw data and performing a correction process for the data of the defective pixel. The data processing device according to any one of 1 to 10. Integrating the first and second flaw data representing the position of the defective pixel with a finite relative distance between the defective pixels as the third flaw data, and
When the third flaw data is dummy flaw data inserted to relay between defective pixels, the relative distance of the dummy flaw data and the relative distance of the third flaw data integrated immediately before. Based on, it is determined whether or not to integrate the dummy flaw data as the third flaw data, and if it is determined not to integrate, to exclude the dummy flaw data from the third flaw data,
A data processing method for a data processing device, comprising:   A program for causing a computer to function as each unit of the data processing device according to claim 1.

Images