JP3757891B2 - Foreign object detection method, foreign object detection apparatus, and imaging apparatus - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a foreign object detection method capable of detecting a foreign object that is present away from a pixel surface of an image sensor, and in particular, can detect a reduction range of received light amount on the pixel surface due to the foreign object and The present invention relates to a foreign object detection method capable of predicting the distance from the pixel surface.
[0002]
[Prior art]
In general, when an image pickup apparatus is assembled, foreign matter such as dust may be mixed inside the image pickup apparatus and may be reflected in the picked-up image. As a countermeasure, for example, Japanese Patent Laid-Open No. 9-51459 describes a method of detecting an image defect due to a foreign substance attached to a lens by taking a plurality of images while changing the lens focus. Japanese Patent Application Laid-Open No. 2000-31314 describes a method of interpolating image data of a defective pixel portion with surrounding image data.
[0003]
[Problems to be solved by the invention]
By the way, in recent years, the spread of mobile terminals such as mobile phones has been rapidly increasing, and accordingly, the demand for small-sized imaging devices incorporated in or connected to mobile terminals has increased rapidly. As such a small image pickup apparatus, a non-focusing fixed lens is used, and the foreign object detection method by moving the lens focus as described in JP-A-9-51459 cannot be used. Further, in order to perform production that satisfies enormous demand, it is necessary to detect foreign matter in as short a time as possible, and there is a demand for a foreign matter detection method that can be performed simply and in a short time.
[0004]
The present invention has been made in view of the circumstances as described above, and provides a foreign object detection method capable of detecting a foreign object even in a non-focusing fixed lens and performing the detection in a short time. To do. In addition, there is provided a foreign matter detection method capable of detecting the presence of the foreign matter even when the foreign matter is present at a position away from the pixel surface. Further, not only the presence / absence of a foreign substance but also a range of reduction in the amount of light received on the pixel surface due to the foreign substance (more specifically, the spread of the range on the pixel surface where the amount of received light is reduced by the same foreign substance) is detected. Provided is a foreign object detection method capable of Furthermore, a foreign object detection method capable of predicting the distance of the foreign object from the pixel surface is provided.
[0005]
[Means for Solving the Problems]
The foreign matter detection method according to the present invention detects foreign matter that exists between a pixel surface in which a plurality of pixels are arranged on the same surface and a lens that emits incident light to the pixel surface. A method for integrating a received light amount in a first pixel region composed of a plurality of pixels on the pixel surface; and a foreign object detecting step for detecting the presence of the foreign material based on a result of the calculation. Prepare.
[0006]
In the foreign matter detection method according to the present invention, the foreign matter detection step compares the total amount of received light in the first pixel region with a predetermined amount of received light, and based on the comparison result, the first pixel. A decrease in the amount of received light due to the foreign matter in the region is detected.
[0007]
In the foreign matter detection method according to the present invention, the predetermined amount of received light in the foreign matter detection step is greater than when the first pixel region is located at the center of the pixel surface. The case where it is located at the edge of the pixel surface is smaller.
[0008]
In the foreign matter detection method according to the present invention, the foreign matter detection step may include the first pixel when the foreign matter is not present between the total amount of received light in the first pixel region and the pixel surface / lens. The total amount of received light in the region is compared, and based on the comparison result, a decrease in the received light amount due to the foreign matter in the first pixel region is detected.
[0009]
In the foreign matter detection method according to the present invention, the foreign matter detection step includes a second amount of light received in the first pixel region and one or a plurality of pixels existing around the first pixel region. The pixel region is normalized and compared with the amount of light received per area, and a decrease in the amount of light received by the foreign matter in the first pixel region is detected based on the comparison result.
[0010]
Further, the foreign matter detection method according to the present invention detects foreign matter existing between a pixel surface formed by arranging a plurality of pixels on the same surface and a lens that emits incident light to the pixel surface. A foreign matter detection method, comprising: a received light amount integration step of calculating a sum of received light amounts in each of a plurality of first pixel regions having the same pixel on the pixel surface and having a different number of pixels, and the foreign matter based on the calculation result A foreign matter detection step of detecting a reduction range of the amount of received light on the pixel surface.
[0011]
In the foreign matter detection method according to the present invention, the foreign matter detection step may include the first pixel when the foreign matter is not present between the total amount of received light in the first pixel region and the pixel surface / lens. The total amount of received light in the region is compared, and the first pixel region in which the difference between them is a predetermined value is detected as a reduction range of the received light amount on the pixel surface due to the foreign matter.
[0012]
Further, in the foreign matter detection method according to the present invention, the foreign matter detection step includes a second pixel region including the first pixel region and one or a plurality of pixels existing around the first pixel region; The rate of change in the amount of received light is calculated for each of the plurality of first pixel regions, and the first pixel region in which the rate of change is the largest is defined as the reduction range of the amount of received light on the pixel surface due to the foreign matter. Is detected.
[0013]
In the foreign matter detection method according to the present invention, the calculation of the change rate may be performed by calculating a sum of received light amounts in the first pixel region and a received light amount in the second pixel region. This is performed by normalizing the quantity and calculating the ratio of the sum after normalization.
[0014]
The foreign object detection method according to the present invention further includes an image correction step of performing a predetermined correction on the output from the pixels existing within the reduction range of the received light amount.
[0015]
Further, in the foreign matter detection method according to the present invention, an image correction step of correcting the output from the pixels existing within the reduction range of the received light amount based on the output of the pixels existing outside the reduction range of the received light amount. Further prepare.
[0016]
The foreign matter detection method according to the present invention further includes a foreign matter position predicting step for predicting a position of the foreign matter in a direction perpendicular to the pixel surface based on the reduction range of the received light amount.
[0017]
In the foreign matter detection method according to the present invention, the foreign matter position predicting step may correspond to a known table associating the reduction range of the received light amount with the foreign matter position or the reduction range of the received light amount and the foreign matter position. The position of the foreign matter is predicted based on a calculation formula.
[0018]
In the foreign matter detection method according to the present invention, the table and the calculation formula of the foreign matter are determined based on the size of the foreign matter expected to be generated in advance.
[0019]
In the foreign matter detection method according to the present invention, the table and the calculation formula are created under the assumption that the sizes of the generated foreign matters are all the same.
[0020]
In the foreign object detection method according to the present invention, the foreign object position predicting step is performed by selecting from a plurality of position candidates where the foreign object may adhere.
[0021]
In the imaging device according to the present invention, the position of a foreign object existing between a pixel surface formed by arranging a plurality of pixels on the same surface and a lens that emits incident light to the pixel surface is determined. A foreign matter detection device for detecting, the received light amount sum calculating means for calculating the sum of received light amounts in a first pixel region comprising a plurality of pixels on the pixel surface, and detecting the presence of the foreign matter based on the calculation result Foreign matter detecting means.
[0022]
In the imaging device according to the present invention, an imaging device having a pixel surface formed by arranging a plurality of pixels on the same surface, photoelectrically converting light incident on the pixel surface and outputting the image signal as an image signal; Based on the stored contents of the lens that emits the incident light to the pixel surface of the image sensor, the correction method storage means that stores the correction method of the image signal, and the correction method storage means, Image signal correcting means for correcting the image signal to be output, and the correction method storage means is configured to output the image signal based on an output of a foreign matter detection device that detects a reduction range of the amount of received light due to the foreign matter on the pixel surface. Output of the correction method determining means for determining the correction method, and the foreign matter detection means receives the signal in each of a plurality of first pixel regions including the same pixel on the pixel surface and having a different number of pixels. Comprising a light quantity integration unit for calculating the sum of the amount, and a foreign object detection section which detects a reduction range of the light receiving amount in the pixel plane by the foreign object based on the calculation result.
[0023]
Furthermore, in the imaging apparatus according to the present invention, the correction method determining means and the foreign object detection means are provided in another device different from the imaging apparatus, and the imaging device outputs the imaging element. An output interface for outputting the image signal to the foreign object detection means provided in the other device as information on the amount of received light in each pixel of the image sensor, and the correction method output by the correction method determination means. And an input interface for inputting to the correction method storage means.
[0024]
DETAILED DESCRIPTION OF THE INVENTION
Embodiment 1 FIG.
FIG. 1 is a cross-sectional configuration diagram illustrating a camera unit of an imaging apparatus according to Embodiment 1 of the present invention. In the figure, 1 is an image sensor, 1a is a pixel surface formed by arranging a plurality of pixels on the image sensor 1, 2 is a lens that emits incident light to the pixel surface 1a, and 4 is the sensor sensitivity of each pixel. An IR (infrared) cut filter using glass as a base material for matching the image with the human visible region, 3 is a support housing that integrally supports the imaging element 1, the lens 2, and the IR cut filter 4. When the imaging device is built in the portable information terminal, the distance from the bottom surface 1b of the imaging element 1 to the top surface 2a of the lens 2 is, for example, 10 mm or less.
[0025]
2A to 2C are diagrams illustrating the state of incident light when a foreign object adheres to the camera unit 61 illustrated in FIG. FIG. 2A is a diagram illustrating the state of incident light when a foreign object 11 adheres to the pixel surface 1a. In the figure, 10a to 10c are pixels included in the pixel surface 1a, 11 is a foreign matter adhering to the pixel surface 1a, and 12 is a cross section (hereinafter referred to as a plane) at a foreign matter position of incident light to the pixel 10b. The amount of light received by the pixel 10b decreases by the ratio of foreign matter occupying the plane 12. That is, in FIG. 2A, since the entire area of the plane 12 is blocked by foreign matter, the amount of decrease in the amount of received light in the pixel 10b is approximately 100%.
[0026]
FIG. 2B illustrates the state of incident light when the foreign matter 11 is attached on the IR cut filter 4 shown in FIG. At this time, the amount of received light decreases not only in the pixel 10b but also in other pixels due to the foreign matter 11. On the other hand, since the ratio of the foreign material 11 occupying the plane 12 decreases, the amount of decrease in the amount of received light in the pixel 10b alone decreases.
[0027]
FIG. 2C is a diagram illustrating the state of incident light when a foreign object adheres to the lens 2 shown in FIG. At this time, the amount of received light decreases in more pixels due to the foreign matter 11. On the other hand, the amount of decrease in the amount of received light in the pixel 10b alone is further reduced than that shown in FIG.
[0028]
By the way, when a foreign substance adheres on the pixel surface 1a, the amount of received light is remarkably reduced in the pixel immediately below. Therefore, the presence of a foreign object can be detected by detecting such a significant decrease in the amount of received light. However, as shown in FIG. 2B and FIG. 2C, when the pixel surface is separated from the foreign matter, the amount of decrease in the amount of received light in each pixel is small, making it difficult to detect the foreign matter.
[0029]
In the first embodiment, the total amount (integration) of the received light amounts in the n × n pixel matrix region is calculated for the foreign matter present at a position away from the pixel surface as shown in FIGS. 2B and 2C. It detects by doing.
[0030]
FIG. 3 is a configuration diagram illustrating the foreign object detection device and the imaging device according to the first embodiment. In the figure, the same or corresponding parts as in FIG. In FIG. 3, 5 is a foreign matter detection device, 51 is an image memory for storing the amount of received light in all the pixels on the pixel surface 1a, 52 is a central pixel selection means for selecting an arbitrary pixel on the pixel surface 1a as a central pixel, 53 Is a pixel area selection means for selecting an n × n pixel area centered on the center pixel selected by the center pixel selection means 52 while sequentially increasing the value of n, and 54 is an n selected by the pixel area selection means 53 The received light amount integrating means for calculating the sum of the received light amounts in the xn pixel region, 55 is the total of the received light amounts in the n × n pixel region calculated by the received light amount integrating means 54, and there is no foreign matter between the lens and the pixel surface. In this case, the received light amount decrease amount calculating means calculates the difference between the total received light amounts in the n × n pixel region and the difference as the received light amount decrease amount in the n × n pixel region. It is assumed that the foreign matter detection device 5 stores in advance the amount of light received by each pixel when no foreign matter exists between the lens and the pixel surface. Reference numeral 56 denotes a received light amount decrease range detecting means for detecting the presence of a foreign substance and a reduction range of the received light amount due to the foreign substance based on the received light amount decrease amount in the n × n pixel area calculated by the received light amount decrease amount calculating means 55. This is an image correction method determination unit that determines a method of correcting the output image of the image sensor 1 based on the detection result of the received light amount decrease range detection unit 56.
[0031]
In the figure, reference numeral 6 denotes an image pickup apparatus, 62 denotes an output interface for outputting an image signal output from the image pickup device 1 to the image memory 51, and 63 denotes an image correction method determined by the image correction method determination means 57 to the image pickup apparatus 6. An input interface for input, 64 is an image correction method storage unit for storing an image correction method input via the input interface 63, and 65 is an image correction method stored in the image correction method storage unit 64 using the image correction method. This is an image signal correction means for correcting the image signal.
[0032]
Next, operations of the foreign object detection device 5 and the imaging device 6 will be described with reference to FIGS. 3 and 4. FIG. 4 is a flowchart illustrating the operations of the foreign object detection device 5 and the imaging device 6 illustrated in FIG. Here, it is assumed that the amount of received light (K is a real number) per pixel when no foreign matter exists between the pixel surface 1a and the lens 2. That is, it is assumed that the total amount of received light in the n × n pixel region (hereinafter referred to as the reference received light amount) is (K × n × n) when no foreign matter exists between the pixel surface 1a and the lens 2.
[0033]
First, the amount of light received at each pixel on the pixel surface 1 a is output from the image sensor 1 as an image signal. This image signal is input to the image memory 51 via the output interface 62 and stored (step 1 in FIG. 4). On the other hand, the center pixel selecting unit 52 sequentially selects all the pixels on the pixel surface 1a one by one as the center pixel (step 2). Hereinafter, the case where the center pixel selecting unit 52 selects the pixel 50 shown in FIG. 5 as the center pixel will be described as an example.
[0034]
First, the pixel region selection unit 53 selects a 1 × 1 pixel region centered on the center pixel 50 selected by the center pixel selection unit 52 as a first pixel region (step 3). The received light amount integrating means 54 calculates the sum of the received light amounts in the first pixel region selected by the received light amount integrating means 54 and outputs the calculation result to the received light amount decrease amount calculating means 55 (step 4). The received light amount decrease amount calculating means 55 calculates the difference between the total received light amount calculated by the received light amount integrating means 54 and the reference received light amount (K × 1 × 1), and the calculated result is detected as the received light amount decrease range. It outputs to the means 56 (step 5). Specifically, the received light amount decrease amount calculated by the arithmetic expression “received light amount decrease amount = reference received light amount (K × 1 × 1) −total received light amount in the first pixel region (1 × 1 pixel region)” is calculated. Output to the received light amount decrease range detecting means 56. When the received light amount decrease amount calculated by the received light amount decrease amount calculating means 55 substantially coincides with the predetermined amount M, the received light amount decrease range detecting unit 56 determines the received light amount as the first pixel region (1 × 1 pixel region). It detects as a fall range (step 6). That is, when the received light amount decrease amount in the first pixel region substantially coincides with the predetermined amount M, the received light amount decrease range detection means 56 detects the first pixel region as the received light amount decrease range due to the same foreign matter.
[0035]
Similarly, the pixel area selecting unit 53 designates a 3 × 3 pixel area centered on the central pixel 50 selected by the central pixel selecting unit 52 to the received light amount integrating unit 54 as a second pixel area ( Step 3). In response to this, the received light amount integrating means 54 calculates the sum of received light amounts in the second pixel region, and outputs the calculation result to the received light amount decrease amount calculating means 55 (step 4). The received light amount decrease amount calculating means 55 calculates the difference between the total received light amount calculated by the received light amount integrating means 54 and the reference received light amount (K × 3 × 3), and the calculated result is detected as the received light amount decrease range. It outputs to the means 56 (step 5). Specifically, the received light amount decrease amount calculated by the formula “received light amount decrease amount = reference received light amount (K × 3 × 3) −total received light amount in the second pixel region” is used as the received light amount decrease range detection unit 56. Output to. When the received light amount decrease amount calculated by the received light amount decrease amount calculating means 55 substantially coincides with the predetermined amount M, the received light amount decrease range detecting means 56 determines the received light amount as the second pixel area (3 × 3 pixel area). It detects as a fall range (step 6). That is, when the received light amount decrease amount in the second pixel region substantially coincides with the predetermined amount M, the received light amount decrease range detection means 56 detects the second pixel region as a received light amount decrease range due to the same foreign matter. Similarly, it is checked whether or not the 5 × 5 pixel area centered on the center pixel 50 is within the reduction range of the amount of received light due to the same foreign substance (steps 1 to 6).
[0036]
After the processes in steps 1 to 6 as described above are repeated for all the pixels on the screen 1a, the image correction method determining unit 57 is based on the received light amount decrease range detected by the received light amount decrease range detecting unit 56. Then, a correction method suitable for the output image of the image sensor 1 is determined. For example, a method for correcting an image of a pixel portion where the amount of received light is reduced is determined based on an image of a pixel portion around which the amount of received light is not reduced.
[0037]
The image correction method determined by the image correction method determination means 57 is stored in the image correction method storage means 64 via the input interface 63, and the imaging device 6 is then disconnected from the foreign object detection device 5. Thereafter, the image signal output from the image sensor 1 is input to the image signal correction unit 65, and the image signal correction unit 65 corrects the input image signal based on the stored contents of the image correction method storage unit 64. The image signal corrected in this way is output to a subsequent circuit (not shown) such as a display unit of a mobile terminal, a storage device of the mobile terminal, a wireless transmission unit of the mobile terminal, and the like.
[0038]
As described above, in the first embodiment, the presence of a foreign object is detected by calculating the total amount of received light in a pixel area composed of a plurality of pixels. The presence of can be detected. In particular, the presence of the foreign matter can be detected even when the foreign matter is present at a position away from the pixel surface.
[0039]
In the first embodiment, the total amount of received light in the pixel region composed of a plurality of pixels is compared with the total amount of received light in the pixel region when no foreign matter exists between the pixel surface and the lens. Since a decrease in the amount of received light due to the foreign matter in the pixel area is detected based on the comparison result, the foreign matter away from the pixel surface can be detected with a simple configuration.
[0040]
Further, in the first embodiment, the reduction range of the received light amount due to the same foreign matter is detected by comparing the reduction amount of the received light amount in the pixel region with the predetermined amount, and therefore the reduction range of the received light amount by a simple method. Can be detected.
[0041]
Here, an n × n pixel region (n = 1, 2, 3,...) Is exemplified as a pixel region selected by the pixel region selecting unit 53, but the present invention is not limited to this, and an arbitrary shape may be used. . Further, here, the processing of steps 1 to 6 is repeated for all the pixels on the screen 1a, but steps 1 to 6 may be performed only for some of the pixels on the screen 1a. In addition, here, the reference light reception amount is calculated by setting the light reception amount per pixel when there is no foreign object to the same value K. For example, as shown in FIG. The reference amount of received light may be calculated assuming that is small.
[0042]
Embodiment 2. FIG.
In the first embodiment, the decrease range of the amount of received light due to the foreign matter is detected by comparison with the reference amount of received light. In the second embodiment, there is provided a foreign object detection method capable of detecting a decrease range of received light amount without using the reference received light amount.
[0043]
FIG. 7 is a configuration diagram illustrating a foreign object detection device and an imaging device according to the second embodiment. In the figure, the same or corresponding parts as in FIG. In FIG. 7, reference numeral 54 denotes a selected area received light amount integrating means for calculating the sum of received light quantities in a pixel area selected by the pixel area selecting means 53 (hereinafter referred to as a selected pixel area), and 58 denotes a periphery of the selected pixel area. Peripheral region received light amount integrating means 59 for calculating the sum of received light amounts in a pixel region (hereinafter referred to as peripheral pixel region) composed of one or a plurality of existing pixels, 59 is a selection region received light amount integrating means and peripheral region received light amount Based on the calculation result of the integration unit 58, the received light amount change rate calculating unit calculates the change rate of the received light amount between the selected pixel region and the peripheral pixel region for each of a plurality of selected pixel regions having different numbers of pixels.
[0044]
For example, as shown in FIG. 8, the received light amount change rate calculating means 59 calculates the change rate between the peripheral pixel regions 584 to 586 for each of the selected pixel regions 581 to 583 centered on the same pixel. The rate of change can be obtained by standardizing and comparing the total amount of received light in the selected pixel region and the total amount of received light in the peripheral pixel region to the same amount of received light. Then, the received light amount decrease range detecting means 60 detects a selected pixel region having the largest change rate among the change rates calculated around these same pixels as the received light amount decrease range.
[0045]
The foreign object detection method as described above is based on the assumption that the received light amount changes abruptly at the boundary in the range where the received light amount is reduced due to the same foreign object.
[0046]
As described above, in the second embodiment, the rate of change in the amount of received light between the selected pixel region and the peripheral pixel region existing around the selected pixel region is set to a plurality of pixels with different numbers of pixels centered on the same pixel. Since the calculation is performed for each of the selected pixel areas and the reduction range of the received light amount is detected based on the calculated rate of change, the received light amount even if the reference received light amount (the received light amount that should be received when there is no foreign object) is unknown Can be detected.
[0047]
Embodiment 3 FIG.
In the first and second embodiments, the method for detecting the decrease range of the amount of received light has been described. In the third embodiment, a method for predicting the position of a foreign object based on the detected decrease range of the received light amount will be described.
[0048]
FIG. 9 is a configuration diagram illustrating a foreign object detection device and an imaging device according to the third embodiment. In the figure, the same or corresponding parts as in FIG. In FIG. 9, reference numeral 91 denotes a foreign object position predicting unit that predicts a foreign object position based on the detection result of the received light amount decrease range detecting unit 56. The foreign object position predicting unit 91 includes a foreign object position reference table 92 that associates the reduction range of the received light amount with the foreign object position.
[0049]
Hereinafter, for simplification of explanation, it is assumed that the foreign matters generated in the imaging apparatus have substantially the same size, and within the range of one pixel when foreign matters are attached on the pixel surface 1a in FIG. It is assumed that the amount of received light decreases by M (M is a real number). Further, when foreign matter adheres to the lower surface 43 of the IR cut filter 4, the total amount of received light decreases by M within the 2 × 2 pixel range, and when foreign matter adheres to the lower surface 22 of the lens 2. Suppose that the total amount of received light decreases by M within the range of 3 × 3 pixels.
[0050]
The foreign object position reference table 92 associates the 1 × 1 pixel range with the pixel surface 1a, associates the 2 × 2 pixel range with the lower surface 43 of the IR cut filter 4, and associates the 3 × 3 pixel range with the lens. 2 on the lower surface 22 is associated.
[0051]
Next, the operation will be described. First, the received light amount decrease range detecting unit 56 outputs the detected decrease range of received light amount to the foreign object position predicting unit 91. If the detection result is such that the detection result is a 1 × 1 pixel range centered on the center pixel 50 and the received light amount is reduced, the position of the foreign object is determined based on the stored contents of the table 92. It is predicted to be directly above 50 and on the pixel surface 1a. Also, if the detection result of the foreign object position predicting unit 91 is such that the 2 × 2 pixel range centered on the central pixel 50 is the reduction range of the received light amount, the position of the foreign object is directly above the central pixel 50 and IR cut. Predicted to be on the lower surface 43 of the filter 4. If the detection result of the foreign object position predicting unit 91 is such that the 3 × 3 pixel range centering on the central pixel 50 is the reduction range of the received light amount, the foreign object position is directly above the central pixel 50 and the lens 2. Is predicted to be on the lower surface 22.
[0052]
As described above, according to the third embodiment, by using a table that associates the reduction range of the received light amount with the position of the foreign matter from the pixel surface, the pixel of the foreign matter is based on the detected reduction range of the received light amount. The position from the surface can be predicted. In place of the table, the foreign object position may be predicted using an arithmetic expression that associates the reduction range of the received light amount with the foreign object position.
[0053]
Embodiment 4 FIG.
In the fourth embodiment, a specific example of the foreign object position detection method exemplified in the second embodiment will be shown. Here, for simplification of explanation, as shown in FIG. 10, the distance between the pixel surface 1a and the lower surface 43 of the IR cut filter 4 is 0.2 mm, the thickness of the IR cut filter 4 is 0.3 mm, and the lens. 2 is a 1.0 mm square, and the distance between the pixel surface 1a and the lens 2, that is, the focal length f is 2.0 mm. The shape of each pixel is a square shape with a side of 20 μm, and the shape of a foreign object is also a similar 20 μm square.
[0054]
Under such conditions, when a foreign object is in contact with the pixel surface 1a, the amount of received light is almost zero in one pixel immediately below it, and when the foreign object exists at a distance of 0.2 mm from the pixel surface, the foreign object When the amount of light received by 4% of each pixel falls within the range of 5 × 5 pixels existing immediately below and a foreign object exists at a position of 0.12 mm from the pixel surface, each pixel 11 within the range of 3 × 3 pixels .1% of light received decreases.
[0055]
FIG. 11 illustrates the amount of light received by each pixel in a 7 × 7 pixel centered on the pixel 50 under such conditions. FIG. 7A shows a case where the foreign matter 11 exists on the pixel surface 1a, and FIG. 7B shows a case where the foreign matter 11 exists at a distance of 0.12 mm from the pixel surface 1a. FIG. 7C shows a case where the foreign material 11 is present at a distance of 0.2 mm from the pixel 1a. In addition, the amount of light received at each pixel in the figure is 100 when the amount of light received by a foreign object is not reduced.
[0056]
In the fourth embodiment, the reduction rate of the received light amount in the N × N pixel region is quantified by calculating the determination value γ shown in the following equation (1) based on the received light amount in each pixel. To do.
γ_{Nx N} = (Α_{N × N} / Β_{N × N} ) × 100 Formula (1)
Where γ_{Nx N} Is the reduction rate of the amount of received light in the N × N pixel region, α_{N × N}Is the average amount of light received per pixel in the N × N pixel area, β_{N × N}Is the average amount of light received per pixel around the N × N pixel area. In the following explanation, this β_{N × N}Is the average amount of light received per pixel in the pixel region obtained by removing the N × N pixel region from the (N + 2) × (N + 2) pixel region surrounding the N × N pixel region. This judgment value γ_{Nx N}It can be determined that the change rate of the amount of received light between the inside of the N × N pixel region and the outside of the pixel region is larger as is smaller.
[0057]
FIG. 12 shows the above-described α for the amount of light received shown in FIG._{N × N} , Β_{N × N} , Γ_{Nx N}Is the result of computing. According to this, when the foreign object is on the pixel surface 1a, the determination value γ is the lowest (the rate of change is maximum) in the 1 × 1 pixel region, and when the foreign object is at a distance of 0.12 mm from the pixel surface 1a. When the determination value γ is the lowest in the 3 × 3 pixel region (the rate of change is maximum) and the foreign object is at a distance of 0.2 mm from the pixel surface 1a, the determination value γ is the lowest in the 5 × 5 pixel region (the change rate is the maximum). )
[0058]
As described above, the determination value γ is calculated for each of a plurality of N × N pixel regions centered on the same pixel, and the pixel region in which the smallest γ is calculated among the calculated γ is reduced. If it detects as a range, the fall range of the light reception amount by a foreign material can be detected. For example, when the foreign substance is on the pixel surface, the determination value γ is calculated for each case of 1 × 1, 3 × 3, 5 × 5, 7 × 7, and the γ shown in FIG._1x1= 0, γ calculated in the 1 × 1 pixel region is the smallest value, and it is estimated that foreign matter is adhered on the pixel surface 1a. Similarly, when the foreign object is at a distance of 0.12 mm from the pixel surface, the determination value γ calculated for the 3 × 3 pixel region is the smallest, and the result indicates that the foreign object is at a distance of 0.12 mm from the pixel surface. It is estimated that there is.
[0059]
The farther the position of the foreign object is from the pixel surface, the smaller the change amount of the determination value γ is compared to the increase of N in the N × N pixel region. This is because the position of the foreign object is further away from the pixel surface. It means that it becomes difficult to detect. However, the further the position of the foreign object is from the pixel surface, the less the image deterioration due to the foreign object, so the necessity of inspection is reduced.
[0060]
【The invention's effect】
Since the present invention is configured as described above, the following effects can be obtained. The foreign matter detection method according to the present invention detects foreign matter that exists between a pixel surface in which a plurality of pixels are arranged on the same surface and a lens that emits incident light to the pixel surface. A method for integrating a received light amount in a first pixel region composed of a plurality of pixels on the pixel surface; and a foreign object detecting step for detecting the presence of the foreign material based on a result of the calculation. Therefore, foreign matter can be detected even with a non-focusing fixed lens, and the detection can be performed in a short time. Further, even when the foreign object is present at a position away from the pixel surface, the presence of the foreign object can be detected.
[0061]
Further, the foreign matter detection method according to the present invention detects foreign matter existing between a pixel surface formed by arranging a plurality of pixels on the same surface and a lens that emits incident light to the pixel surface. A foreign matter detection method, comprising: a received light amount integration step of calculating a sum of received light amounts in each of a plurality of first pixel regions having the same pixel on the pixel surface and having a different number of pixels, and the foreign matter based on the calculation result And a foreign matter detection step for detecting a reduction range of the received light amount on the pixel surface, so that not only the presence or absence of the foreign matter but also a reduction range of the received light amount on the pixel surface due to the foreign matter can be detected.
[0062]
In the foreign matter detection method according to the present invention, the foreign matter position predicting step may correspond to a known table associating the reduction range of the received light amount with the foreign matter position or the reduction range of the received light amount and the foreign matter position. Since the position of the foreign matter is predicted based on the arithmetic expression to be attached, the distance of the foreign matter from the pixel surface can be predicted.
[Brief description of the drawings]
FIG. 1 is a configuration diagram illustrating a configuration of a camera unit of a general imaging apparatus.
FIG. 2 is an explanatory diagram illustrating the state of incident light incident on a pixel surface.
3 is a configuration diagram illustrating a foreign object detection device and an imaging device according to Embodiment 1. FIG.
4 is a flowchart illustrating a foreign object detection method according to Embodiment 1. FIG.
FIG. 5 is an explanatory diagram illustrating a method for selecting N × N pixels of the foreign object detection device according to the first embodiment;
FIG. 6 is a diagram illustrating the amount of received light received on a pixel surface when no foreign matter is present.
FIG. 7 is a configuration diagram illustrating a foreign object detection device and an imaging device according to a second embodiment.
FIG. 8 is an explanatory diagram illustrating a peripheral pixel region and a selected pixel region.
9 is a configuration diagram illustrating a foreign object detection device and an imaging device according to Embodiment 3. FIG.
10 is a diagram illustrating dimensions of a lens portion for explanation in the third embodiment. FIG.
FIG. 11 is a diagram illustrating the amount of received light in a 7 × 7 pixel region for explanation in the third embodiment.
12 is a diagram illustrating received light amount determination values for explanation in the third embodiment. FIG.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 5 Foreign substance detection apparatus, 6 Imaging device, 51 Image memory, 52 Center pixel selection means, 53 Pixel area selection means, 54 Light reception amount integration means, 55 Light reception amount fall amount calculation means, 56 Light reception amount fall range detection means, 57 Image correction Method determining means, 58 peripheral area received light amount integrating means, 59 received light quantity change rate calculating means, 60 received light quantity change range detecting means, 61 camera unit, 62 output interface, 63 input interface, 64 image correction method storage means, 65 image signal Correction means, 91 Foreign object position prediction means, 92 table.

Claims (19)

A foreign matter detection method for detecting foreign matter existing between a pixel surface in which a plurality of pixels are arranged on the same surface and a lens that emits incident light to the pixel surface,
A received light amount integrating step of calculating a sum of received light amounts in a first pixel region composed of a plurality of pixels on the pixel surface;
A foreign matter detection method comprising: a foreign matter detection step for detecting the presence of the foreign matter based on the calculation result. The foreign matter detection step compares the total amount of received light in the first pixel region with a predetermined amount of received light, and detects a decrease in the amount of received light due to the foreign matter in the first pixel region based on the comparison result. The foreign object detection method according to claim 1, wherein: The predetermined amount of received light in the foreign object detection step is greater when the first pixel region is located at the edge of the pixel surface than when the first pixel region is located at the center of the pixel surface. The foreign matter detection method according to claim 2, wherein the foreign matter is small. The foreign matter detection step compares the total amount of received light in the first pixel region with the total amount of received light in the first pixel region when no foreign matter exists between the pixel surface and the lens, The foreign object detection method according to claim 1, wherein a decrease in the amount of received light due to the foreign substance in the first pixel region is detected based on the comparison result. In the foreign matter detection step, the received light amount in the first pixel region and the second pixel region including one or a plurality of pixels existing around the first pixel region are standardized to the received light amount per the same area. 2. The foreign object detection method according to claim 1, wherein a decrease in the amount of light received by the foreign object in the first pixel region is detected based on the comparison result. A foreign matter detection method for detecting foreign matter existing between a pixel surface in which a plurality of pixels are arranged on the same surface and a lens that emits incident light to the pixel surface,
A received light amount integrating step for calculating a sum of received light amounts in each of a plurality of first pixel regions including the same pixel on the pixel surface and having a different number of pixels;
A foreign matter detection method comprising: a foreign matter detection step of detecting a reduction range of the amount of received light on the pixel surface by the foreign matter based on the calculation result. The foreign matter detection step compares the total amount of received light in the first pixel region with the total amount of received light in the first pixel region when no foreign matter exists between the pixel surface and the lens, The foreign object detection method according to claim 6, wherein the first pixel area in which the difference is a predetermined value is detected as a reduction range of the amount of received light on the pixel surface due to the foreign object. In the foreign object detection step, the rate of change in the amount of received light between the first pixel region and a second pixel region including one or a plurality of pixels existing around the first pixel region is set to The first pixel area where the rate of change is the largest is detected as a reduction range of the amount of received light on the pixel surface due to the foreign matter. Item 7. A foreign matter detection method according to Item 6. The calculation of the rate of change is performed by normalizing the total amount of received light in the first pixel region and the total amount of received light in the second pixel region to the amount of received light per the same area, The foreign object detection method according to claim 8, wherein the foreign object detection method is performed by calculating a ratio. The foreign object detection method according to claim 6, further comprising an image correction step for performing a predetermined correction on an output from a pixel existing in a reduction range of the received light amount. The image correction step of correcting an output from a pixel that exists within the reduction range of the received light amount based on an output of a pixel that exists outside the reduction range of the received light amount. The foreign object detection method according to any one of 10. The foreign substance position prediction step of predicting the position of the foreign substance in a direction perpendicular to the pixel surface based on the reduction range of the received light amount. Foreign object detection method. The foreign object position prediction step predicts the position of the foreign object based on a known table that associates the decrease range of the received light amount with the foreign object position or an arithmetic expression that associates the decrease range of the received light amount with the foreign object position. The foreign object detection method according to claim 12, wherein: The foreign object detection method according to claim 13, wherein the table and the calculation formula of the foreign object are determined based on a size of the foreign object expected to be generated in advance. 15. The foreign object detection method according to claim 14, wherein the table and the arithmetic expression are created under the assumption that all the generated foreign substances have the same size. 16. The foreign object detection method according to claim 12, wherein the foreign object position predicting step is performed by selecting from a plurality of position candidates where a foreign object may be attached. A foreign matter detection device that detects a position of a foreign matter existing between a pixel surface formed by arranging a plurality of pixels on the same surface and a lens that emits incident light to the pixel surface,
A received light amount sum calculating means for calculating a sum of received light amounts in a first pixel region comprising a plurality of pixels on the pixel surface;
A foreign matter detection device comprising foreign matter detection means for detecting the presence of the foreign matter based on the calculation result. An image sensor that has a pixel surface in which a plurality of pixels are arranged on the same surface, photoelectrically converts light incident on the pixel surface, and outputs the image signal;
A lens that emits incident light to the pixel surface of the image sensor;
Correction method storage means for storing the correction method of the image signal;
Image signal correction means for correcting the image signal output by the imaging device based on the stored contents of the correction method storage means;
The correction method storage unit stores an output of a correction method determination unit that determines a correction method of the image signal based on an output of a foreign matter detection device that detects a reduction range of the amount of received light due to the foreign matter on the pixel surface. And
The foreign matter detecting means includes a received light amount integrating unit that calculates a sum of received light amounts in each of a plurality of first pixel regions including the same pixel on the pixel surface and having a different number of pixels, and the pixel due to the foreign matter based on the calculation result An imaging apparatus comprising: a foreign matter detection unit that detects a reduction range of the amount of received light on the surface. The correction method determining means and the foreign object detection means are provided in another device different from the imaging device,
The image pickup apparatus outputs the image signal output from the image pickup element as information on the amount of received light in each pixel of the image pickup element to the foreign object detection means provided in the other device; and 19. The imaging apparatus according to claim 18, further comprising an input interface for inputting the correction method output from the correction method determination unit to the correction method storage unit.

Images