JP2009017158A - Camera inspection device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a camera inspection device which is capable of precisely detecting a stain in image data by reducing an influence of luminance shading that an optical system of a camera has. <P>SOLUTION: The camera inspection device 1000 includes an image data acquisition part 110 for acquiring image data obtained by imaging a reference subject, a double differential value calculation part 140 for calculating a double differential value of luminance along a prescribed direction from the image data, and a stain position detection part 161 for detecting a position of a stain in the image data from the calculation result of the double differential value calculation part 140, and detects the stain in an image caused by a foreign matter stuck to the light receiving surface of an imaging device of the camera or its peripheral part. The double differential value calculation part 140 varies a differential interval at the time of calculating the double differential values according to a position along the prescribed direction on the basis of luminance shading characteristics causing a luminance distribution in the picked-up image of the reference subject. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a camera inspection device that detects a stain caused by foreign matter such as dust and dust attached to a light receiving surface of an image sensor or its peripheral part based on image data captured by a camera having the image sensor. The camera is typified by a camera mounted on a mobile device such as a mobile phone.

  In recent years, cameras having an image sensor are mounted on various electronic devices. In a camera having an image sensor, dust and dust may adhere to the image sensor and the lens unit as foreign matters during the manufacturing process, and the foreign matters may appear as spots in the captured image of the camera. FIG. 18 shows an example of a captured image in which a stain is reflected. In the example of FIG. 18, a spot 1801 appears on the lower left side of the captured image 1802, and the image quality of the captured image is degraded due to the presence of the spot 1801.

  In cameras mounted on mobile devices such as mobile phones, the pitch per pixel is decreasing with the demand for higher resolution from the market, and with the request for thinner mobile device bodies. The distance between the light receiving surface of the image sensor and the lens portion is becoming shorter. For this reason, in a camera for a portable device, there is a case where a stain appears in a captured image simply because finer dust and dust adhere to the imaging element and the lens unit.

  Conventionally, an inspection relating to a spot in a captured image of a camera has been performed by human visual judgment. However, the stains that appear in the captured image are weak, and in the sensitive evaluation by human visual inspection, the judgment criteria differ depending on the inspector, and this difference causes variations in the inspection results and affects the manufacturing yield. Was exerting. Under the current situation in which the number of mass-produced portable devices has increased remarkably, it is desirable to detect stains in captured images efficiently and with high accuracy without relying on visual inspection.

As an inspection technique not based on visual inspection, for example, a camera inspection technique described in Patent Document 1 is known. In the camera inspection technique described in Patent Document 1, a differential value is calculated twice at each position on a luminance curve along a predetermined direction from image data captured by a camera. The luminance curve is a curve representing the luminance change along the predetermined direction, and is specifically expressed by the luminance value of each pixel on the line along the predetermined direction. Then, utilizing the fact that the second derivative value represents the shape of the luminance curve, the position of the spot is detected based on the calculation result of the second derivative value, and the quality of the camera is determined based on the detected spot density It is carried out.
JP 2006-191231 A

  However, the conventional camera inspection technology has a problem in that the accuracy of spot detection is lowered because the luminance shading characteristics of the optical system of the camera are not taken into consideration. Luminance shading is a phenomenon that depends on an optical system such as a camera lens unit and an image sensor, and is a phenomenon in which a luminance difference occurs between the central part and the peripheral part of a captured image of the camera. According to the luminance shading characteristic, the luminance at the peripheral portion of the captured image of the camera may suddenly decrease compared to the central portion. Further, for example, as shown in FIG. 19, not only a luminance difference occurs between the peripheral portion and the central portion of the captured image of the camera, but also a concave dark portion may occur in the peripheral portion. If luminance shading or large luminance changes occur due to luminance shading, these may be erroneously detected as spots.

  Particularly in cameras for portable devices, the thickness of optical parts such as image sensors and lens parts has become extremely thin as the thickness of portable equipment has been reduced, and the brightness shading characteristics of the optical system of the camera have become stains. It tends to greatly affect the detection accuracy. Therefore, it is desirable to detect a spot with high accuracy without being affected by luminance shading.

  The present invention has been made in order to solve the above-described conventional problems, and provides a camera inspection apparatus capable of accurately detecting a stain in image data by reducing the influence of luminance shading of a camera optical system. The purpose is to provide.

  The camera inspection apparatus according to the present invention detects a stain in an image caused by a foreign matter attached to a light receiving surface of the image sensor or its peripheral portion based on image data captured by a camera having an image sensor. An image data acquisition unit that acquires image data obtained by imaging a reference subject using the camera, and a second derivative that calculates a second derivative value of luminance along a predetermined direction from the image data. A value calculation unit; and a position detection unit that detects a position of a spot in the image data based on a calculation result of the two-time differential value calculation unit, wherein the two-time differential value calculation unit is an optical unit of the camera. Based on a luminance shading characteristic that is a characteristic of the system and produces a luminance distribution in the captured image of the reference subject, a differential interval for calculating the second differential value along the predetermined direction Has a structure made different depending on the location.

  With this configuration, a twice differential value of luminance along a predetermined direction is calculated from the image data, and the position of a spot in the image data is detected based on the twice differential value. In this spot position detection, the position where the twice-differentiated value becomes a large negative value (a negative value with a large absolute value) is specified as the end of the spot. In the present invention, especially when calculating the differential value of the luminance twice for spot position detection, the differential interval is set to a position along a predetermined direction based on the luminance shading characteristic that produces a luminance distribution in the captured image of the reference subject. Different depending on the situation. With this configuration, it is possible to prevent erroneous detection of a luminance change in luminance shading as a stain by appropriately changing the differential interval in accordance with the gradual change in luminance due to luminance shading. As a result, it is possible to reduce the influence of luminance shading of the optical system of the camera and to accurately detect spots in the image data.

  In the camera inspection apparatus of the present invention, the second differential value calculation unit sets a differential interval at a peripheral portion of the image data smaller than a differential interval at a central portion of the image data. It has a configuration.

  With this configuration, the differential value at the peripheral edge outside the central part is set smaller than the differential distance at the central part of the image data, and the twice differential value of luminance is calculated from the image data. This configuration is suitable for the inspection of a camera equipped with an optical system having luminance shading characteristics such that the luminance at the peripheral portion of the image data is sharply reduced compared to the central portion. In the central part of the image data, by setting the differential interval larger than that of the peripheral part, the double differential value at the stain end increases in the negative direction, and the stain end becomes a feature point at the double differential value. Remarkably appear, and the position of the spot can be easily detected. On the other hand, by setting the differential interval smaller at the periphery of the image data than at the center, even if the stain is at the edge of the image, the twice differential value becomes a large negative value, and the feature point is reliably In addition, it is possible to prevent the differential value from becoming a large negative value twice at the point of sudden change in luminance due to luminance shading, and to clearly distinguish between the stain and the luminance change due to luminance shading. In this way, by setting the differential interval at the peripheral portion to be smaller than the differential interval at the center portion of the image data, the influence of the luminance shading of the camera optical system can be reduced. Can be detected with high accuracy.

  Further, in the camera inspection apparatus of the present invention, the position detection unit has a configuration for detecting the position of the stain in the image data by comparing the twice differential value with a predetermined stain end determination threshold value. ing. As a result, the position where the twice-differentiated value becomes a large negative value can be specified as a feature point with reference to a predetermined spot edge determination threshold, so that the position of the spot in the image data can be easily detected.

  Further, in the camera inspection apparatus of the present invention, the position detection unit sets the position where the double differential value is equal to or greater than the predetermined stain end threshold value as the start position of the stain, and then the predetermined stain end. The position of the spot is detected by setting a position that is equal to or greater than a threshold value as an end position of the spot. As a result, based on a predetermined spot edge determination threshold, the position where the twice differential value becomes a negative negative value first in the image data is set as the spot start position, and the double differential value becomes a negative negative value later. Since the position can be specified as the end position of the stain, the position of the stain in the image data can be easily detected.

  The camera inspection program of the present invention performs a process of detecting a stain in an image caused by a foreign matter attached to the light receiving surface of the image sensor or its peripheral part based on image data captured by a camera having an image sensor. A camera inspection program to be executed by a computer, the step of acquiring image data obtained by imaging a reference subject using the camera, and calculating a twice differential value of luminance along a predetermined direction from the image data And a step of causing the computer to execute a step of detecting a position of a stain in the image data based on a calculation result of the double differential value calculation unit, and calculating a double differential value of the luminance. , A characteristic of the optical system of the camera, and a luminance shading characteristic that generates a luminance distribution in the captured image of the reference subject Zui and has a configuration in which different differential distance when calculating the second derivative values depending on the position along said predetermined direction. This configuration also provides the advantages of the present invention described above.

  Another aspect of the present invention is a computer-readable recording medium that records the above-described camera inspection program.

  Further, the camera inspection method of the present invention detects a stain in an image caused by a foreign matter attached to the light receiving surface of the image sensor or its peripheral part based on image data captured by a camera having an image sensor. A camera inspection method, the step of acquiring image data obtained by imaging a reference subject using the camera, the step of calculating a twice differential value of luminance along a predetermined direction from the image data, Detecting a position of a spot in the image data based on a calculation result of the double differential value calculation unit, and calculating the double differential value of the brightness in the optical system of the camera A differential interval for calculating the second differential value based on a luminance shading characteristic that produces a luminance distribution in a captured image of the reference subject. It has a configuration in which different depending on the position along the direction. The advantages of the present invention described above can also be obtained by this camera inspection method.

  In the present invention, when calculating a differential value of luminance twice in order to detect a stain in image data, the luminance interval of the optical system of the camera is obtained by varying the differential interval depending on the location based on the luminance shading characteristics. It is possible to provide a camera inspection apparatus having an effect of reducing the influence of shading and detecting a stain in image data with high accuracy.

  Hereinafter, a camera inspection apparatus according to an embodiment of the present invention will be described with reference to the drawings. A functional block diagram of the camera inspection apparatus according to the embodiment of the present invention is shown in FIG. 1, and a configuration of the camera inspection apparatus viewed from the hardware side is shown in FIG. First, the configuration of the camera inspection apparatus will be described from the hardware aspect with reference to FIG. 2, and then each function of the camera inspection apparatus will be described based on FIG.

  The camera inspection apparatus 1000 is an apparatus for detecting a spot in a captured image of a camera and inspecting the camera. The camera is a camera module mounted on a mobile device such as a mobile phone, for example, and includes a lens unit and an image sensor in an outer casing. The lens unit images light from the subject on the image sensor, and the image sensor photoelectrically converts the image formed by the lens unit to generate image data of the captured image.

  In the process of manufacturing a camera, if foreign matter such as dust or dust adheres to the light receiving surface of the image sensor or its peripheral part (including the lens part), it may appear as a spot in the image data captured by the camera. . The camera inspection apparatus 1000 inspects the camera by detecting a stain in the image captured by the camera based on image data captured by the camera and determining whether the stain level is acceptable.

  The camera inspection device 1000 is configured by a computer device, and includes a computer main body 210, a display unit 220, and an operation unit 230, as shown in FIG. The computer main body 210 includes a program execution unit 211, a communication unit 212, a memory unit 213, and a program storage unit 214.

  The program execution unit 211 includes a processor such as a CPU, and executes a spot inspection program 214 a stored in the program storage unit 214. The communication unit 212 is connected to an external device via a communication cable such as a USB cable, and can send and receive data to and from the external device. For example, the external device is a data storage device, and image data captured by a camera in advance is stored in the data storage device. Then, image data can be taken into the computer main body 210 from the data storage device via the communication unit 212. The external device may be a portable device equipped with a camera. In this case, image data captured in advance by the camera and stored in the storage unit of the portable device can be taken into the computer main body 210. A camera may be directly connected to the camera inspection apparatus 1000 and a captured image may be captured from the camera.

  The memory unit 213 is a working memory used when the program execution unit 211 performs processing such as executing a spot inspection program. The program storage unit 214 stores a spot inspection program 214a. The spot inspection program 214a is a program for realizing the camera inspection apparatus and the camera inspection method according to the present embodiment, and in particular, based on the image data captured by the camera, the second derivative of the luminance value of the image data. The position of the spot is detected from the value, a spot level representing the degree of the detected spot is calculated, and the camera is inspected by performing pass / fail judgment on the spot level. By executing the spot inspection program 214a, the camera detection apparatus and the camera detection method in the present embodiment are realized. The contents of the spot inspection program 214a will be described in a camera inspection apparatus and a camera inspection method which will be described in detail later.

  The display unit 220 is a display, and the display unit 220 displays processing information of the inspection program 214a. The operation unit 230 is a keyboard or the like on which numeric keys and the like are arranged, and is used, for example, for inputting an instruction regarding execution of the inspection program 214a.

  Next, the configuration of the camera inspection apparatus will be further described with reference to FIG. In FIG. 1, the camera inspection apparatus 1000 is represented by functional blocks. Each configuration of FIG. 1 is realized by the computer apparatus of FIG. 2, specifically, when the program execution unit 211 executes the spot detection program 214 a of the program storage unit 214. The camera inspection apparatus 1000 includes an image data acquisition unit 110, a luminance value calculation unit 120, a luminance value averaging unit 130, a twice differential value calculation unit 140, a differential interval storage unit 150, a spot detection unit 160, A stain determination threshold value storage unit 170, a pass / fail determination unit 180, and a determination result output unit 190 are provided.

  The image data acquisition unit 110 acquires image data obtained by imaging a reference subject using a camera. The image data acquired by the image data acquisition unit 110 is raw data output from the image sensor (camera) and is not subjected to various image processing such as white balance adjustment processing and wavelet transform processing. The reference subject is an achromatic subject such as white or gray and has a uniform color throughout, and a panel or the like is used. If foreign matter such as dust or dust adheres to the light receiving surface of the image sensor of the camera or its peripheral part (including the lens part), the foreign matter appears as a shadow on the captured image. The program execution unit 211 and the communication unit 212 in FIG. 2 function as the image data acquisition unit 110 described above.

The luminance value calculation unit 120 calculates the luminance value of each pixel of the image data acquired by the image data acquisition unit 110. If the pixel values of the red (R) signal, the green (G) signal, and the blue (B) signal, which are the three primary color image signals, are R, G, and B, respectively, the luminance value Y is expressed as in Expression (1). . The luminance value calculation unit 120 calculates the luminance value by converting the RGB pixel value into an 8-bit grayscale luminance signal according to the equation (1). The program execution unit 211 in FIG. 2 functions as the luminance value calculation unit 120 described above.
Y = 0.299 × R + 0.587 × G + 0.114 × B (1)

  The luminance value averaging unit 130 performs an averaging process on the luminance value Y of each pixel calculated by the luminance calculating unit 120. Here, even if a reference subject with smooth illuminance is imaged, the image data of the camera includes fluctuations due to electrical noise. The averaging process reduces or eliminates such electrical noise. The program execution unit 211 in FIG. 2 functions as the luminance value averaging unit 130 described above.

  In the averaging process, for example, the following simple averaging is performed. That is, the luminance value averaging unit 130 calculates an average value of luminance values in a predetermined range including the target pixel (for example, a total of 49 pixels of 7 × 7 pixels), and replaces the luminance value of the target pixel with the average value. This process is performed for all pixels in the image data. By repeating this simple averaging process, for example, three times, a luminance change caused by a stain is retained, and electrical noise is eliminated.

  FIG. 3 shows the luminance value averaging process. 3 (a) and 3 (b), the horizontal axis represents the position of the pixel on the image data, the vertical axis represents the luminance value, and the luminance curve in each figure represents each pixel arranged in a predetermined direction of the image data. The change of the luminance value is shown. FIG. 3A is a diagram showing a luminance curve before the averaging process, and FIG. 3B is a diagram showing a luminance curve after the averaging process. As shown in FIG. 3A, the luminance curve before the averaging process has many luminance fluctuations due to electrical noise. On the other hand, as shown in FIG. 3B, the electric noise is removed from the luminance curve after the averaging processing, and the luminance fluctuation is reduced.

  The averaging process of the luminance value of each pixel is not limited to the above method. It is necessary to optimize the averaging method according to the size and level of the stain to be detected. Further, the amount of noise to be reduced or removed varies depending on the specifications of the camera, but it is necessary to optimize the averaging processing method in accordance with the amount of noise to be reduced or removed. Therefore, the luminance value averaging unit 130 is configured to perform processing that satisfies these requirements.

  The twice differential value calculation unit 140 calculates a double differential value of luminance along a predetermined direction from the image data. The twice-differential value calculation unit 140 calculates the above-described twice-differential value using the image data after the luminance value averaging processing by the luminance value averaging unit 130. The twice differential value calculation unit 140 selects one line in the horizontal direction or the vertical direction of the image data, and calculates a double differential value of the luminance along this line. The program execution unit 211 in FIG. 2 functions as the above-described double differential value calculation unit 140.

  The twice differential value calculation unit 140 calculates the differential value twice for each of a plurality of lines in a predetermined direction. The predetermined direction may be a horizontal direction, a vertical direction, or both directions. Further, the differential value may be calculated twice for all lines. Alternatively, appropriate thinning may be performed, and the differential value may be calculated twice for the lines at predetermined intervals. In this case, the line interval is appropriately set according to the size of the stain to be detected. Subsequent processes such as the calculation of the differential value twice, the spot detection using the calculation result, and the quality determination are performed in the same manner for each line. Therefore, hereinafter, the camera inspection apparatus 1000 according to the present embodiment will be described with a focus on processing for one horizontal line.

The twice-differential value calculation unit 140 calculates the twice-differential value according to the following equations (2) and (3). Formula (2) is a formula for calculating a single differential value, and Formula (3) is a formula for calculating a differential value twice from a single differential value.
D1 _n = (Y _n −Y _n−1 ) / {n− (n−α)} (2)
D2 _n = (D1 _n −D1 _n−1 ) / {n− (n−α)} (3)

Here, n is a pixel coordinate on the selection line to be calculated, Y _n is a luminance value of n pixels, D1 _n is a one-time differential value of n pixels, and D2 _n is 2 of n pixels. It is a differential value. In the present embodiment, as shown in Expression (2) and Expression (3), the differential value is calculated twice from the luminance values at two points separated by the differential interval α.

  In the present embodiment, the differential interval is set differently depending on the position on the image data. In accordance with this differential interval setting, the double differential value calculation unit 140 calculates the differential value twice using a differential interval that varies depending on the position on the image data. The differential interval is set based on the luminance shading characteristics of the camera to be inspected. Luminance shading is a phenomenon that depends on an optical system such as an image sensor or a lens unit of a camera. Due to the luminance shading characteristics, a luminance distribution is generated in the captured image of the reference subject.

  In the present embodiment, it is assumed as the luminance shading characteristic that the luminance at the peripheral portion of the image data is suddenly lowered as compared with the central portion. Therefore, the differential interval is set differently between the predetermined range in the central portion of the image data and the outer peripheral portion, and specifically, the differential interval at the peripheral portion is set smaller than the central portion. . The setting of the differential interval and its effect will be described in detail later.

  The differential interval storage unit 150 stores a differential interval for calculating the differential value twice. As described above, the central differential interval 151 and the peripheral differential interval 152 are set as differential intervals when calculating the differential value twice, and the latter is set smaller. The memory unit 213 in FIG. 2 functions as the differential interval storage unit 150 described above.

  The spot detection unit 160 includes a spot position detection unit 161 and a spot level calculation unit 162. The spot position detection unit 161 detects the position of the spot in the image data based on the calculation result of the twice differential value calculation unit 140. The stain level calculation unit 162 calculates a stain level based on the stain position detected by the stain position detection unit 161. The program execution unit 211 in FIG. 2 functions as the spot detection unit 160 described above.

  Here, with reference to FIG. 4, the relationship between the stain in the image and the above-described twice differential value, and the stain detection and the stain level calculation based on the double differential value will be described. FIG. 4 is an enlarged view of a luminance curve at a position where a spot is generated in the image data. As shown in the drawing, the luminance value is smaller at the position where the spot is generated as compared with the periphery of the spot. As shown in FIG. 4, the brightness changes particularly sharply at the edge portions A and B of the stain in the image data, and the brightness curve bends sharply. Moreover, the brightness curve is “convex upward” at the edges A and B of the stain. Accordingly, in the calculation result of the twice-difference value of luminance, the end portions A and B of the stain appear as feature points having a large absolute value and a negative second-derivative value of luminance. By specifying this feature point, the start position (stain end A) and end position (stain end B) of the stain can be detected, and a space between them can be detected as a stain.

  Specifically, the spot position detection unit 161 detects the spot position as follows. The stain position detection unit 161 compares the stain end determination threshold value 171 stored in the stain determination threshold value storage unit 170 with the twice differential value calculated by the twice differential value calculation unit 140, and stores the image data in the image data. Detect the position of the spot. At this time, the spot position detection unit 161 sets the position where the double differential value is equal to or higher than the spot end determination threshold first as the spot start position, and then sets the position where the second differential value is equal to or higher than the spot end determination threshold as the spot end position. Detect the position of the stain. In the example of FIG. 4, the spot end A is the spot start position A, and the spot end B is the spot end position.

The spot level calculation unit 162 identifies a position (lowest brightness position M) having the lowest brightness between the start position (spot end A) and the end position (stain end B), and extracts a brightness value at that position. The stain level is calculated according to the following equation (4). The stain level is a ratio of “the luminance value at the position with the lowest luminance” to “the average value of luminance at the start position and the end position of the stain”. The “average value of the luminance at the start position and the end position of the spot” is used as a value representing the luminance at the corresponding position when there is no spot.

  The spot determination threshold storage unit 170 stores a spot end determination threshold 171 and a spot level determination threshold 172. As described above, the spot end determination threshold 171 is used by the spot position detection unit 161 to detect the start position and the end position of the spot in the image data. The stain level determination threshold value 172 is used by the pass / fail determination unit 180 described below to determine whether the stain level, which is a calculation result of the stain level calculation unit 162, is a satisfactory level. The memory 213 in FIG. 2 functions as the above-described spot determination threshold storage unit 170.

  The pass / fail determination unit 180 compares the stain level calculated by the stain level calculation unit 162 with the stain level determination threshold value 172. As a result of the comparison, the pass / fail determination unit 180 determines “good” when the stain level is smaller than the stain level determination threshold 172, and determines “no” when the stain level is greater than the stain level determination threshold 172. The program execution unit 211 in FIG. 2 functions as the above pass / fail judgment unit 180. The determination result output unit 190 outputs the determination result of the pass / fail determination unit 180. The display unit 220 in FIG. 2 functions as the determination result output unit 190 described above.

  Here, the setting of the differential interval when calculating the differential value twice in the above-described double differential value calculation unit 140 will be described. Here, first, the relationship between the differential interval when the stain is located at the central portion and the peripheral portion in the image data and the result of the twice differential value calculation will be described based on a specific example. Further, the influence of luminance shading and the differential interval The relationship will be described, and then the preferred differential interval setting of the present embodiment will be described.

  First, an example in which a stain appears in the center of the image data will be described. FIG. 5 is a diagram illustrating an example of an image, and a stain is reflected in the center of the image. FIG. 6 is a luminance curve corresponding to FIG. 5 and shows a luminance curve (horizontal cross-sectional waveform) along a line passing through the stain of FIG. 5 in the horizontal direction (left and right direction on the paper surface). FIG. 7 shows a calculation result of calculating the twice differential value of the luminance represented by the luminance curve of FIG. 6 with the differential interval being 16 pixel intervals. FIG. 8 shows a calculation result obtained by calculating the twice differential value of the luminance represented by the luminance curve of FIG. 6 by setting the differential interval to 32 pixel intervals larger than FIG.

  In the example of FIG. 5, a spot 501 is generated at the center of the image 502, and as a result, the brightness drops at a position corresponding to the spot 501 as shown in FIG. 6. Then, as shown in FIGS. 7 and 8, in the calculation result of the double differential value, the characteristic points a, b, c, where the double differential value becomes a large negative value at the spot ends A1 and B1. d appears. When comparing FIG. 7 and FIG. 8, the feature points clearly appear in both cases, but the feature points c and d in FIG. 8 are the absolute values of the secondary differential values than the feature points a and b in FIG. 7. large. Thus, by setting the differential interval large, the absolute value of the secondary differential value at the feature point corresponding to the stain edge can be increased, and the position of the stain in the image data can be easily detected.

  Next, an example in which a stain appears in the peripheral portion of the image data will be described. FIG. 9 is a diagram illustrating an example of an image, in which a stain appears in the upper left edge of the image. FIG. 10 is a luminance curve corresponding to FIG. 9 and shows a luminance curve passing through the upper left spot in the horizontal direction. 11 and 12 are diagrams showing the calculation result of the twice differential value of the luminance represented by the luminance curve of FIG. 10, in which the differential interval is 20 pixel intervals in FIG. 11, and the differential interval is FIG. The interval is 8 pixels.

  In the example of FIG. 9, a stain 901 occurs in the upper left part of the image 902, and as a result, as shown in FIG. 10, the luminance is lowered and falls at the left end of the luminance curve. According to the luminance curve of FIG. 10, characteristic points should appear in two places in the curve of the twice differential value corresponding to both ends A2 and B2 of the stain. However, in FIG. 11 where the differential interval is large, only one feature point e appears, and the position of the point A2 corresponding to the start position of the spot 901 cannot be specified. On the other hand, in FIG. 12 in which the differential interval is set to be as small as 8 pixels, two feature points f and g clearly appear, and both the points A2 and B2 corresponding to the start position and the end position of the spot 901 are displayed. The position can be reliably identified. Therefore, at the peripheral portion of the image data, by reducing the differential interval when calculating the differential value of the brightness twice, the position of the stain can be reliably detected even when a stain occurs at the edge of the image. .

  Next, the relationship between the influence of luminance shading and the differential interval will be described. FIG. 13 is a diagram illustrating an example of an image, and a stain appears in the upper left of the image. FIG. 14 is a luminance curve corresponding to FIG. 13 and shows a luminance curve along a line passing through the stain of FIG. 13 in the horizontal direction. FIGS. 15 and 16 are diagrams showing the calculation results of the twice differential value of the luminance represented by the luminance curve of FIG. 14. In FIG. 15, the differential interval is 32 pixel intervals, and in FIG. 16 the differential interval is The interval is 16 pixels.

  In the example of FIG. 13, a stain 1301 is generated at the upper left portion of the image 1302. For this reason, the luminance is lowered and falls on the left side of the luminance curve of FIG. 14. Also in the curve, feature points appear at locations corresponding to the spots 1301. Looking further at FIG. 15, in the right part of FIG. 15, the double differential value has a large negative value despite the fact that no stain has occurred in the right part of the screen in FIG. 13. Appears. The absolute value of the twice differential value at the feature point has reached the same level as the position of the spot. The occurrence of this feature point is due to luminance shading. As shown in FIG. 14, the luminance suddenly decreases due to luminance shading in the right portion of the image, and in FIG. 15, the twice-differentiated value becomes a large negative value in that portion. Such feature points cause false detection of spots.

  On the other hand, in FIG. 16, the differential interval is set smaller than that in FIG. By setting the differential interval small, the double differential value becomes a large negative value at a sharp bend at the edge of the stain, but the double differential value does not increase at locations where the luminance gradient is simply large due to luminance shading. . Therefore, as shown in FIG. 16, the appearance of feature points at locations where the luminance changes suddenly by luminance shading is suppressed while maintaining the characteristic points at the positions of the spots. In this way, by setting the differential interval to be small, it is possible to separate the spot portion from the luminance shading portion, and this reduces the influence of luminance shading on the camera optical system more appropriately. Thus, it is possible to detect a stain in the image data with higher accuracy.

  As described above, the relationship between the differential interval and the twice differential value is described for each of when the stain occurs in the central portion and the peripheral portion of the image data, and further, the relationship between the influence of luminance shading and the differential interval is described. In the present embodiment, the differential interval is set based on these relationships. In the present embodiment, a luminance shading characteristic is assumed in which the luminance is sharply reduced in the peripheral portion of the image data as compared with the central portion.

  In the central portion of the image data, that is, a predetermined range in the center of the image, the differential interval is set larger than that of the peripheral portion. In the peripheral part which is both sides of the central part, the differential interval is set smaller than that in the central part. The boundary between the central portion and the peripheral portion is set in advance at a predetermined position on both sides across the center of the image, that is, one location in one half of the image and one location in the other half. More specifically, the boundary between the central portion and the peripheral portion is a place where the degree of luminance change due to luminance shading changes, and where the luminance starts to suddenly decrease from the central portion to the peripheral portion of the image data. The boundary between the central portion and the peripheral portion is set in advance at a predetermined position inside the place where the luminance change is steep. With the sudden change start portion of the brightness as a boundary, a differential interval at the center is set in an area inside (center side) of the image, and a differential interval at a peripheral edge is set in an area outside (both ends) of the image.

  With these settings, a spot edge appears more prominently as a feature point in the differential value at the center of the image data. At the periphery of the image data, even when the stain is at the edge of the image, the differential value becomes a large negative value twice, and the feature point appears reliably, and the brightness change due to the brightness shading is separated from the stain. Can be done clearly. By such setting, it is possible to reduce the influence of luminance shading of the optical system of the camera and accurately detect a stain in the image data.

  The functions of the camera inspection apparatus 1000 according to the present embodiment have been described above. Next, the operation of the camera inspection apparatus 1000 according to the embodiment of the present invention will be described with reference to the drawings. FIG. 17 is a diagram illustrating an operation flow of the camera inspection apparatus 1000. When the program execution unit 211 executes the inspection program 214a in the program storage unit 214, the camera inspection apparatus 1000 performs the camera inspection process of FIG.

  First, as shown in FIG. 17, the image data acquisition unit 110 acquires image data obtained by imaging a reference subject with a camera (step (STEP: hereinafter referred to as S) 1701). The luminance calculation unit 120 calculates the luminance value of each pixel of the image data acquired by the image data acquisition unit 110 (S1702). At this time, as described above, the luminance calculation unit 120 calculates a luminance value by conversion into an 8-bit grayscale luminance signal according to the equation (1).

  Next, the luminance value averaging unit 130 performs an averaging process on the luminance value of each pixel calculated by the luminance calculating unit 120 (S1703). Specifically, an average value of luminance values in a predetermined range including the target pixel (for example, a total of 49 pixels of 7 × 7 pixels) is calculated, and the luminance value of the target pixel is replaced with the average value. This process is repeated three times for all pixels in the image data. With this averaging process, as described with reference to FIGS. 3A and 3B, the luminance change caused by the stain is retained, and the electrical noise is eliminated.

  Next, the twice differential value calculation unit 140 selects one line in the horizontal direction or the vertical direction of the image data, and calculates the double differential value of the luminance along this line using the above-described equations (2) and (3). ) In accordance with (1704). The twice differentiated value is calculated from the image data after the luminance value averaging processing by the luminance value averaging unit 130. As described above, the camera inspection apparatus 1000 sequentially selects a large number of lines and performs a process of calculating a differential value twice and detecting a subsequent spot for each line. In the following description, one line is mainly used. An explanation will be given focusing on the line.

  Next, the spot position detection unit 161 detects the position of the spot in the image data based on the calculation result of the twice differential value calculation unit 140 (S1705). Specifically, the stain position detection unit 161 compares the stain end determination threshold value 171 stored in the stain determination threshold storage unit 170 with the double differential value calculated by the double differential value calculation unit 140. Thus, the position of the spot in the image data is detected. At this time, the spot position detection unit 161 detects a position where the twice differential value is equal to or larger than the spot end determination threshold as the spot position. More specifically, the spot position detection unit 161 sets the position where the double differential value is equal to or higher than the spot end determination threshold first, and then sets the position where the second differential value is equal to or higher than the spot end determination threshold to the end of the spot. As the position, the position of the spot is detected.

  Here, as described in detail above, in the second derivative calculation process of S1704, the central differential interval 151 having a large differential interval is set in the central portion of the image data, and the central portion is set in the peripheral portion. The differential interval 152 for the peripheral portion, which has a smaller differential interval compared to, is set. Thereby, in the spot position detection process of S1705, the influence of the luminance shading of the optical system of the camera can be reduced more appropriately, and the spot in the image data can be detected with higher accuracy.

  Next, the stain level calculation unit 162 calculates a stain level based on the stain position detected in S1705 (S1706). Specifically, as described with reference to FIG. 4, the spot level calculation unit 162 identifies a position (the lowest brightness position) having the lowest brightness between the start position and the end position of the spot, and the brightness value at that position. And the stain level is calculated according to the above equation (4).

  After the stain level is calculated, the pass / fail determination unit 180 compares the calculated stain level with the stain level determination threshold 172 to determine pass / fail of the stain level (S1707). Specifically, the pass / fail determination unit 180 compares the calculated stain level with the stain level determination threshold 172, and determines that the stain level is “good” when the stain level is smaller than the stain level determination threshold 172. The stain level is greater than the stain level determination threshold 172. In this case, “No” is determined. The determination result of the pass / fail determination unit 180 is output by the determination result output unit 190, and the inspector can check the pass / fail determination result of the stain level. As described above, the operation of the camera inspection apparatus 1000 according to the embodiment of the present invention has been described.

  According to the camera inspection apparatus of the embodiment of the present invention as described above, the twice differential value of the luminance along the predetermined direction is calculated from the image data, and the position of the stain in the image data is determined based on the twice differential value. To detect. In this spot position detection, the position where the twice-differentiated value becomes a large negative value (a negative value with a large absolute value) is specified as the end of the spot. In the present invention, especially when calculating the differential value of the luminance twice for spot position detection, the differential interval is set to a position along a predetermined direction based on the luminance shading characteristic that produces a luminance distribution in the captured image of the reference subject. Different depending on the situation. With this configuration, it is possible to prevent erroneous detection of a luminance change in luminance shading as a stain by appropriately changing the differential interval in accordance with the gradual change in luminance due to luminance shading. As a result, it is possible to reduce the influence of luminance shading of the optical system of the camera and to accurately detect spots in the image data.

  In addition, according to the camera inspection apparatus of the present invention, the differential interval at the peripheral edge outside the central portion is set smaller than the differential interval at the central portion of the image data, and the luminance of the image data is increased. A differential value is calculated twice. This configuration is suitable for inspection of a camera including an optical system having a general luminance shading characteristic in which the luminance is sharply decreased in the peripheral portion of the image data as compared with the central portion. In the central part of the image data, by setting the differential interval larger than that of the peripheral part, the double differential value at the stain end increases in the negative direction, and the stain end becomes a feature point at the double differential value. Remarkably appear, and the position of the spot can be easily detected. On the other hand, by setting the differential interval smaller at the periphery of the image data than at the center, even if the stain is at the edge of the image, the twice differential value becomes a large negative value, and the feature point is reliably In addition, it is possible to prevent the differential value from becoming a large negative value twice at the point of sudden change in luminance due to luminance shading, and to clearly distinguish between the stain and the luminance change due to luminance shading. In this way, by setting the differential interval at the peripheral portion to be smaller than the differential interval at the center portion of the image data, the influence of the luminance shading of the camera optical system can be reduced. Can be detected with high accuracy.

  Further, according to the camera inspection apparatus in the embodiment of the present invention, the position of the stain in the image data is detected by comparing the differential value twice with a predetermined stain end determination threshold value. Accordingly, the position where the twice differential value becomes a large negative value can be specified as a feature point with reference to a predetermined spot edge determination threshold, so that the position of the spot in the image data can be easily detected.

  Further, according to the camera inspection apparatus in the embodiment of the present invention, the differential value of the second time exceeds the predetermined spot end threshold at the position where the predetermined threshold end threshold is exceeded, and then exceeds the predetermined spot end threshold. The position of the spot is detected with the position as the end position of the spot. As a result, based on a predetermined spot edge determination threshold, the position where the twice differential value becomes a negative negative value first in the image data is set as the spot start position, and the double differential value becomes a negative negative value later. Since the position can be specified as the end position of the stain, the position of the stain in the image data can be easily detected.

  The embodiments of the present invention have been described above by way of example, but the scope of the present invention is not limited to these embodiments, and can be changed or modified according to the purpose within the scope of the claims. is there.

  In the above embodiment, the camera in the embodiment of the present invention has been described as a camera module mounted on a portable terminal such as a mobile phone terminal or a portable information terminal. However, the present invention is not limited to this. The camera in may be a general digital camera.

  In the above embodiment, it has been described that the twice differential value calculation unit 240 selects one line in the horizontal direction or the vertical direction of the image data and calculates the double differential value of the brightness along this line. The differential value calculation unit 240 may calculate a double differential value of the luminance along any predetermined direction of the image data, not limited to the horizontal direction or the vertical direction.

  In the above embodiment, the camera inspection apparatus and the camera inspection method according to the present invention have been specifically described. However, the present invention is not limited to these. The present invention may be a camera inspection program that causes a computer to execute each step of the above-described camera inspection method, or may be a computer-readable recording medium that records the camera inspection program.

  As described above, the present invention has the effect of reducing the influence of luminance shading of the optical system of the camera and accurately detecting spots in the image data, and is imaged by a camera having an image sensor. It is useful for a camera inspection apparatus that detects spots caused by foreign matters such as dust and dust attached to the light receiving surface of the image sensor or its peripheral part based on the image data thus obtained.

Functional block diagram of a camera inspection apparatus in an embodiment of the present invention The figure which shows the structure of the camera inspection apparatus in embodiment of this invention from a hardware surface (A) The figure which shows the luminance curve before the averaging process (b) The figure which shows the luminance curve after the averaging process The figure which shows an example of the brightness | luminance curve which expanded the part which the stain has arisen in image data The figure which shows an example of the image which the spot produced in the center part The figure which shows an example of the brightness | luminance curve of an image The figure which shows the calculation result of the twice differential value of the brightness | luminance in case a differentiation space | interval is a 16-pixel space | interval. The figure which shows the calculation result of the twice differential value of the brightness | luminance in case a differential interval is a 32 pixel interval. The figure which shows an example of the image which the spot produced in the peripheral part The figure which shows an example of the brightness | luminance curve of an image The figure which shows the calculation result of the twice differential value of the brightness | luminance in case a differentiation space | interval is a 20 pixel space | interval. The figure which shows the calculation result of the twice differential value of the brightness | luminance in case a differential interval is an 8-pixel interval. The figure which shows an example of the image which the spot has arisen in the upper left The figure which shows an example of the brightness | luminance curve of an image The figure which shows the calculation result of the twice differential value of the brightness | luminance in case a differential interval is a 32 pixel interval. The figure which shows the calculation result of the twice differential value of the brightness | luminance in case a differentiation space | interval is a 16-pixel space | interval. The figure which shows the operation | movement flow of the camera inspection apparatus in embodiment of this invention The figure which shows the example of the picked-up image where the spot is reflected The figure which shows an example of the brightness | luminance curve of an image

Explanation of symbols

DESCRIPTION OF SYMBOLS 110 Image data acquisition part 120 Luminance value calculation part 130 Luminance value averaging part 140 Two-time differential value calculation part 150 Differential interval storage part 160 Spot detection part 161 Spot position detection part 162 Spot level calculation part 170 Spot determination threshold storage part 180 Pass / fail Determination unit 190 Determination result output unit 1000 Camera inspection device 210 Computer main body 211 Program execution unit 212 Communication unit 213 Memory unit 214 Program storage unit 214a Spot inspection program 220 Display unit 230 Operation unit

Claims (7)

A camera inspection device for detecting a stain in an image caused by a foreign matter attached to a light receiving surface of the image sensor or its peripheral portion based on image data captured by a camera having an image sensor,
An image data acquisition unit for acquiring image data obtained by imaging a reference subject using the camera;
A two-time differential value calculating unit for calculating a double differential value of luminance along a predetermined direction from the image data;
A position detection unit that detects a position of a stain in the image data based on a calculation result of the second derivative calculation unit;
The two-time differential value calculation unit is a characteristic of the camera optical system, and is a derivative for calculating the second-time differential value based on a luminance shading characteristic that generates a luminance distribution in a captured image of the reference subject. A camera inspection apparatus, wherein the interval is varied according to a position along the predetermined direction.   2. The differential value calculation unit sets the differential interval at the peripheral portion of the image data to be smaller than the differential interval at the central portion of the image data. Camera inspection equipment.   3. The camera according to claim 1, wherein the position detection unit detects the position of the spot in the image data by comparing the twice-differentiated value with a predetermined spot end determination threshold value. Inspection device.   The position detection unit sets the position where the double differential value first becomes equal to or greater than the predetermined stain end threshold as the start position of the stain, and then sets the position where the second differential value is equal to or greater than the predetermined stain end threshold as the end of the stain The camera inspection apparatus according to claim 3, wherein the position of the spot is detected as a position. A camera inspection program for causing a computer to perform a process of detecting a stain in an image caused by a foreign substance attached to the light receiving surface of the image sensor or its peripheral part based on image data captured by a camera having an image sensor. There,
Obtaining image data obtained by imaging a reference subject using the camera;
Calculating a twice differential value of luminance along a predetermined direction from the image data;
Detecting the position of a stain in the image data based on the calculation result of the twice differential value calculation unit, causing the computer to execute,
In the step of calculating the twice differential value of the luminance, the second differential value is calculated based on a luminance shading characteristic that is a characteristic of the optical system of the camera and causes a luminance distribution in the captured image of the reference subject. A camera inspection program characterized in that a differential interval at the time of performing is varied according to a position along the predetermined direction.   A computer-readable recording medium on which the camera inspection program according to claim 5 is recorded. On the basis of image data captured by a camera having an image sensor, a camera inspection method for detecting a stain in an image caused by a foreign matter attached to the light receiving surface of the image sensor or its peripheral portion,
Obtaining image data obtained by imaging a reference subject using the camera;
Calculating a twice differential value of luminance along a predetermined direction from the image data;
Detecting a position of a stain in the image data based on a calculation result of the twice differential value calculation unit,
In the step of calculating the twice differential value of the luminance, the second differential value is calculated based on a luminance shading characteristic that is a characteristic of the optical system of the camera and causes a luminance distribution in the captured image of the reference subject. A camera inspection method, characterized in that a differential interval at the time of performing is varied according to a position along the predetermined direction.

Images