JP2007130329A - Appearance inspection method by image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an appearance inspection method by an image which can extract a trait portion so as to be in agreement with feeling seen by a person. <P>SOLUTION: In a collar image which is acquired by photographing an inspected thing (S1), a color separation shaft, which passes through a background portion and a trait portion within color space, is established and a single channel image, which has a location on the color separation shaft when projecting each picture element on the color separation shaft in a pixel value, is formed (S2). Next, a connection domain comprising a picture element, where the difference of the pixel value between the adjoining picture elements is below a specified value, is formed (S3) in the single channel image. Then, a trait determination domain is formed by unifying the connection domain using the distribution of the internal and external pixel values in the connection domain which serves as the candidate of the trait portion (S4-S6). The trait portion is classified into the items of two or more skin troubles as classification by classifying the amount of traits determined from the trait determination domain (S7-S8). <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention makes it possible to extract a characteristic part on the appearance of an inspected object from not only an area having a large change in brightness but also an area having a small change in brightness by using a grayscale image of the inspected object. The present invention relates to an appearance inspection method using an image for inspecting stains and color unevenness generated on the surface of products, metal products, and the like.

  2. Description of the Related Art Conventionally, there has been provided an appearance inspection apparatus that extracts and evaluates candidates for characteristic portions on the appearance of an inspection object using a grayscale image obtained by imaging the inspection object. The inspection object to be inspected is often an industrial product, but recently the human skin is used as the inspection object in order to quantitatively confirm and evaluate the effects of cosmetics and cosmetic treatments. Even in this case, an image inspection technique using images has been adopted.

  That is, conventionally, as a method for confirming the state of the skin, an inquiry to the person himself / herself, visual inspection or palpation by an expert are common, and qualitative evaluation has been performed. In addition, although equipment for measuring the degree of skin roughness etc. is also provided, the features on the appearance of the skin such as spots, wrinkles, blackened pores and uneven color (hereinafter referred to as “skin trouble”) However, the current situation depends mainly on visual inspection.

  As for the above-mentioned skin troubles, i.e., spots, wrinkles, pore darkening, and color unevenness, the brightness of a monochrome grayscale image is lower than that of the surroundings. It is conceivable to binarize a grayscale image with respect to brightness after performing a process for removing the image (hereinafter referred to as “shading removal process”) (see, for example, Patent Document 1). By extracting connected components of pixels that are candidates for feature parts in this binary image and evaluating various feature values (area, length in inertia principal axis direction, circularity, average brightness, etc.) for the connected components It is judged whether it is a skin trouble or not. In addition, if it is a skin trouble, the type is classified, and the amount of occurrence such as the number and area is obtained quantitatively for each classification item, and the degree of skin trouble is quantitatively determined using the brightness of the gray image of the skin trouble area. Ask for.

Here, the binary image is generated using an appropriate threshold value with respect to the lightness, the pixel value of the portion where the lightness is lower (that is, dark) than the threshold value is “1”, and the remaining pixel values are “0”. It is an image. In this binary image, a connected component of pixels that are candidates for feature portions is formed by tracking a pixel having a pixel value “1” in the vicinity of 8 pixels.
JP-A-7-19839

  As described above, when a skin trouble is extracted using a binary image that has been binarized after the shading removal process is performed on the grayscale image, the connected component is extracted using the binary image. The size of the connected component changes according to the threshold value at the time of the value conversion. That is, the range of skin troubles may be divided into a plurality of connected components even though it can be regarded as one region by visual inspection. There may be cases where only one connected component is extracted.

  In particular, spots and wrinkles on human skin have a small difference in brightness from the surroundings, and the brightness is often changing gently, and the range in which the threshold value used for binarization can be set is narrow. Problems are likely to occur. For example, when it is visually recognized that there is one large spot with uneven brightness, depending on the threshold value, a plurality of low brightness areas are extracted within the spot, and a plurality of small spots are extracted. As a result, a determination result different from the visual result is generated.

  In order to solve the above problem, it is conceivable to increase the region where the pixel value is “1” by changing the threshold value in binarization. Despite being recognized as a plurality of small spots, the determination result may be regarded as one large spot.

  In short, it is required that a characteristic portion seen by a person like a skin trouble on the face is obtained with a determination result that matches a sense visually recognized by the person. It is difficult to select a threshold value with a simple method that only involves value conversion, and the threshold setting range is narrow, so it is difficult to set the threshold value itself. Are difficult to match.

  The present invention has been made in view of the above-mentioned reasons, and the object thereof is to be inspected such that the change in lightness inside the characteristic part is gentle and the lightness distribution inside the characteristic part is uneven. An object of the present invention is to provide an appearance inspection method using an image that can extract a feature portion so as to match a sense visually recognized by a person.

  In the first aspect of the present invention, for a single channel image obtained by photographing an object to be inspected, after generating a connected region composed of pixels in which a difference in pixel value between adjacent pixels is equal to or less than a first specified value, A feature determination region is generated by integrating the connected regions using the distribution of pixel values inside and outside the connection region that is a candidate for the feature portion, and the feature portion is classified into a plurality of feature portions by classifying the feature amount obtained from the feature determination region. It is characterized by being classified into types.

  According to this method, instead of using a connected component generated by a binary image obtained by binarizing a grayscale image as in the prior art, the pixel value difference is equal to or less than the first specified value. Since the connected area is generated, the connected area can be set even when the brightness difference is small or when the brightness changes gently. In addition, since the feature determination region is generated by integrating the connected regions using the internal and external distributions of the pixel values, the range of the feature portion can be accurately extracted. In other words, the connected area is generated based on the similarity of the pixel values. On the other hand, when the feature determination area is generated by integrating the connected areas, the difference in the pixel value is used for the feature portion. An appropriate range can be extracted as a feature determination area even when the brightness is small or the brightness changes gently. As described above, the feature determination region can be appropriately extracted and quantitatively determined, so that the reproducibility of the determination result can be improved. Furthermore, if the classification method of the feature quantity is given appropriately, it will be possible to obtain a judgment result that matches the sense visually recognized by humans, so that anyone can make judgments similar to expert judgments. Become.

  The invention according to claim 2 is a connected region composed of pixels in which a difference in pixel value between adjacent pixels is equal to or less than a first specified value for a single channel image obtained by photographing a human skin as an object to be inspected. A feature determination region that separates feature portions that are candidates for skin trouble from a background portion that is normal skin using a distribution of pixel values inside and outside of the connected region and integrates the connected regions that are feature portions. The skin troubles are classified into a plurality of types by generating and classifying the feature amounts obtained from the feature determination area.

  According to this method, the same effect as in the first aspect can be expected. In particular, the difference in pixel value is also the first for skin troubles where there is little lightness difference from normal skin, such as spots, wrinkles, darkening of pores, and color unevenness in human skin, and skin troubles where the lightness changes slowly. A connected region composed of a pixel group that is equal to or less than a prescribed value of the above is generated, and the difference between the pixel value of each connected region and a normal skin portion is evaluated and integrated to generate a feature determination region, which is reliably extracted. be able to. That is, it is possible to obtain a determination result that matches the sense visually recognized by humans with respect to skin troubles, which has conventionally been difficult to obtain a determination result that matches the sense visually recognized by humans. Moreover, since the skin trouble is quantitatively determined, a determination result with good reproducibility can be obtained.

  According to a third aspect of the present invention, in the first or second aspect of the present invention, the image obtained by photographing the object to be inspected is a color image, and after the shading removal process is performed on the color image, the brightness and the color A color image is binarized using a temporary threshold set for at least one to separate the temporary feature portion and the temporary background portion, and the color passes through the temporary feature portion, the temporary background portion, and each pixel group. By projecting each pixel of a color image onto a color separation axis set in space, an image having a pixel value at a position on the color separation axis is generated, and this image is used for the single channel image.

  According to this method, it is possible to generate a single channel image in which the difference in pixel value between the feature portion and the background portion is large. That is, the color separation axis is set so as to pass through the provisional feature portion and the provisional background portion, the pixel is projected onto the color separation axis, and the position on the color separation axis is set as the pixel value, thereby the feature portion and the background portion. A single channel image with a large difference in pixel value can be generated. By using this single channel image to generate a connected region and a feature determination region, even when there is little color difference between the feature portion and the background portion It becomes easy to separate. Also, setting the color separation axis is equivalent to performing calibration related to the background color of the background portion, so that it becomes easy to extract a feature portion regardless of the background color of the background portion.

  According to a fourth aspect of the present invention, in the invention according to any one of the first to third aspects, when the feature determination area is generated by integrating the connected areas, the connected area of each connected area to be integrated is determined. The difference between the average value of the internal pixel values and the average value of the pixel values of the background portion around the connected area is obtained, and when the difference is larger than the second specified value, the connected area is defined as a feature part. When the difference is smaller than the second specified value, the connected region is integrated as a background portion, and a set of connected regions that are characteristic portions is integrated as a feature determination region.

  According to this method, since the difference between the average values of the pixel values is used when integrating the connected area into the feature determination area, the connected area can be easily and appropriately allocated to the feature portion and the background portion.

  According to a fifth aspect of the present invention, in the fourth aspect of the present invention, a skeleton is obtained for the feature determination area, and the length of a line segment connecting between the skeleton end points of two different characteristic determination areas is equal to or greater than a third specified value. The ratio of the length of the line segment to the average value of the distance between each pixel on the skeleton of both feature determination areas and the line segment is equal to or greater than a fourth specified value, and Both feature determination areas are integrated into one feature determination area when the color difference is equal to or smaller than a fifth specified value.

  According to this method, when a portion to be integrated into one feature determination region is divided into a plurality of feature determination regions, it is possible to determine whether integration is possible and to integrate as much as possible.

  The invention according to claim 6 is characterized in that, in the invention according to any one of claims 1 to 5, when classification into a plurality of types by a neural network that receives a plurality of types of feature values obtained from the feature determination area, The teaching data obtained by interactively inputting the reference value for classifying the classification, the classification result based on the reference value, and the classification correction result are accumulated, and the neural network is taught using the accumulated teaching data. .

  In this method, since a feature portion is classified using a neural network, an appropriate classification result can be obtained from a plurality of feature amounts. Moreover, by interactively inputting the teaching data of the neural network, the reference value can be converged so that the judgment by visual observation and the judgment result using the feature amount extracted from the feature judgment area can be made to coincide with each other. Difficult reference values can be set relatively easily.

  According to the method of the present invention, a connected region is generated based on the similarity of pixel values, and the difference between pixel values is used when generating a feature determination region by integrating the connected regions. Even if the brightness difference between the two is small or the brightness changes gently, the appropriate range can be extracted as the feature judgment area, and the classification result is obtained using the feature quantity of this feature judgment area. There is an advantage that can be increased. In addition, if a feature amount classification method is appropriately given, there is an advantage that anyone can make a judgment similar to that of an expert.

  In the following, a case will be described in which the skin of a person's face used in the description of the conventional configuration is an object to be inspected, and skin troubles such as spots, wrinkles, blackened pores, and color unevenness are extracted and evaluated as characteristic portions. The technical idea of the present invention can also be applied to detection of the presence or absence of spots on the copper-clad laminated substrate and spots, wrinkles, and color unevenness in synthetic resin molded products.

  In the present embodiment, as shown in FIG. 3, an imaging device 11 including a TV camera that captures the face of a person H who is an object to be inspected is provided, and a so-called ring light is used to illuminate the entire face uniformly. Used for. The illuminating device 16 has a light emitting surface formed in an annular shape, and performs imaging by the imaging device 11 through a central hole surrounded by the light emitting surface. Therefore, the illuminating device 16 performs shadowless illumination so that no shadow is generated on the inspection object by irradiating the inspection object with light from all directions around the optical axis of the imaging device 11. The illuminating device 16 of the present embodiment uses a fluorescent lamp that is formed in an annular shape and radiates at a high frequency as the light source 16a, and uses a diffuser plate 16b that is larger than the object to be inspected in front of the light source 16a. Further, the light emission luminance is set to such a level that the person H does not feel extremely dazzling and does not cause shine due to sebum.

  The imaging device 11 is configured to output a grayscale image of a digital signal, and the grayscale image obtained by photographing the inspection object with the imaging device 11 is stored in a storage device 12 including a frame memory. In the present embodiment, a color TV camera is assumed as the imaging device 11, and a light source having a good color rendering property is used as the light source of the illumination device 16. The storage device 12 may be a mass storage device such as a hard disk device.

  The characteristic portion of the inspection object is extracted by the image processing device 13 using the grayscale image stored in the storage device 12. The processing contents of the image processing device 13 will be described later. Connected to the image processing device 13 are an output device 14 for displaying images and analysis results, and an input device 15 for giving instructions to the image processing device 13. In this embodiment, a color monitor is used for the output device 14 and a mouse and a keyboard are used for the input device 15. 3 is realized by using a personal computer and executing a program having functions to be described later.

  Hereinafter, in describing the operation of the image processing device 13, it is assumed that the grayscale image shown in FIG. 4 obtained by the imaging device 11 is obtained. In the example shown in FIG. 4, wrinkles 4A due to skin sag due to aging, stains 4B due to pigment deposition, black spots 4C due to sebum dirt clogged in pores, and color unevenness 4D with pigments deposited in spots. To do.

  In order to obtain the grayscale image shown in FIG. 4 by the imaging device 11, the illumination device 16 is turned on, and the face of the person H is photographed by the imaging device 11. The grayscale image obtained by the imaging device 11 is transferred to the storage device 12 (S1 in FIG. 1). As described above, since the imaging device 11 is a color TV camera, the grayscale image is a color image, and an image signal that corresponds to the RGB color system is output here (that is, a general image is used). Use a color TV camera of the configuration).

  The RGB color system color image is a three-channel image composed of three monochrome images. In this embodiment, a process of converting the three-channel image into a single-channel image is performed (S2 in FIG. 1). The pixel value of the single channel image generated by the method described below does not represent brightness, but has a pixel value corresponding to the position on the straight line when the color of each pixel is projected onto one straight line in the color space. is doing. The grayscale image has a pixel value corresponding to the position on the lightness axis when the color of each pixel is projected onto the lightness axis in the color space, and is treated as an example of a single channel image described below. Can do. Conversely, it can be said that the single channel image described below is also a grayscale image in a broad sense. In the present embodiment, the human face is the object to be inspected, the inspection item is skin trouble, and the eyes and nose are not inspection targets. Therefore, as shown in FIG. (Rectangular) mask 4P is used, and only the image inside the mask 4P is set as an inspection target.

As a method of generating a single channel image from a three-channel image, a weighted average value of each channel of a color image is used, or an RGB image is converted into a color image of L ^* a ^* b ^* color system and L representing the brightness. ^* Channel components can be used. However, these methods cannot sufficiently reflect the color change in the single-channel image even though it is possible to capture a decrease in brightness due to skin trouble. In particular, since the L ^* channel of the color image of the L ^* a ^* b ^* color system does not have hue and saturation information, a single channel image generated by this method cannot detect a color change.

  Therefore, in the present embodiment, a part considered to be normal (hereinafter referred to as “background part”) excluding the characteristic part of the skin as the object to be inspected, and a characteristic part considered to cause the skin trouble In order to make it easier to separate the two, a straight line passing through the background portion and the feature portion in the color space is obtained. Hereinafter, this straight line is referred to as a “color separation axis”. That is, assuming that the pixel value in the color space of the background portion and the feature portion is represented by a linear combination of a small number of latent variables, a straight line represented by the latent variables is obtained as a color separation axis. The color separation axis corresponds to a basic formula in the factor analysis model.

The procedure for converting an RGB color system color image into a single channel image in step S2 of FIG. 1 is specifically as shown in FIG. That is, the RGB color system color image stored in the storage device 12 is first converted to an L ^* a ^* b ^* color system color image (S11). By the way, since the face that is the object to be inspected has irregularities, even if the illumination device 16 that illuminates without shadow is used, a difference in brightness occurs gently in the color image according to the irregularities of the face (see FIG. 4). The shaded area represents the part with low lightness).

Therefore, shading removal processing is applied to the L ^* channel representing the lightness to remove the change in lightness (S12). In this shading removal process, a spatial frequency filter that removes low-frequency components is used. For example, a Laplacian Gaussian filter is used as this type of spatial frequency filter. FIG. 5 shows an image obtained by removing low-frequency components from the L ^* channel image in step S12 with respect to the color image of FIG.

In order to obtain the color separation axis described above, first, the background portion and the feature portion are provisionally separated using the image shown in FIG. That is, after the shading removal process is performed on the image of the L ^* channel, the threshold value is temporarily set and binarization is performed (S13). In this binarization, since only the brightness information is used, the background portion and the feature portion cannot be accurately separated. However, as shown in FIG. It can be separated into a “provisional background portion” 41 and a feature portion (black portion) candidate (referred to as “provisional feature portion”) 42. It is also possible to binarize using color information instead of lightness, or binarize using both lightness and color information.

For each pixel belonging to the provisional background portion 41 and the provisional feature portion 42 thus obtained, the pixel value of the L ^* channel after the shading removal processing and the pixel value of the a ^* channel and b ^* channel are represented. Are plotted in the color space of the L ^* a ^* b ^* color system (S14). Since the pixels plotted in the color space of the L ^* a ^* b ^* color system have the background portion and the feature portion candidate colors, the color separation axis can be obtained by using these colors.

Now, it is assumed that the pixels are plotted as shown in FIG. 7 in the color space of the L ^* a ^* b ^* color system by the process of step S14. The pixels plotted in the color space as described above are considered that the pixels in the background portion are close to each other in the color space, and the pixels in the characteristic portion are close to each other in the color space. That is, it is considered that the pixel group 51 of the background part and the pixel group 52 of the characteristic part are formed. Therefore, in order to set the color separation axis, average values e51 and e52 of colors are obtained for the pixel group 51 of the background portion and the pixel group 52 of the feature portion, respectively (S15). If the straight line passing through the average values e51 and e52 in the color space is taken as the color separation axis 53 (S16), when the color of each pixel is projected onto the color separation axis 53, the pixels of the background portion are projected on the color separation axis 53. It is separated into a group and a pixel group of the characteristic part.

  Therefore, after obtaining the color separation axis 53, the pixels subjected to the shading removal process in step S12 are newly projected onto the color separation axis 53, and the distance from the reference point appropriately set on the color separation axis 53 is set as the pixel value. A single channel image is generated (S17). That is, a single channel image can be generated from a color image that is a three-channel image. In this single channel image, the difference in pixel value between the background portion and the feature portion becomes large, so that the background portion and the feature portion can be easily separated. Here, the positive direction of the color separation axis 53 is set in a direction in which the brightness increases.

  The following processing is performed using the single channel image obtained by the above method. The process of extracting a feature using a single channel image includes the process of grouping pixels with similar tones, the process of evaluating the feature quantity of the grouped area, and the result of the evaluation of the feature quantity. The pixel grouping process is larger than the connected area by determining whether or not the connected areas can be integrated and the process of generating a connected area that is a set of a relatively small number of pixels. The process includes a process of generating a feature determination area that is a set. In the process of evaluating the feature quantity, the feature quantity is evaluated for the feature determination area. In classifying skin troubles, a neural network (neurocomputer) is used in this embodiment.

  In the process of generating the connected region (S3 in FIG. 1), the single channel images generated by the above-described method are grouped using the pixel values as they are without binarizing the pixel values. In other words, in the prior art, a binary image obtained by binarizing a grayscale image by applying an appropriate threshold value for brightness is used, and a connected component is extracted from the binary image, and a feature amount of the connected component is evaluated. However, in the present embodiment, as shown in FIG. 8, the difference between the pixel values of the pixel of interest Pi and the pixels in the vicinity of 8 is obtained for the pixel of interest Pi, and the difference in pixel value of the pixels in the vicinity of 8 is equal to or less than the specified value. If it is, the pixel is grouped as a pixel in the same connected area as the pixel Pi of interest. Here, the specified value to be compared with the difference between the pixel values depends on the relationship between the resolution of the imaging device 11 and the size of the feature portion, but is the smallest size among the skin troubles to be extracted as the feature portion in the present embodiment. There are pores, and if an attempt is made to evaluate pores, an area of about 10 pixels is required in the image. Therefore, in this embodiment, for a candidate for a feature portion, the size of one connected area is about 10 pixels. It is set to become. In other words, when the characteristic portion is projected onto the color separation axis 53, the specified value for grouping is set so that it can be separated into about 10 pixels on the color separation axis 53. FIG. 8 shows an example in which four connected regions A1 to A4 are obtained from the feature portion candidates.

  When the connected region is generated, it is evaluated whether or not the connected region can be further integrated, and the feature determining region is generated by grouping the connected regions that can be integrated (S4 to S6). FIG. 9 shows a state in which the feature determination areas B61 to B610 are generated. In FIG. 9, the lines inside the feature determination areas B61 to B610 represent the boundary lines of the connection areas. In FIG. 9, the feature determination area B63 is a part visually equivalent to a spot, and it is assumed that this part includes five connection areas A631 to A635 as shown in FIG. That is, since a stain having a large area has color unevenness, it is divided into a plurality of connection regions A631 to A635 even within a range that can be regarded as one stain visually. However, since it can be regarded as one spot visually, it is evaluated whether or not a plurality of connection regions A631 to A635 can be integrated in order to match the result visually.

  When generating a connected area, the degree of similarity between pixel values is evaluated, but when generating a feature determination area, the degree of difference between each connected area and the background is evaluated. That is, in the example of FIG. 10, the difference between the pixel values inside each of the connection regions A631 to A635 and the background pixel values around the connection regions A631 to A635 for evaluating whether or not to integrate them is evaluated.

  Specifically, for each of the connection regions A631 to A635, first, average values of pixel values of internal pixels (hereinafter referred to as “internal average values”) InA631 to InA635 are obtained. Further, for each of the connection regions A631 to A635, an average pixel of pixel values of pixels that exist in a circle having a specified radius r centered on the position of the center of gravity G1 and do not belong to any of the connection regions A631 to A635 ( Hereinafter, OutA631 to OutA635 are referred to as “external average values”. Next, a difference (contrast value) CA63n (= InA63n−OutA63n) (n = 1 to 5) between the internal average value InA631 to InA635 and the external average value 0utA631 to OutA635 is obtained for each connection region A631 to A635, The contrast value CA63n is compared with the specified value T. In general, since the characteristic part is darker than the background part, the sign of the specified value T with respect to the contrast value CA63n obtained by the above arithmetic expression is negative. The radius r is set so that the whole of the connection areas A631 to A653 to be determined as to whether or not integration is possible is included in the circle, and usually the maximum distance from the center of gravity G1 of the connection areas A631 to A635 to the contour line. Set 3 to 5 times.

  If the condition that the absolute value of the contrast value CA63n is larger than the absolute value of the specified value T is satisfied, it can be determined that the characteristic part is different from the background part. These are integrated into one feature determination area (S4, S5 in FIG. 1). That is, the contrast value CA631 to CA635 obtained for each of the connection regions A631 to A635 is set to −17, −8, −24, −20, and −13 in the example of FIG. Then, the connection areas A631, A633, and A634 satisfy the conditions. That is, the feature determination region B63 is generated by integrating the connection regions A631, A633, and A634 that are in contact with each other (S6 in FIG. 1).

  In the processing described above, the feature determination region is generated by integrating the connected regions that are in contact with the boundary. However, it may be possible to integrate the connected regions that are not in contact with the boundary. For example, wrinkles of skin troubles may be divided, and the boundaries of the extracted connected areas may be separated. In the example of FIG. 11, the feature determination regions B62 and B69 are regions that are visually equivalent to wrinkles, and the feature determination region B62 has one feature determination region by the above-described processing because the connected region touches the boundary. Although the feature determination region B69 is desired to be integrated into one feature determination region, a plurality of (three in the illustrated example) feature determination regions B691 to B693 are generated in the above-described processing. It will be.

  Therefore, hereinafter, a process of integrating the long and narrow feature determination areas into one will be described. First, skeletons (skeleton lines or core lines) SK are extracted from the feature determination areas B61 to B68, B691 to B693, and B610 obtained by integrating the connected areas in the above-described procedure. The skeleton SK is longer when the feature determination regions B61 to B68, B691 to B693, and B610 have a long and narrow shape such as a region corresponding to a wrinkle than the circular shape. Therefore, by evaluating the shapes of the feature determination regions B61 to B68, B691 to B693, and B610, it is determined whether or not the feature determination regions B61 to B68, B691 to B693, and B610 can be further integrated.

  First, a line segment ps1-pe1 that connects both end points of the skeleton SK is set, and the length La of this line segment ps1-pe1 is compared with a specified value TL1. If the length La is equal to or greater than the specified value TL1, it is determined that the feature determination areas B61 to B68, B691 to B693, and B610 have an elongated shape. In the illustrated example, since the feature determination region B62 satisfies this condition, the average value Lb of the distance between each point on the skeleton SK and the line segment ps1-pe1 is obtained for the feature determination region B62. Here, in order to evaluate the linearity of the feature determination region B62, the ratio of the length La to the average value Lb (= La / Lb) is obtained, and La / Lb is compared with the specified value TL2. If La / Lb is equal to or greater than the specified value TL2, the feature determination area B62 can be regarded as a wrinkle candidate.

  In addition, even if the above procedure is not regarded as a wrinkle candidate, as shown in FIG. 12, a line segment ps2-pe2 connecting two end points of two different feature determination regions B691 to B693 is set ( In order not to perform useless determination, it is desirable to set an upper limit based on the measurement result in the length of the line segment), and the length Lc of this line segment ps2-pe2 is compared with the specified value TL1. When the length Lc is equal to or greater than the specified value TL1, the average value Ld of the distances between the points on the skeleton SK of both feature determination regions B691 to B693 and the line segment ps2-pe2 is obtained. The ratio of the length Lc to the average value Ld (= Lc / Ld) is obtained, and when Lc / Ld is equal to or greater than the specified value TL2, one wrinkle may be divided into two feature determination regions B691 to B693 Therefore, the color difference between the feature determination areas B691 to B693 is obtained. That is, in a single channel image, the average value of the pixel values of both feature determination regions B691 to B693 is obtained, and if the difference between the average values of the pixel values is within a specified value, both feature determination regions B691 to B693 are one feature. Integrate as a judgment area.

  The above-described processing starts from the two feature determination regions B691 to B693 in which the length Lc of the line segment ps2-pe2 is the minimum, and the feature determination regions B691 to B693 in which the length Lc of the line segment ps2-pe2 increases in order. The evaluation is performed, and the process is repeated until the maximum length of the line segment ps2-pe2 that satisfies the condition is obtained. By this procedure, it is possible to obtain a feature determination region B69 that is integrated in order from the closest feature determination regions B691 to B693 and becomes the maximum within the range that can be integrated. FIG. 12 shows an example in which three feature determination regions B691 to B693 are integrated into one feature determination region B69 by the above-described processing.

  When the feature determination areas B61 to B610 are obtained by grouping the pixels, the feature amount is evaluated for each of the feature determination areas B61 to B610, and the type of skin trouble is classified (S7 in FIG. 1). When classifying skin troubles, the same inspection items (wrinkles, spots, darkening of pores, color unevenness) may also be classified in terms of degree (size and darkness) (S8 in FIG. 1). For example, if it is a wrinkle, you may classify | categorize about a depth and a magnitude | size.

When classifying skin troubles, first, feature amounts relating to the shape and distribution of pixel values are obtained for the feature determination regions B61 to B610. For example, as shown in FIG. 13, the feature amount of the shape of the feature determination regions B61 to B610 is the center of gravity G (coordinate position (x, y) in the image), area A, perimeter length P, and inertial principal axis direction D. The maximum width W in the direction orthogonal to the inertial principal axis direction D, the maximum length L in the inertial principal axis direction D, and the circularity R = 4πA / P ² (or flatness (= 1−R)) are used. Further, the average value and standard deviation of the pixel values are used as the feature values of the pixel value distribution. Further, a color difference from the average color may be used.

  In order to classify the feature determination areas B61 to B610 with respect to the type and degree of skin trouble, the criteria as shown in Table 1 are set, and the feature values obtained for the feature determination areas B61 to B610 match the criteria in Table 1. And can be classified into the types described at the left end of Table 1. If any item does not satisfy the criteria, the type of skin trouble is processed as “unknown”.

一例として、特徴判定領域Ｂ６１〜Ｂ６１０について求めた特徴量と、当該特徴量を表１の基準に適用して求めた肌トラブルの種別とを表２にまとめて示す。

As an example, Table 2 collectively shows the feature amounts obtained for the feature determination areas B61 to B610 and the types of skin troubles obtained by applying the feature amounts to the criteria in Table 1.

なお、表１、表２において、面積の単位は［ｍｍ^２］、周囲長、長さ、幅の単位はいずれも［ｍｍ］、方向の単位は［°］である。

In Tables 1 and 2, the unit of area is [mm ² ], the units of circumference, length, and width are all [mm], and the unit of direction is [°].

  By the way, if the reference values as shown in Table 1 are set in advance, it is easy to classify the types of skin troubles as shown in Table 2. However, the visual impression of the person and the determination result should be matched. It is not easy to set a reference value for each, and statistical work on a considerable number of measurement results is required to set an appropriate reference value.

  Therefore, it is desirable to use a neural network (neurocomputer) 25 for this classification. As shown in FIG. 14, the neural network used in the present embodiment is a three-layer neural network including an input layer Ly1, an intermediate layer Ly2, and an output layer Ly3. The input layer Ly1 has the number of features described above. The number of nodes in the intermediate layer Ly2 is larger than that in the input layer Ly1, and the number of nodes in the output layer Ly3 is the number of types of skin trouble types. Note that when classifying the type of skin trouble, it may not be possible to classify it into wrinkles, spots, darkening of pores, and color unevenness, so the node of the output layer Ly3 also includes “unknown”. That is, there are four types of skin troubles to be classified in the present embodiment, but the number of nodes in the output layer Ly3 is five.

  In order to classify the types of skin troubles by inputting feature values into this neural network, the neural network needs to be trained. In learning the neural network, the screen shown in FIG. 15 (displayed in the form of a dialog box) is displayed on the color monitor which is the output device 14, and necessary items are input using the mouse and keyboard which are the input devices 15. To do. As shown in FIG. 15, the screen is displayed for each type of skin trouble (in the illustrated example, the case where the skin trouble is wrinkled) is displayed. For each feature amount described above, fields 92 and 93 for inputting an upper limit value and a lower limit value are provided. In addition, in order to select a feature amount necessary to detect the skin trouble, a check box 91 is provided for each feature amount, and only the feature amount for which the check box 91 is checked detects the skin trouble. Used for. At the bottom of the screen, a “test” button 94, a “teach data registration” button 95, and a “neuro teaching” button 96 are displayed.

  Next, an operation for learning the neurocomputer 25 will be described. For each type of skin trouble, the operator selects a feature amount used for classification on the screen as shown in FIG. 15, and checks the check box 91 corresponding to the selected feature amount (by clicking the mouse on the check box 91). Check is turned on and off). Further, appropriate values are input in the fields 92 and 93 for inputting the upper limit value and the lower limit value of the feature amount, and the “test” button 94 is clicked with the mouse. By such an operation, the upper limit value and the lower limit value of the set feature value are used, and the upper limit value is set so that the determination result of the visual skin trouble matches the determination result using the upper limit value and the lower limit value of the feature value. Adjust the value and lower limit.

  When the type of feature amount used for classification of each skin trouble is determined and the adjustment of the upper limit value and the lower limit value is completed, the determination result is incorrect in the grayscale image captured by the imaging device 11 and displayed on the screen of the output device 14. Click with the mouse on the part that was on. By this mouse click, a pull-down menu in which skin trouble types are arranged is displayed on the screen. When the correct type is selected from the pull-down menu, the upper limit value and the lower limit value of each set feature amount are associated with the skin trouble type. When the “teach data registration” button 95 is clicked after the above-described operation is completed, data of a set of feature amounts and determination results obtained from each feature determination area is stored. After the teaching data is accumulated by performing this operation for a large number of people, a “neuro teaching” button 96 is clicked with a mouse, and weighting calculation of each node of the neural network is executed using the teaching data.

  As described above, after the upper limit value and the lower limit value are adjusted so that the determination result using the feature value matches the visual result by inputting appropriate upper limit value and lower limit value on the screen of the output device 14. The adjustment result is adjusted so that the adjusted upper limit value and lower limit value are correct, the teaching data is accumulated, and the weighting coefficient of the neural network is set by using a large number of teaching data. It can be used for classification of skin trouble types. That is, the neural network is learned to output a result close to the taught determination result to the output layer Ly3 with respect to the feature amount of the feature determination region input to the input layer Ly1, and thereafter, the upper limit value and the lower limit A determination result close to human visual determination can be obtained without manually setting the value.

  In addition to the color TV camera, a digital camera or a monochrome TV camera can be used for the imaging device 11, and a special camera that captures invisible light such as ultraviolet rays can be used. In addition, sine wave fringes (striped light patterns whose intensity distribution changes in a sine wave pattern) are projected onto the object to be inspected (from the deformation state of the sine wave fringes due to the unevenness of the surface of the object to be inspected) By using a phase shift three-dimensional measuring device for obtaining the distance information, a single channel image in which the pixel value becomes the height information may be acquired. The illuminating device 16 is not necessary because an ultraviolet light source is used when the imaging device 11 is an ultraviolet camera, and a signal source is provided when the imaging device 11 is a three-dimensional measuring device. Further, when a monochrome TV camera, an ultraviolet camera, or a three-dimensional measuring device is used, it is originally a single channel image, so that it is converted into a single channel image in step S2 in FIG. 1 (that is, steps S11 to S17 in FIG. 2). No processing is required. Of the skin troubles, wrinkles have three-dimensional information, so the three-dimensional measuring device is particularly effective in detecting wrinkles.

As a method for generating a single channel image, instead of the method described above, in the color space of the L ^* a ^* b ^* color system, an average color of the background portion on the a ^* b ^* plane is obtained, and each pixel is related. A single channel image having the difference between the color (a ^* , b ^* ) and the average color (that is, the distance on the a ^* b ^* plane) as a pixel value may be generated. In particular, it is better to employ this method than to employ a method using a color separation axis in order to extract color unevenness with little change in brightness. Therefore, skin troubles that involve changes in both brightness and color, such as spots, wrinkles, and darkening of pores, are identified by processing using the color separation axis, and color unevenness that does not significantly change in brightness is extracted. May be used in combination, as determined by processing that does not use the color separation axis.

  In the above-described embodiment, when generating a feature determination region by grouping connected regions, it is determined whether or not to be a feature determination region based on a contrast value that is a difference between an internal average value and an external average value. If the resolution of the grayscale image obtained by the imaging device 11 is sufficiently high and the number of pixels in the connected area can be taken to a statistically significant level, a statistically significant difference between the inside and outside of the connected area You may make it perform the test of a population average difference about presence or absence. Similarly, the color difference is used when determining whether or not to divide the divided feature determination areas, but in this case as well, a test of the population average difference may be performed.

It is operation | movement explanatory drawing which shows embodiment of this invention. It is operation | movement explanatory drawing which shows the production | generation process of a single channel image in the same as the above. It is a schematic block diagram same as the above. It is a figure which shows the example of the gray image in the same as the above. It is a figure which shows the example of the image after the shading removal process in the same as the above. It is a figure which shows the example of the binary image by the temporary threshold value same as the above. It is a figure which shows the setting concept of a color separation axis | shaft in the same as the above. It is a figure which shows the concept of the connection area | region in the same as the above. It is a figure which shows the concept of the feature determination area | region in the same as the above. It is a figure which shows the example of a production | generation of the feature determination area | region in the same as the above. It is a figure which shows the other example of a production | generation of the characteristic determination area | region in the same as the above. It is a figure which shows the other example of a production | generation of the feature determination area | region in the same as the above. It is a figure which shows the feature-value of the shape calculated | required from the feature determination area | region in the same as the above. It is a figure which shows an example of the neural network used for the same as the above. It is a figure which shows the example of a screen used for the teaching of the neural network used for the same as the above.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 Imaging device 12 Memory | storage device 13 Image processing apparatus 14 Output device 15 Input device 16 Illumination device 51,52 Pixel group 53 Color separation axis A1-A4 Connection area | region B61-B610 Feature determination area CA631-CA635 Contrast value H Human body SK Skeleton

Claims (6)

  For a single channel image obtained by photographing an object to be inspected, after generating a connected region composed of pixels whose pixel value difference between adjacent pixels is equal to or less than a first specified value, a connected candidate for a feature portion A feature determination region is generated by integrating connected regions using distribution of pixel values inside and outside the region, and feature parts are classified into a plurality of types by classifying feature quantities obtained from the feature determination region. Appearance inspection method using images.   For a single channel image obtained by photographing a human skin as an object to be inspected, after generating a connected region composed of pixels in which the difference in pixel value between adjacent pixels is equal to or less than a first specified value, the connected region A feature determination area is generated by separating a feature portion that is a candidate for skin trouble from a background portion that is normal skin using a distribution of pixel values inside and outside the image, and integrating a connected region that is a feature portion, and from the feature determination region An appearance inspection method using an image, wherein skin troubles are classified into a plurality of types by classifying the obtained feature values.   The image obtained by photographing the object to be inspected is a color image, and after the shading removal process is performed on the color image, the color image is binarized using a temporary threshold set for at least one of brightness and color. To separate the provisional feature portion and the provisional background portion, and project each pixel of the color image on the color separation axis set in the color space so as to pass through the provisional feature portion, the provisional background portion, and each pixel group. The image inspection method according to claim 1 or 2, wherein an image having a pixel value at a position on the color separation axis is generated by the method and the image is used for the single channel image.   When generating the feature determination area by integrating the connected areas, for each candidate connected area to be integrated, an average value of pixel values inside the connected area and an average value of pixel values of the background portion around the connected area When the difference is larger than the second specified value, the connected area is used as a feature part. When the difference is smaller than the second specified value, the connected area is used as a background part. The visual inspection method using an image according to any one of claims 1 to 3, wherein a set of connected regions as a part is integrated as a feature determination region.   A skeleton is obtained for the feature determination region, and when the length of a line segment connecting the end points of two skeletons of two different feature determination regions is equal to or greater than a third specified value, each of the skeletons of both feature determination regions Both feature determinations when the ratio of the length of the line segment to the average value of the distance between the pixel and the line segment is equal to or greater than a fourth specified value and the color difference between both feature determination regions is equal to or less than the fifth specified value 5. The method according to claim 4, wherein the regions are integrated into one feature determination region.   When classifying into a plurality of types by a neural network that receives a plurality of types of feature values obtained from the feature determination area, a reference value for classifying feature values, a classification result based on the reference value, and a classification correction result are interactively displayed. 6. The visual inspection method using an image according to any one of claims 1 to 5, wherein the teaching data input to is stored and a neural network is taught using the stored teaching data.

Images