JP2022091065A - Color determination device and color determination method - Google Patents

Abstract

To provide a color determination device and color determination method which can determine whether an object is in a solid color or in a non-solid color easily in a short time.SOLUTION: A color determination device comprises: imaging means 10 which images a determination object M so as to include a curved surface part; image cutting means 20 which cuts off a determination image of a determination range including the curved surface part of the determination object M from a photographed image obtained by the imaging means 10; and color determination means 40 which determines whether the determination object M is in a solid color or in a non-solid color on the basis of a color determination model 41 which has learned a feature by executing machine learning using a learning image that becomes a positive example of the determination range and a determination image obtained by the image cutting means 20.SELECTED DRAWING: Figure 1

Description

The present invention relates to a color discriminating device and a color discriminating method for discriminating whether the color of a discriminating object is a solid color containing no bright material or a non-solid color containing a bright material.

In the production of automobiles, for example, parts such as mirror covers to be attached after assembling a car body are inspected for correct colors. As an inspection method, for example, an image of an attached part is taken, and color information (HSV value) is acquired from the obtained image to determine the color of the part. However, since the value of the color information is close to the solid color of similar colors and the non-solid color such as metallic color or pearl color (mica color), it is difficult to distinguish by the color information. Therefore, conventionally, the metallic color is visually discriminated by utilizing the flip-flop property in which the brightness changes depending on the viewing angle. However, skill is required for visual discrimination, and there is a concern that an amateur may make a mistake in making a correct / incorrect judgment.

Further, as a method of discriminating between a solid color and a non-solid color, for example, an inspection object is photographed from a plurality of angles with a camera having spectral sensitivity, and two xy chromaticity histogram distributions or XYZ colors of the inspection object and the reference object are taken. A method of making a judgment by calculating a color distribution match index indicating the overlapping ratio of the degree histogram distribution has also been proposed (see, for example, Patent Document 1). However, the method described in Patent Document 1 requires a camera having spectral sensitivity, so that there is a problem that the equipment cost is high. Further, since it is necessary to photograph the inspection object from a plurality of angles, there is a problem that it takes time to determine one place.

Japanese Patent No. 6038965

The present invention has been made based on such a problem, and provides a color discriminating device and a color discriminating method capable of easily discriminating between a solid color and a non-solid color in a short time. The purpose is.

The color discriminating device of the present invention is based on a photographing means for photographing an object to be discriminated and an image obtained by the photographing means, and whether the color of the object to be discriminated is a solid color containing no bright material or containing a bright material. The color discriminating means is provided with a color discriminating means for discriminating whether the color is a non-solid color, and the color discriminating means executes machine learning using a learning image which is a positive example of a captured image to perform machine learning depending on the presence or absence of a bright material and the type of reflected light. The characteristics of the above are discriminated by a trained color discrimination model.

The color discrimination method of the present invention is based on a photographing procedure for photographing an object to be discriminated and an image obtained in the photographing procedure, and whether the color of the object to be discriminated is a solid color containing no bright material or containing a bright material. The color discrimination procedure includes a color discrimination procedure for discriminating which of the non-solid colors is used, and in the color discrimination procedure, machine learning is performed using a learning image that is a positive example of the captured image, and the reflected light depending on the presence / absence and type of the bright material is executed. The characteristics of the above are discriminated by a trained color discrimination model.

According to the present invention, machine learning using a learning image as a positive example is performed to discriminate by a color discrimination model trained in the presence / absence of a bright material and the characteristics of reflected light depending on the type. Therefore, it is easy to make a solid. It is possible to determine whether it is a color or a non-solid color. Therefore, it is possible to inspect in a flowing state without stopping the production line.

In addition, the part including the curved surface portion of the discrimination target is set as the discrimination range, and the color discrimination means discriminates based on the discrimination image including the curved surface portion. It is possible to easily determine whether the color is a solid color or a non-solid color by using. Therefore, it can be easily discriminated by photographing the portion including the curved surface portion once.

It is a figure which shows the structure of the color discrimination apparatus which concerns on 1st Embodiment of this invention. It is a figure explaining the detection of the discriminant object in the image photographed by the photographing means. It is a figure which shows the result of having visualized the learning result which machine-learned about the solid color of white. It is a figure which shows the result of having visualized the learning result which machine-learned about the metallic color of white. It is a figure which shows the result of having visualized the learning result which machine-learned about the solid color of black. It is a figure which shows the result of having visualized the learning result which machine-learned about the metallic color of black. It is a figure explaining the difference of the reflected light depending on the presence or absence of a bright material. It is a figure which shows one configuration example of the hardware of the basic color discriminating means, the image cutting means, and the color discriminating means. It is a figure which shows the flow of the color discrimination method which concerns on 1st Embodiment of this invention. It is a figure which shows the structure of the color discrimination apparatus which concerns on 2nd Embodiment of this invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(First Embodiment)
FIG. 1 shows the configuration of the color discriminating device 1 according to the first embodiment of the present invention. The color discriminating device 1 discriminates whether the color of the discriminating object M is a solid color or a non-solid color. A solid color is a color that does not contain a bright material, and a non-solid color is a color that contains a bright material. Examples of the non-solid color include a metallic color and a pearl color (also referred to as mica color). In the present embodiment, for example, in the production process of a car, the mirror cover is set as the discrimination object M, and it is determined whether the color of the mirror cover attached to the vehicle body is a solid color or a non-solid color. explain.

The color discrimination device 1 includes, for example, a photographing means 10 for photographing the discrimination object M, an image cutting means 20 for cutting a discrimination image from the captured image obtained by the photographing means 10, and a captured image obtained by the photographing means 10. Based on the basic color discriminating means 20 for discriminating which of the basic colors the color of the discriminating object M is, and the captured image obtained by the photographing means 10, the color of the discriminating object M is a solid color. It is provided with a color discriminating means 40 for discriminating whether the color is a non-solid color or a non-solid color.

The photographing means 10 is configured by, for example, a color camera, and is fixedly arranged so as to photograph a mirror cover which is a discrimination object M attached to a vehicle body transported on a production line. It is preferable that the photographing direction and the photographing range of the photographing means 10 are set so that the curved surface portion of the discrimination object M is captured and the captured image includes the curved surface portion. Since the observation angle of the curved surface portion 10 changes depending on the position, the change in brightness on the curved surface portion differs depending on whether it is a solid color or a non-solid color due to the flip-flop property in which the brightness changes depending on the viewing angle. This is because it is possible to extract the characteristics of the reflected light depending on the presence / absence and type of the bright material.

It is preferable that the photographing range of the photographing means 10 includes a region around the curved surface portion of the discrimination object M, and a wide range including a part of the vehicle body may be photographed. Since the position of the vehicle body transported on the production line varies depending on the individual, this is to ensure that the curved surface portion of the discrimination object M can be photographed. Further, the number of photographing means 10 may be one, but for example, when there are a plurality of discrimination objects M, a plurality of shooting means 10 may be provided corresponding to each discrimination target M.

The image cutting means 20 can be configured by, for example, a computer, and is configured to function as the image cutting means 20 by executing a program. Specifically, the image cutting means 20 is connected to, for example, the photographing means 10, the portion including the curved surface portion of the discrimination object M is set as the discrimination range, and the discrimination range is discriminated from the captured image obtained by the photographing means 10. It is preferably configured to be cropped as an image. This is because it is possible to extract the characteristics of the reflected light depending on the presence / absence and type of the bright material on the curved surface portion.

Specifically, the image cutting means 20 is, for example, a discrimination range between the object detecting means 21 that detects the discriminant object M from the captured image input from the photographing means 10 and the discriminating object M detected by the object detecting means 21. It is preferable to have a discrimination range determining means 22 for cutting out the image as a discrimination image. The position and angle of the vehicle body transported on the production line fluctuate depending on the individual, and the position and angle of the discrimination object M in the photographed image photographed by the photographing means 10 fluctuate depending on the individual. This is because it is not possible to accurately cut out the discrimination range by cutting out.

The object detection means 21 can use a known object detection technique, for example, object detection learning in which features are learned by executing machine learning using a learning image that is a positive example of the discriminant object M to be detected. It is preferable to use a model. The discrimination range determining means 22 is preferably configured to, for example, cut out the discrimination range including the curved surface portion from the image portion of the discrimination target object M detected by the object detection means 21 as a discrimination image. FIG. 2 shows an example of a portion detected as the discrimination object M in the captured image captured by the photographing means 10 and a portion cut out as the discrimination image. In FIG. 2, the range detected as the discrimination target object M detected by the object detecting means 31 is shown by a broken line, and the range of the discrimination image cut out by the discrimination range determining means 32 is shown by a dashed line.

The basic color discriminating means 20 can be configured by, for example, a computer, and is configured to function as the basic color discriminating means 20 by executing a program. Specifically, the basic color discrimination means 20 is connected to, for example, the image cutting means 20, and acquires color information (HSV value) from the discrimination image cut by the image cutting means 20, and is preset from the value. It is configured to determine which of the plurality of basic colors it is. The basic color is a basic color and does not distinguish between a solid color and a non-solid color. Examples of the basic color include white, black, gray, red, yellow, green, blue, orange, purple, light blue, and pink.

The color discriminating means 40 can be configured by, for example, a computer, and is configured to function as the color discriminating means 40 by executing a program. The color discrimination means 40 is based on the color discrimination model 41 in which the characteristics of the reflected light depending on the presence or absence of the bright material are learned by executing machine learning using the learning image which is a positive example of the captured image, and the captured image. , Whether the color of the object M to be discriminated is a solid color or a non-solid color is discriminated.

Specifically, the color discrimination means 40 includes, for example, a color discrimination model 41 whose features are learned by executing machine learning using a learning image which is a positive example of the discrimination range, and a discrimination image in the color discrimination model 41. Is input, and a determination unit 42 for determining whether the color is a solid color or a non-solid color based on the output value obtained by the input is provided. Further, it is preferable that the color discrimination means 40 has, for example, a generation means 43 for generating the color discrimination model 41.

The color discrimination model 41 can be generated by the generation means 43. The generation means 43 prepares, for example, a plurality of regular learning images in which the discrimination target M is photographed by the photographing means 10 and the discrimination range is cut by the image cutting means 30, and deep learning (deep learning) is performed. It is preferable that it is configured to extract features by. For deep learning, it is preferable to use a convolutional neural network (CNN), and for example, it is preferable to use about 300 to 500 learning images. Further, it is preferable that the color discrimination model 41 has an individual model in which the characteristics of the reflected light depending on the presence / absence and type of the bright material are learned for each basic color, for example.

3 to 6 show the gradient weighted class activation mapping (Grad-CAM) for the training results trained by the convolutional Neilal network using 50 training images for each of the solid color and metallic color of white and black. ) The result visualized by the method is shown. FIG. 3 shows the learning result of the white solid color, FIG. 4 shows the learning result of the white metallic color, FIG. 5 shows the learning result of the black solid color, and FIG. 6 shows the learning result of the black metallic color. As shown in FIGS. 3 to 6, it can be seen that the extracted features differ between the solid color and the metallic color. That is, by deep-layer learning using learning images of solid color and non-solid color, it is possible to extract different characteristics of reflected light depending on the presence and type of bright material, thereby distinguishing between solid color and non-solid color. I know I can do it.

The difference in the characteristics of the learning result is due to the fact that the change in brightness in the curved surface portion differs depending on the presence and type of the bright material, and the different characteristics of the reflected light are extracted by this. FIG. 7 shows the difference in the reflected light depending on the presence / absence and type of the bright material. 7A and 7B show the reflected light when the discriminant object M having the coating film M1 formed on the surface is irradiated with light, FIG. 7A is a solid color, and FIG. 7B is a non-solid color. This is the case.

In the case of a solid color, as the reflected light when the discrimination object M is irradiated with light, the mirror surface reflected light L1 reflected in the normal reflection direction and the intralayer diffused reflected light L2 reflected in all directions can be obtained. .. The specular reflected light L1 is an illumination color, and the diffuse reflected light L2 in the layer is the color (basic color) of the coating film M1. Therefore, the color of the discriminant object M does not change depending on the angle at which the discriminant object M is viewed. On the other hand, in the case of the non-solid color, the reflected light is the mirror reflected light L1 reflected in the normal reflection direction and the intra-layer diffused reflected light L2 reflected in all directions, which are the largest in the normal reflection direction. A bright material reflected light L3 that becomes smaller as the distance from the normal reflection direction increases is obtained. The mirror-reflected light L1 is an illumination color, the intra-layer diffused reflected light L2 is the color (basic color) of the coating film M1, and the bright material reflected light L3 is the reflected light by the bright material. Therefore, the color of the discriminant object M changes depending on the angle at which the discriminant object M is viewed, and the degree of change varies depending on the type of the bright material. In FIG. 7, the specular reflected light L1 is represented by a gray background, the intralayer diffuse reflected light L2 is represented by a white background, and the bright material reflected light L3 is represented by a satin background.

Therefore, by using the curved surface portion where the viewing angle of the discriminant object M changes, the color discrimination model 41 can be trained by using the difference in the reflected light depending on the presence / absence and type of the bright material. It is possible to determine whether the color is a solid color or a non-solid color. Further, since the color discrimination device 1 discriminates by using the difference in the characteristics of the reflected light on the curved surface portion, it is possible to reduce the influence of disturbance such as the reflection of a person or the reflection of lighting, and the discrimination is performed with high accuracy. You can do it.

The determination unit 42 is connected to, for example, the basic color discrimination means 20, the discrimination range determining means 32, and the color discrimination model 41, respectively. For example, the discrimination unit 42 selects an individual model corresponding to the basic color discriminated by the basic color discrimination means 20 as the color discrimination model 41, and inputs the discrimination image obtained by the discrimination range determining means 32 into the selected individual model. It is preferable that it is configured to do so. Further, the determination unit 42 is configured to determine that the color has the higher degree of matching based on the output value obtained from the color discrimination model 41, for example, the score values of the solid color and the non-solid color. Is preferable.

FIG. 8 shows an example of the hardware configuration of the basic color discriminating means 20, the image cropping means 30, and the color discriminating means 40. The basic color discriminating means 20, the image cutting means 30, and the color discriminating means 40 are, for example, a CPU (Center Processing Unit) 51, a ROM (Read Only Memory) 52, a RAM (Random Access Memory) 53, and an HDD (hard disk). It has a drive) 54 and an operation interface (operation I / F) 55. The CPU 51 executes various processes according to various programs recorded in the ROM 52 or various programs loaded from the HDD 54 into the RAM 53. The RAM 53 also appropriately stores data and the like necessary for the CPU 51 to execute various processes. Various data are stored in the HDD 54.

This color discriminating device 1 is used, for example, as follows. FIG. 9 shows the flow of the color discrimination method using the color discrimination device 1. First, as a preparation procedure, the color discrimination model 41 is generated using the generation means 43 (preparation procedure; step S110). Specifically, for example, a mirror cover, which is a discrimination target object M, is photographed by the photographing means 10 on a production line, and the discrimination target object M is detected by the object detecting means 21 from the obtained photographed image, and the discrimination range determination means. The discrimination range including the curved surface portion of the discrimination target object M is cut out by 22, a plurality of learning images serving as positive examples of the discrimination range are prepared, and the color discrimination model 41 is generated by deep learning. At that time, as the color discrimination model 41, it is preferable to generate an individual model in which the characteristics of the reflected light depending on the presence / absence and type of the bright material are learned for each basic color.

Next, the color of the mirror cover, which is the discrimination object M attached to the vehicle body transported on the production line, is discriminated as follows. First, the object to be discriminated M is photographed by the photographing means 10 fixed and arranged (imaging procedure; step S121). At that time, as described above, the image is taken so that the curved surface portion of the discrimination object M is included. Next, for example, the image cutting means 20 cuts the discrimination range including the curved surface portion of the discrimination target M from the captured image obtained by the photographing means 10 as a discrimination image (image cutting procedure; step S122). Specifically, for example, the object detecting means 21 detects the discriminant object M from the captured image obtained by the photographing means 10, and the discriminating range determining means 22 cuts out the discriminating range of the discriminating object M.

Subsequently, for example, the basic color discriminating means 30 acquires color information from the discriminating image cut by the image cutting means 20, and discriminates which of the plurality of preset basic colors is used (basic color discriminating procedure; Step S123). After that, the color discrimination means 40 determines whether the color of the discrimination target M is a solid color or a non-solid color based on the color discrimination model 41 and the discrimination image obtained by the image cropping means 20. Discrimination (color discrimination procedure; step S124). Specifically, for example, the discrimination unit 42 selects an individual model which is a color discrimination model 41 corresponding to the basic color input from the basic color discrimination means 30, inputs a discrimination image, and uses the color discrimination model 41. Based on the obtained output value, it is determined that the color has a high degree of matching, either a solid color or a non-solid color.

Using this color discrimination device 1, a mirror cover attached to the vehicle body was used as the discrimination target M, and a color inspection was performed for the case where the basic color was white and the case where the basic color was black. The inspection was carried out in a state where the discriminant object M was being conveyed without stopping the production line. The number of car bodies inspected is 350 for white and 500 for black. As a result, the discrimination accuracy was 100%.

As described above, according to the present embodiment, the color discrimination model 41 is used to learn the characteristics of the reflected light depending on the presence or absence of the bright material by performing machine learning using a learning image as a positive example. , It is possible to easily determine whether the color is a solid color or a non-solid color. Therefore, it is possible to inspect in a flowing state without stopping the production line.

Further, since the part including the curved surface portion of the discrimination target M is set as the discrimination range and the color discrimination means 40 discriminates based on the discrimination image including the curved surface portion, the characteristic of the reflected light depending on the presence or absence of the bright material in the curved surface portion. It is possible to easily determine whether the color is a solid color or a non-solid color by using. Therefore, it can be easily discriminated by photographing the portion including the curved surface portion once.

(Second embodiment)
In the first embodiment, the portion including the curved surface portion of the discrimination target M is set as the discrimination range, and the curved surface portion has different characteristics of the reflected light depending on the presence or absence of the bright material, so that the color is solid or non-solid. Although the case of discriminating which of the above is used has been described, for example, it is utilized that the reflected light when the discrimination target M is irradiated with light differs depending on the presence / absence and type of the bright material according to the light receiving angle of the photographing means 10. It can also be determined by

FIG. 10 shows a configuration example of the color discriminating device 2 according to the present embodiment. The color discrimination device 2 has the same configuration as that of the first embodiment except that the captured image (specifically, the discrimination image) used for discrimination is different. Therefore, the common components will be described with the same reference numerals, and the description of the common parts will be omitted.

The light discrimination device 2 includes, for example, a light irradiation means 60 for irradiating the discrimination target M with light, in addition to the photographing means 10, the image cutting means 20, the basic color discrimination means 30, and the color discrimination means 40. Is preferable. The light irradiation means 60 is preferably configured by, for example, an LED spotlight. Further, it is more preferable to have a shielding means for shielding light other than the light irradiating means 60.

As shown in FIG. 10A, for example, the light irradiating means 60 and the photographing means 10 are arranged by fixing one light irradiating means 60, and the light irradiating means 60 irradiates the object M to be discriminated. A plurality of (for example, two) photographing means 10 for capturing the light reflected by the above may be fixedly arranged at positions having different light receiving angles. Further, for example, as shown in FIG. 10B, one light irradiation means 60 is fixedly arranged and one photographing means 10 is arranged so as to be movable at a position where the light receiving angle is different. You may do so.

Further, for example, as shown in FIG. 10C, a plurality of (for example, two) light irradiation means 60 are fixedly arranged at positions having different incident angles, and one photographing means 10 is fixed. It may be arranged. In addition, for example, as shown in FIG. 10D, one light irradiation means 60 is arranged so as to be movable at positions having different incident angles, and one photographing means 10 is fixedly arranged. You may try to do it. Furthermore, for example, as shown in FIG. 10 (E), a plurality of discrimination objects M are used by one light irradiation means 60 by using a plurality of (for example, two) surfaces having different observation angles by the photographing means 10. The surface of the surface may be irradiated with light, and the reflected light from a plurality of surfaces may be imaged by one photographing means 10.

Even with this configuration, a color discrimination model that learns the characteristics of reflected light depending on the presence and type of bright material by performing machine learning using the image captured by the imaging means 10 and the learning image that is a positive example thereof. With 41, it is possible to determine whether the color is a solid color or a non-solid color. The image cutting means 20 is configured to, for example, cut out a portion including a portion irradiated with light by the light irradiating means 60 from the captured image obtained by each photographing means 10 as a discrimination range. Is preferable.

The color discriminating device 2 can be used, for example, in the same manner as in the first embodiment, and can obtain the same effect as in the first embodiment.

Although the present invention has been described above with reference to embodiments, the present invention is not limited to the above embodiments and can be variously modified. For example, in the above-described embodiment, each component has been specifically described, but the specific structure and shape of each component may be different, and all of the above-mentioned components may not be provided. , Other components may be provided.

Further, in the above embodiment, the discrimination image is cut out by the image cutting means 20 from the shot image obtained by the shooting means 10, and the basic color of the discrimination target M is discriminated by the basic color discriminating means 30 for the discriminated image. Further, although the case of discriminating whether the color is a solid color or a non-solid color by the color discriminating means 40 has been described, different images may be used between the basic color discriminating means 30 and the color discriminating means 40.

In addition, in the above embodiment, a case where the basic color of the discrimination object M is discriminated by the basic color discriminating means 30 and whether it is a solid color or a non-solid color is discriminated by the color discriminating means 40 will be described. However, the color discriminating means 40 may discriminate the basic color and discriminate whether it is a solid color or a non-solid color. In that case, for example, in the color discrimination model 41, it is preferable to learn the characteristics of the basic color and the solid color / non-solid color type.

1,2 ... Color discriminating device, 10 ... Shooting means, 20 ... Image cutting means, 21 ... Object detecting means, 22 ... Discrimination range determining means, 30 ... Basic color discriminating means, 40 ... Color discriminating means, 41 ... Color discriminating model , 42 ... determination unit, 43 ... generation means, 51 ... CPU, 52 ... ROM, 53 ... RAM, 54 ... HDD, 55 ... operation interface, 60 ... light irradiation means

Claims (4)

Shooting means to shoot the object to be discriminated,
Based on the photographed image obtained by the photographing means, a color discriminating means for determining whether the color of the discriminating object is a solid color containing no bright material or a non-solid color containing a bright material is provided. Prepare,
The color discrimination means is characterized in that it discriminates by a color discrimination model in which machine learning using a learning image which is a positive example of the captured image is executed and the characteristics of reflected light depending on the presence / absence and type of a bright material are learned. Color discriminator. A basic color discriminating means for discriminating which of the basic colors the color of the discriminating object is based on the captured image obtained by the photographing means is provided.
The color discriminating means has, as the color discriminating model, an individual model in which the characteristics of reflected light depending on the presence or absence of a bright material are learned for each basic color, and corresponds to the basic color discriminated by the basic color discriminating means. The color discriminating device according to claim 1, wherein an individual model is used to discriminate whether the color is a solid color or a non-solid color. A portion including a curved surface portion of the discrimination object is set as a discrimination range, and an image cutting means for cutting the discrimination range as a discrimination image from a shot image obtained by the shooting means is provided.
The color discriminating device according to claim 1 or 2, wherein the color discriminating means discriminates whether the color is a solid color or a non-solid color based on the discriminating image. The shooting procedure for shooting the object to be discriminated and
Based on the photographed image obtained in the photographing procedure, a color discrimination procedure for determining whether the color of the object to be discriminated is a solid color containing no bright material or a non-solid color containing a bright material is provided. Including,
The color discrimination procedure is characterized in that machine learning is performed using a learning image that is a positive example of the captured image, and discrimination is performed by a color discrimination model that learns the characteristics of reflected light depending on the presence / absence and type of bright material. Color discrimination method.

Images