JP2022147313A - Image learning device for visual inspection and visual inspection device - Google Patents

Abstract

To provide an image learning device for visual inspection which can accelerate determination and simplify processing, and a visual inspection device using the same.SOLUTION: An image learning device for visual inspection having one learning unit in an image learning system for visual inspection which learns inspection information being an inspection result on whether or not the appearance of an article corresponding to image data for visual inspection of the article is either normal or abnormal by the different learning units for respective inspection points, comprises: an image input unit which receives input of the image data including a visual inspection spot of the article; and an inspection information input unit which receives input of inspection information being an inspection result on whether or not the appearance of the article corresponding to the image data is either normal or abnormal. The learning unit learns association between the article in at least one inspection point and the inspection result on the basis of the image data and the inspection information.SELECTED DRAWING: Figure 3

Description

The present disclosure relates to a visual inspection image learning device for learning image data for visual inspection of an article, and a visual inspection device using the same.

Conventionally, machine learning image learning devices for visual inspection using AI (Artificial Intelligence) and visual inspection devices acquire multiple data sets for learning, each containing a combination of image data and correct data. Set the difficulty of judgment for each training data set according to the degree to which the result of judging the quality of the product reflected in the image data of the training data set by the first classifier matches the correct answer indicated by the correct data. (See, for example, Patent Literature 1).

WO2019/176989

However, the conventional machine learning has a problem that it does not take into account speeding up of determination of learning data set and simplification of processing.

The present disclosure has been made to solve the above problems, and aims to obtain an image learning device for visual inspection that can speed up determination and simplify processing, and a visual inspection device using the same. and

An image learning apparatus for visual inspection according to the present disclosure inspects inspection information, which is an inspection result indicating whether the appearance of an article is normal or abnormal, corresponding to image data for visual inspection of the article. An image learning device for visual inspection having one learning unit in an image learning system for visual inspection in which learning is performed by different learning units for each viewpoint, wherein the image input includes the image data including the visual inspection portion of the article. and an inspection information input unit for inputting inspection information, which is an inspection result indicating whether the appearance of the article corresponding to the image data is normal or abnormal, wherein the learning unit receives the image data and the inspection information, learning the association between the article and the inspection result in at least one of the viewpoints to be inspected.

The appearance inspection apparatus according to the present disclosure provides inspection information, which is an inspection result indicating whether the appearance of the article corresponding to image data for inspecting the appearance of the article is normal or abnormal, for each inspection viewpoint. A visual inspection image learning device having one learning unit in a visual inspection image learning system that learns by different learning units, wherein the image input unit receives the image data including the visual inspection location of the article; an inspection information input unit for inputting inspection information, which is an inspection result indicating whether the appearance of the article corresponding to the image data is normal or abnormal, wherein the learning unit receives the image data and the inspection; A visual inspection apparatus using a learning result of a visual inspection image learning apparatus characterized by learning an association between the article and the inspection result in at least one of the viewpoints to be inspected based on the information. a new image input unit for inputting new image data obtained by newly reading the article; and based on the learning result learned by the learning unit, whether the appearance of the article corresponding to the new image data is normal or abnormal. and an image judgment unit for judging from the inspection information.

The appearance inspection apparatus according to the present disclosure provides inspection information, which is an inspection result indicating whether the appearance of the article corresponding to image data for inspecting the appearance of the article is normal or abnormal, for each inspection viewpoint. A visual inspection image learning device having one learning unit in a visual inspection image learning system that learns by different learning units, wherein the image input unit receives the image data including the visual inspection location of the article; an inspection information input unit for inputting inspection information, which is an inspection result indicating whether the appearance of the article corresponding to the image data is normal or abnormal, wherein the learning unit receives the image data and the inspection; A plurality of visual inspection image learning devices are prepared for each of the inspected viewpoints, characterized by learning an association between the article and the inspection result in at least one of the inspected viewpoints based on the information, A visual inspection apparatus using learning results for each viewpoint to be inspected, comprising: a new image input unit for inputting new image data obtained by newly reading the article; and based on the learning result learned by the learning unit, the and an image judgment unit for judging from the inspection information whether the appearance of the article corresponding to the new image data corresponding to the new image data for each learning unit is normal or abnormal.

According to the present disclosure, an image learning apparatus for visual inspection that can be speeded up and simplified by learning associations between an article and the inspection results in at least one aspect of inspecting the article (defective mode). can be obtained.

1 is a functional block diagram of an appearance inspection image learning apparatus (appearance inspection image learning system) according to Embodiment 1; FIG. 1 is a functional block diagram of an appearance inspection image learning apparatus (appearance inspection image learning system) according to Embodiment 1; FIG. 5 is a flow chart for explaining the operation of the appearance inspection image learning apparatus (appearance inspection image learning method) according to Embodiment 1; 1 is a functional block diagram of a visual inspection apparatus according to Embodiment 1; FIG. 1 is a functional block diagram of a visual inspection apparatus according to Embodiment 1; FIG. 4 is a flowchart for explaining the operation (appearance inspection method) of the appearance inspection apparatus according to Embodiment 1; 1 is a functional block diagram of a visual inspection apparatus according to Embodiment 1; FIG. 4 is a flowchart for explaining the operation (appearance inspection method) of the appearance inspection apparatus according to Embodiment 1;

Embodiment 1.
The image learning system for visual inspection, the image learning device for visual inspection, and the visual inspection device using them (the visual inspection device according to the first embodiment) according to the first embodiment will be described below with reference to FIGS. 1 to 7. explain. Specifically, the image learning device for visual inspection uses the inspection information, which is the result of inspection as to whether the appearance of the article corresponding to the image data for visual inspection of the article, is either normal or abnormal, from the viewpoint of inspecting ( It has one learning unit in an image learning system for visual inspection that learns with a different learning unit for each failure mode.

1 and 2 are diagrams for explaining a visual inspection image learning system and a visual inspection image learning apparatus according to Embodiment 1. FIG. FIG. 3 is a diagram for explaining the operation of the visual inspection image learning apparatus according to Embodiment 1 (visual inspection image learning method according to Embodiment 1). 4, 5, and 7 are diagrams for explaining the visual inspection apparatus according to the first embodiment. 6 and 8 are diagrams for explaining the operation of the visual inspection apparatus according to Embodiment 1 (visual inspection method according to Embodiment 1). In the drawings, the same reference numerals denote the same or corresponding parts, and detailed descriptions thereof are omitted.

In FIGS. 1 and 2, a visual inspection image learning system 1 and a visual inspection image learning device 2 learn image data for visual inspection of an article, and construct a learning model (embodiment 1). The article can be a pattern on the board or a part mounted on the board, such as an electronic part or a screw. The visual inspection image learning system 1 shown in FIG. 1 is provided with a plurality of visual inspection image learning devices 2 for each inspection viewpoint (defect mode). In the visual inspection image learning system 1 shown in FIG. 2, the image input unit 3 and the inspection information input unit 4 of the visual inspection image learning device 2 are shared for each inspection viewpoint (defect mode). 2 has a plurality of learning units 5. FIG.

An example of an article (image data of an article) is an image showing a state in which a screw is attached to a substrate via a washer. In this example, the article would be a screw or a screw and washer. Also, the appearance being normal means that the screws and washers are correctly attached to the board. Similarly, abnormal appearance means that screws and washers are not correctly attached to the board. Here, the viewpoint of inspecting the appearance (defective mode) is a viewpoint from which even if the image data is the same, the parts for judging whether the appearance of the article is normal or abnormal are classified. Viewpoints (defective modes) to be inspected include, for example, presence/absence of attachment of an article, difference in the type of article, attachment position of the article, tilt of the article, positional relationship between the article to which the article is attached and the article, damage to the article, etc. be done. Also, the same can be considered when there are two or more types of articles in the image data. For example, if a screw, a board pattern, and a soldered component are included in the same image, the appearance inspection may include the mounting state of the screw, the board pattern state, and the soldered state of the component.

In FIGS. 1 and 2, the image input unit 3 receives image data including an appearance inspection portion of an article. When there are two or more types of articles in the image data, the image input unit 3 can be said to include a plurality of articles in the input image data. The inspection information input unit 4 receives inspection information, which is an inspection result indicating whether the appearance of the article corresponding to the image data is normal or abnormal. The learning unit 5 is provided for each inspection viewpoint (defect mode) in the visual inspection image learning system 1 . Therefore, the learning unit 5 learns the association between the article and the inspection result in at least one inspection viewpoint (defect mode) based on the image data and the inspection information. In the case of image data containing two or more types of articles, different inspection viewpoints (defective modes) may be set for each type of article, or the same inspection viewpoints (defective modes) may be set. good.

For example, the learning unit 5 determines whether or not an article is attached, the difference in the type of the article, the attachment position of the article, the inclination of the article, and the object to which the article is attached, as at least one of the viewpoints (defect mode) to be inspected as described above. It can be said that it learns the association between the article and the inspection result in either the positional relationship between the object and the article or the damage of the article. The same is true when learning image data containing two or more types of articles.

Machine learning such as AI may be applied to the learning unit 5 (appearance inspection image learning device 2). The learning unit 5 (appearance inspection image learning device 2) constructs and accumulates a learning model. The learning unit 5 improves the judgment accuracy based on the image data and the inspection information as the amount of learning progresses with a plurality of image data with different bordering positions generated by trimming from the same image data for each article. Further, in the appearance inspection image learning system (appearance inspection image learning apparatus) according to the first embodiment, the learning unit 5 learns with inspection information for each inspection viewpoint (defect mode) even with the same image data. Since each is provided, learning efficiency is good. The same applies to the operation of the visual inspection image learning apparatus according to the first embodiment (the visual inspection image learning method according to the first embodiment) described below.

In FIG. 1, the inspection information input unit 4 inputs inspection information associated with an inspection viewpoint (defect mode) that matches the inspection viewpoint (defect mode) set for each learning unit 5 by switching the connection to the learning unit. 5. The examination information input unit 4 itself may grasp the correspondence relationship between the examination information and the learning unit 5, or when the examination information is input to the examination information input unit 4, the learning unit 5 to which the examination information is sent in timing and order. You can decide.

Of course, in FIG. 1, the inspection information that is not associated with the inspection point of view (defect mode) that the learning unit 5 is in charge of may be discarded without being used as teacher data. In this case, determination by the examination information input unit 4 is unnecessary. At this time, the test information input unit 4 may switch the connection so that the test information is not sent to the learning unit 5 and the test information input unit 4 sends the test information to all the learning units 5 . Furthermore, although not shown, the test information input unit 4 may send test information to one learning unit 5 and the test information may be communicated between the learning units 5 .

In FIG. 2, similarly to FIG. 1, the inspection information input unit 4 inputs the inspection information associated with the inspection viewpoint (defect mode) that matches the inspection viewpoint (defect mode) set for each learning unit 5 to the learning unit. 5. Since the examination information input unit 4 and the learning unit 5 are provided on a one-to-one basis, the examination information input unit 4 itself can grasp the correspondence relationship between the examination information and the learning unit 5. Alternatively, inspection information that is not associated with an inspection viewpoint (defect mode) handled by the connected learning unit 5 may be discarded together with the image data without being used as teacher data.

Of course, in FIG. 2, when the examination information is input to the examination information input section 4, the learning section 5 to which the examination information is sent may be determined according to timing and order. Of course, the learning unit 5 may determine the timing and order of inputting test information to the test information input unit 4 , or the timing and order of inputting test information to the learning unit 5 .

In other words, it can be said that the inspection information input unit 4 receives inspection information, which is the inspection result of the inspection viewpoint (defect mode) when the learning unit 5 learns the association between the article and the inspection result. Further, the inspection information input unit 4 extracts inspection information, which is the inspection result of the inspection viewpoint (defect mode) when the learning unit 5 learns the association between the article and the inspection result, from the input inspection information. It can be said that the data is sent to the learning unit 5 as a result. Furthermore, it can be said that the learning unit 5 extracts the inspection information, which is the inspection result regarding the viewpoint of inspection when the learning unit 5 learns the association between the article and the inspection result, from the input inspection information. .

Here, the operation of the visual inspection image learning apparatus according to the first embodiment (visual inspection image learning method according to the first embodiment) will be described with reference to FIG. In FIG. 3, step 1 is a processing step in which image data including the appearance inspection portion of an article is input to the image input unit 3 . Step 2 is a processing step in which inspection information, which is an inspection result indicating whether the appearance of the article corresponding to the image data is normal or abnormal, is input to the inspection information input unit 4 . Steps 1 and 2 may be performed in any order. It can be done at the same time. Step 3 is a processing step that causes the learning unit 5 to learn associations between articles and inspection results in at least one of inspection viewpoints (defective modes) based on image data and inspection information.

Next, a visual inspection apparatus according to Embodiment 1 will be described with reference to FIGS. 4 and 5. FIG. In FIGS. 4 and 5, the visual inspection apparatus 6 uses the learning result (learning model) of the visual inspection image learning apparatus 2 (visual inspection image learning system 1) shown in FIGS. The new image input unit 7 receives new image data obtained by newly reading an article. Here, the new image data is the existing image data, which is simply referred to as "new" in order to distinguish it from the image data used by the learning unit 5 (appearance inspection image learning device 2) to construct the learning model. Include. Therefore, the new image data may be called image data for determination.

4 and 5, the image determination unit 8 determines the appearance of the article corresponding to the new image data based on the learning result (learning model) learned by the learning unit 5, based on the learning result (learning model) learned by the learning unit 5. is normal or abnormal based on inspection information. Furthermore, the image determination unit 8 may determine that the appearance of the article corresponding to the new image data cannot be determined based on information other than the inspection information. By doing so, even if there is no image data that is close to the image data learned by the learning unit 5, or if the new image data (image data) is completely unrelated to the article, the AI can forcibly generate similar images. The data can be linked to avoid judging whether the appearance of the article is normal, abnormal, or the like.

The image determination unit 5 newly generates inspection information with the determined result as the inspection result and sends it to the learning unit 5. The learning unit 5 generates new image data based on the new image data and the inspection information sent from the image determination unit. may be used to learn associations between articles and inspection results. In the case of this learning, it can be said that the association with the inspection viewpoint (defect mode) used by the image determination unit 5 is originally associated with the learning result (learning model) learned by the learning unit 5 .

In FIG. 4, the new image acquiring section 9 acquires new image data (image data) which is image data for judgment and sends it to the new image input section 7 . A camera, an image sensor, or the like is suitable for the new image acquisition unit 9 . The new image acquisition unit 9 may be configured as the appearance inspection device 6 . Of course, the appearance inspection image learning device 2 may be configured as the appearance inspection device 6 . These are the same mainly in the operation of the visual inspection apparatus (visual inspection method according to the first embodiment) according to the first embodiment described below.

The visual inspection apparatus 6 shown in FIG. 4 performs inspection from one inspection viewpoint (defective mode). Therefore, when inspection is required for all inspection viewpoints (defective modes), appearance inspection devices 6 may be prepared for the number of inspection viewpoints (defective modes). In other words, parallel processing is performed in a distributed manner for each inspection point of view (failure mode), so the inspection time can be shortened.

Of course, as shown in FIG. 5, the visual inspection apparatus according to the first embodiment is prepared for each viewpoint (defective mode) to be inspected by the visual inspection image learning system 1 (visual inspection image learning apparatus 2). A learning result for each viewpoint (defect mode) to be inspected may be used. In this case, the image judgment unit 8 inspects whether the appearance of the article corresponding to the new image data is normal or abnormal in the inspection viewpoint (defect mode) for each learning unit 5 based on the learning result learned by the learning unit 5. It is determined from information. In FIG. 5, the image determination unit 8 performs inspection for one inspection viewpoint (defect mode), but as shown in FIG. You can prepare.

Here, the operation of the visual inspection apparatus (shown in FIGS. 4 and 5) according to the first embodiment (the visual inspection method according to the first embodiment) will be mainly described with reference to FIG. In FIG. 6, step 11 is a processing step in which new image data obtained by newly reading an article is input to the new image input unit 7 . Step 12 is a processing step for inputting new image data from the new image input unit 7 to the learning unit 5 and using the learning model. Step 13 is a processing step in which the image determination unit 8 determines whether the appearance of the article corresponding to the new image data is normal or abnormal based on the learning result (learning model) learned by the learning unit 5 from the inspection information. Furthermore, step 13 may determine that the appearance of the article corresponding to the new image data cannot be determined on the basis of information other than the inspection information.

As described above, FIG. 7 shows the appearance inspection apparatus according to the first embodiment, in which the image determination section 8 is formed for each inspection viewpoint (defective mode). In FIG. 7, each image determination unit 8 determines the appearance of the article corresponding to the new image data in the inspection viewpoint (defect mode) for each learning unit 5 based on the learning result (learning model) learned by the learning unit 5. is normal or abnormal from the inspection information (second determination). The second image determination unit 10 determines that the appearance of the article is normal when the appearance of the article is determined to be normal in all inspection viewpoints (defect modes), or determines that the appearance of the article is normal in at least one inspection viewpoint (defect mode). The appearance of the article is determined to be abnormal when the appearance of the article (defect mode) in 1 is determined to be abnormal. The second image determination section 10 and the plurality of image determination sections 8 may be collectively referred to as the image determination section 8 .

In addition, the second image determination unit 10 (image determination unit 8) immediately determines that the appearance of the article is abnormal when any one of the inspected appearances (defective mode) is determined to be abnormal. can be sped up. Of course, even after the final judgment has been made, it is possible to continue judging the remaining appearances (defective modes) and obtain results as to whether or not there are other appearances (defective modes) to be inspected that are determined to be abnormal. . This processing is particularly important in the case of feedback, which will be described next.

In the case of feedback, the second image determination unit 10 (image determination unit 8) newly generates inspection information as inspection results based on the results determined for each inspection viewpoint (defect mode). , and the learning unit 5 learns to associate the article with the inspection result based on the new image data and the inspection information sent from the second image determination unit 10 (image determination unit 8). You may make it

Here, the operation of the visual inspection apparatus (shown in FIG. 7) (appearance inspection method according to the first embodiment) according to the first embodiment will be mainly described with reference to FIG. In FIG. 8, step 11 is a processing step in which new image data obtained by newly reading an article is input to the new image input unit 7, as in FIG. Step 12 is a processing step for inputting new image data from the new image input unit 7 to each learning unit 5 and using the learning model. Step 13 is a processing step in which each image determination unit 8 determines whether the appearance of the article corresponding to the new image data is normal or abnormal based on the learning result (learning model) learned by each learning unit 5 from the inspection information. is. Furthermore, as in FIG. 6, step 13 may determine that the appearance of the article corresponding to the new image data cannot be determined based on information other than the inspection information.

In FIG. 8, in step 14, the second image judgment unit 10 (image judgment unit 8) judges that the appearance of the article is normal when the appearance of the article is judged to be normal in all inspection viewpoints (defective modes). Alternatively, when the appearance (defect mode) of the article in at least one inspection viewpoint (defect mode) is determined to be abnormal, the appearance of the article is determined to be abnormal (second determination). In this second determination step 14, when the second image determination unit 10 (image determination unit 8) determines that any one of the appearances to be inspected (defect mode) is abnormal, the appearance of the article is determined to be abnormal. It may include a process of determining that Furthermore, even after the final judgment is made, step 14 continues to judge the remaining appearances (defective modes) to determine whether there are other appearances (defective modes) to be inspected that are determined to be abnormal. It may include a process of issuing

As described above, the visual inspection image learning system, the visual inspection image learning device, and the visual inspection device using the same according to the first embodiment can learn from each viewpoint (defect mode) of inspecting an article even with the same image data. Therefore, in addition to the learning efficiency of the visual inspection image learning system and the visual inspection image learning apparatus, the judgment processing speed of the visual inspection apparatus can be increased and the processing can be simplified.

1 visual inspection image learning system, 2 visual inspection image learning device, 3 image input unit,
4 inspection information input unit, 5 learning unit, 6 appearance inspection device,
7 new image input unit, 8 image determination unit, 9 new image acquisition unit,
10 second image determination section (image determination section);

Claims (12)

Appearance inspection for learning inspection information, which is an inspection result indicating whether the appearance of the article is normal or abnormal, corresponding to image data for visual inspection of the article, in a learning unit that differs for each inspection viewpoint. An image learning device for visual inspection having one learning unit in an image learning system,
An image input unit for inputting the image data including the appearance inspection points of the article, and an inspection information that is an inspection result indicating whether the appearance of the article corresponding to the image data is normal or abnormal is input. and an examination information input unit that
The image learning device for appearance inspection, wherein the learning unit learns association between the article and the inspection result in at least one of the viewpoints to be inspected based on the image data and the inspection information. 2. The image learning apparatus for appearance inspection according to claim 1, wherein the image input unit includes a plurality of articles included in the input image data. The learning unit determines whether or not the article is attached, the difference in the type of the article, the attachment position of the article, the inclination of the article, the object to which the article is attached and the article, as at least one of the viewpoints to be inspected. 3. The visual inspection image learning apparatus according to claim 1, wherein association between said article and said inspection result in either positional relationship of said article or breakage of said article is learned. 3. The inspection information input unit receives the inspection information, which is the inspection result with respect to the viewpoint to be inspected when the learning unit learns the association between the article and the inspection result. The visual inspection image learning device according to any one of claims 1, 2, and 3. The inspection information input unit extracts, from the input inspection information, the inspection information, which is the inspection result regarding the viewpoint to be inspected when the learning unit learns to associate the article with the inspection result. 4. The visual inspection image learning device according to claim 1, wherein the image is sent to the learning unit. The learning unit extracts, from the input inspection information, the inspection information that is the inspection result regarding the inspection viewpoint when learning the association between the article and the inspection result. The visual inspection image learning device according to any one of claims 1, 2, and 3. A visual inspection device using the learning result of the visual inspection image learning device according to any one of claims 1 to 6,
a new image input unit for inputting new image data newly read from the article; and based on the learning result learned by the learning unit, whether the appearance of the article corresponding to the new image data is normal or abnormal An appearance inspection apparatus comprising: an image determination unit for determining from inspection information. The image determination unit newly generates the inspection information with the determined result as the inspection result and sends it to the learning unit, and the learning unit receives the new image data and the 8. The visual inspection apparatus according to claim 7, wherein association between said article and said inspection result is learned based on inspection information. A plurality of image learning devices for visual inspection according to any one of claims 1 to 6 are prepared for each viewpoint to be inspected, and a visual inspection device using learning results for each viewpoint to be inspected,
a new image input unit for inputting new image data obtained by newly reading the article; An appearance inspection apparatus, comprising: an image judgment unit for judging whether the appearance of an article is normal or abnormal based on the inspection information. 10. The appearance inspection apparatus according to claim 9, wherein the image determination unit determines that the appearance of the article is normal when the appearance of the article is determined to be normal in all of the inspected viewpoints. 11. The image determination unit determines that the appearance of the article is abnormal when the appearance of the article is determined to be abnormal in at least one of the inspected viewpoints. Appearance inspection device. The image determination unit newly generates the inspection information as the inspection results based on the result of determination for each viewpoint to be inspected, and sends the inspection information to each of the learning units for each viewpoint to be inspected. 9, 10, and 11, wherein association between the article and the inspection result is learned based on the new image data and the inspection information sent from the image determination unit. Appearance inspection device according to any one of the above.

Images