JP2019190891A - Inspection management system, inspection management device, and inspection management method - Google Patents

Abstract

To provide means that, in appearance inspection of a sheet-like article in which an object to be inspected is photographed in a plurality of different methods and a defect is detected for each of the photographing methods, reduces a harmful effect caused by individually managing a plurality of pieces of defective image data indicating the same defect.SOLUTION: An inspection management system for inspecting a sheet-like object to be inspected comprises: an appearance inspection unit including a plurality of photographing means that photographs the appearance of an object to be inspected T in different methods, and detection means that detects a defect in the object to be inspected on the basis of images photographed by the plurality of photographing means; a storage unit that stores defective image data in which the defect detected by the detection means is photographed for each of the photographing means having the plurality of different methods; and an inspection management unit including same defective image integration means processes the plurality of pieces of defective image data stored in the storage unit in which the same defect in the object to be inspected is photographed, as one same defective image set.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a technique for detecting an abnormal portion of a sheet-like inspection object, and particularly to a technique for analyzing a detected abnormality.

  In the production line for manufacturing or processing sheet-like articles, abnormalities in the sheet (contamination of foreign matter) are detected using images obtained by irradiating the sheet with visible light or ultraviolet light and photographing the transmitted or reflected light with a camera. , Dirt, wrinkles, etc. (hereinafter also referred to as defects) are used (for example, Patent Document 1, Patent Document 2, etc.).

  In such an apparatus, the inspection object is imaged by a plurality of different imaging methods such as a surface reflection image, a back surface reflection image, a transmission image, a visible light image, an infrared light image, etc., and a defect is detected in each image. Some devices perform (good / bad determination), thereby improving the accuracy of inspection (reducing missed defects).

  Then, using the apparatus as described above, not only the defect of the product is detected and the defective product is removed, but the detected defect is classified for each occurrence factor (hereinafter also referred to as a type), and each defect is classified. By generating and analyzing the frequency of occurrence, it is also expected to be used for grasping the status of the quality of manufactured products, early detection of abnormal parts in manufacturing processes, and predictive maintenance.

  As described above, in effectively utilizing data classified by defect type, it is important that the number of occurrences, classification, and the like are accurate, but a plurality of different imaging methods (hereinafter referred to as inspection methods) as described above. May also be harmful to the accuracy of the data obtained.

  For example, even when trying to find the exact number of defects for each defect type, if multiple inspection methods are used, defects may be detected in each of the multiple inspection methods for one defect. When the number of defects obtained by the method is simply summed, there is a problem that the number does not match the original number of defects on the product.

  On the other hand, in patent document 2, the defect information about the same defect imaged with a different camera apparatus is compared, and the image data imaged with one camera apparatus of the different camera apparatuses determined according to the determination rule are compared. An inspection system having a function of determining and displaying defect information extracted based on representative defect information has been proposed. However, in such a method, defect image data other than the representative defect information acquired by a plurality of cameras is thinned out and cannot be used effectively.

  In addition, when classifying defect types, AI obtained by deep learning of defect images as teacher data is also used. However, when registering teacher data, defect images that are difficult to identify defect types are actually used. There is also a human error that registration is performed in association with a defect type different from the defect type. Further, at this time, there is a possibility that different defect types may be registered for each inspection method for a plurality of defect image data that should originally indicate the same defect. And the data classified by the AI learned by using the erroneous teacher data registered in this way will cause doubt about the accuracy.

In addition, acquiring defect images for each of a plurality of inspection methods and individually registering them with defect types is registered. If there are many types of inspection methods, the labor for registering teacher data simply increases accordingly. It was also inefficient.

Japanese Patent Laying-Open No. 2015-172519 Japanese Patent No. 5305002

  The present invention has been made in view of the above circumstances, and the object of the present invention is to provide a sheet-like article for photographing an inspection object by a plurality of different methods and detecting defects for each of the photographing methods. It is an object of the present invention to provide means capable of reducing adverse effects caused by individually managing a plurality of defect image data indicating the same defect.

  In order to solve the above-described problems, an inspection management system according to the present invention is an inspection management system for inspecting a sheet-like inspection object, and takes two images of the appearance of the inspection object by different methods. Appearance inspection unit comprising the above imaging unit, and a detection unit that detects a defect of the inspection object based on the respective images captured by the plurality of imaging units, and the plurality of different types of imaging units For each, a storage unit in which defect image data in which a defect detected by the detection unit is photographed is recorded, and a set of the defect image data recorded in the storage unit is identical in the inspection object. And an inspection management unit including an identical defect image integration unit that processes a plurality of defect image data in which defects are photographed as one identical defect image set.

  Here, “photographing by different methods” means, for example, that the surface of the object to be photographed may be different, the wavelength of the detected light may be different, or the reflected light is photographed. It may be a difference between what to do and what to photograph the transmitted light. Further, “the same defect in the inspected object is photographed” may be recognized as described below when the position of the defect in the inspected object is the same, as will be described later. It should be noted that each element constituting the system does not have to be a separate body, and may be housed in an integrated housing, or may be partially integrated.

  According to the configuration as described above, even when a plurality of methods are used to photograph the inspection object and a plurality of defect image data of different imaging methods are recorded for one defect in the inspection object, the same defect is detected. A plurality of data shown can be managed as one set. As a result, it is possible to reduce adverse effects caused by individually managing a plurality of defect image data indicating the same defect.

  The inspection management unit may further include defect counting means for counting the number of the same defect image sets and calculating the number of defects for each predetermined inspection unit. Here, the “predetermined inspection unit” can be set and changed by the user at an arbitrary timing. With such a configuration, the number of defects is counted for each defect in the object to be inspected instead of separately counting a plurality of defect images taken for the same defect. Can be calculated.

Further, the inspection management system further includes an output unit, and the inspection management unit notifies the fact through the output unit when the calculated number of defects exceeds a predetermined value. A defect number warning means may be further provided. By adopting such a configuration, when the number of defects exceeds a predetermined value, the user can easily know that fact, so that inspection management can be performed efficiently.

  Note that the output unit may be any device that can be notified by a user's perceptible method. For example, the output unit may be a display device such as a liquid crystal display, an audio output device such as a speaker, or printing. It may be a device. Moreover, you may notify using these together.

  The inspection management unit may further include defect type classification means for classifying the defect type for each of the same defect image sets. With such a configuration, it is possible to prevent a plurality of defect image data that should originally indicate the same defect from being classified into different defect types for a plurality of different photographing means. .

  Further, the inspection management unit counts the number of the same defect image sets for each defect type classified by the defect type classification unit, and calculates the number of defects for each defect type for each predetermined inspection unit. A type counting means may be further provided. With such a configuration, an accurate number of defects can be obtained for each defect type.

  The inspection management system includes an output unit, and the inspection management unit determines that the calculated number of defects for each defect type exceeds a predetermined value determined for each defect type. Further, a defect type defect number warning means for notifying the fact via the output unit may be further provided. The defect type defect count warning means may also notify the defect type exceeding a predetermined value. With such a configuration, the user can know when the number of defects exceeds a predetermined value for each defect type, and can perform more detailed inspection management.

  In the case where the inspection system is configured to include the defect number warning unit, the output unit may be configured in common with the defect number warning unit.

  The defect type classification means may include inference means that has been learned by a deep learning technique. In recent years, image recognition technology based on artificial intelligence using a deep learning method has achieved high results. By using such technology, classification of defect types can be performed efficiently and automatically.

  The inspection management system further includes a display unit, and the inspection management unit simultaneously displays defect image data constituting the set for each of the same defect image sets on the display unit, and displays the display. Deep learning teacher data registration means for requesting the user to register the defect type corresponding to the set of defect image data thus performed may be further provided.

  Here, the display unit is generally a display device such as a liquid crystal display, but other display means such as a display by a projector may be used. According to the above configuration, defect image data indicating the same defect can be registered in a lump, and teacher data can be registered efficiently. In addition, since the user can determine and register the defect type by comparing the defect images by a plurality of inspection methods included in the same defect image set, the defect type can be registered only for individual defect image data. In comparison, it is possible to reduce the selection of the defect type or the time required for the determination.

  Further, the defect type classification means classifies the defect type for the same defect image set as unknown when the defect type classification of the same defect image set cannot be performed with an accuracy higher than a predetermined accuracy. Also good. According to such a configuration, it is possible to prevent the reliability of data obtained as a result from being lowered by forcibly classifying defect types with low accuracy.

  In addition, the defect type classification unit may determine the classification of the defect type by the user for the same defect image set in which the defect type is classified as unknown. According to such a configuration, it is possible to prevent the same defect image set classified as having an unknown defect type from remaining as unidentified data.

  Further, the two or more photographing means for photographing the appearance of the inspection object by different methods include a surface photographing means for photographing an image by reflected light reflected by the first surface of the inspection object, Any of a back surface photographing means for photographing an image by reflected light reflected by a second surface opposite to the first surface, and a transmitted light photographing means for photographing an image by transmitted light that has passed through the inspection object Or two or more. According to such a configuration, it is possible to detect and / or classify defects corresponding to various types of defects.

  Further, the two or more imaging means for imaging the appearance of the object to be inspected by different methods include: a first wavelength imaging means for imaging the object to be inspected with light of a first wavelength; and the first wavelength. And second wavelength imaging means for imaging the object to be inspected with light of different wavelengths. According to such a configuration, it is possible to detect and / or classify defects corresponding to various types of defects.

  Further, the same defect image integration means may integrate a plurality of different defect image data in which the position of the defect in the inspection object is in the same range as one same defect image set.

  Here, “within a position within the same range” means that the position is not limited to a completely matching position, but includes a position within a predetermined allowable range. In a plurality of defect image data, as a method for identifying whether or not the photographed defect is the same, the same defect position in the product is the image data obtained by photographing the same defect. Can be adopted. In order to specify the position of the defect in the inspection object, for example, the position of the defect may be calculated based on a predetermined location of each imaging unit, an imaging range, a conveyance speed of the inspection object, and the like. .

  In order to solve the above-described problems, an inspection management apparatus according to the present invention is an inspection management apparatus that processes defect image data acquired by photographing the appearance of a sheet-like inspection object using a plurality of different photographing methods. And a plurality of defect image data in which the same defect in the inspection object is photographed from the set of defect image data as a single defect image set.

  In order to solve the above-described problem, an inspection management method according to the present invention is a method for managing an appearance inspection of a sheet-like inspection object, wherein the inspection object is imaged by two or more different methods. One step, a second step of detecting a defect of the inspection object based on images of the inspection object taken by a plurality of different imaging methods imaged in the first step, and a defect detected in the second step And a plurality of defect image data in which the same defect in the inspection object is photographed out of the set of defect image data recorded in the third step. Are integrated as one identical defect image set, and the number of defects is calculated by counting the number of identical defect image sets integrated in the fourth step for each predetermined inspection unit. It has a step of 5.

  The inspection management method may further include a sixth step of notifying that when the number of defects calculated in the fifth step exceeds a predetermined value.

  Note that the above processes and means can be freely combined and implemented as long as there is no technical contradiction.

  According to the present invention, a plurality of defect image data indicating the same defect are individually managed in the appearance inspection of a sheet-like article in which an inspection object is imaged by a plurality of different methods and a defect is detected for each imaging method. It is possible to reduce the harmful effects caused by doing so.

FIG. 1 is a diagram schematically illustrating a configuration example of an inspection management system according to an application example. FIG. 2 is a flowchart showing a flow of processing performed in the inspection management system according to the application example. FIG. 3 is a diagram schematically illustrating a configuration example of the inspection management system according to the embodiment. FIG. 4 is a flowchart illustrating the flow of defect type classification processing in the embodiment.

  Hereinafter, an example of an embodiment of the present invention will be described with reference to the drawings.

<Application example>
The present invention can be applied, for example, as an inspection management system 9 as shown in FIG. FIG. 1 is a diagram schematically illustrating a configuration example of an inspection management system 9 according to this application example. The inspection management system 9 is a system that detects a defect of a sheet-like article and manages information related to the inspection, and includes, as main components, a front-surface reflection light source 911, a rear-surface reflection light source 912, and a transmission light source. 913, a front surface photographing camera 921 as a measurement system, a back surface photographing camera 922, a control terminal 93, and a transport mechanism (not shown).

  As shown in FIG. 1, the inspection object T is conveyed in the horizontal direction (arrow direction) by a conveyance mechanism (not shown), and the appearance image of the inspection object T is continuously acquired by the measurement system during the conveyance, An inspection is carried out based on this. The inspection object T is formed in a sheet shape, and examples thereof include paper, cloth, film, resin, and cellulose. Further, the sheet body is not limited to a single material, and may be a sheet body having a plurality of layers such as a wrapping paper in which a film and a nonwoven fabric are bonded together. Further, it may be a food such as dried seaweed.

  The front surface reflection light source 911 of the illumination system is arranged so as to irradiate visible light (for example, white light) to the surface (first surface) of the inspection object T, and the back surface reflection light source 912 similarly applies visible light to the inspection object. It arrange | positions so that it may irradiate with respect to the back surface (2nd surface) of T. Further, the transmissive light source 913 is disposed so as to irradiate the back surface (second surface) of the inspection object T with infrared rays.

  Although not shown, the surface photographing camera 921 of the measurement system includes a signal output unit, a visible light receiving sensor, an infrared light receiving sensor, a spectral prism, and a lens, and is arranged so as to image the surface of the inspection object T. Specifically, light irradiated from the surface reflection light source 911 and reflected from the surface of the inspection object T (hereinafter referred to as surface reflection light), infrared light irradiated from the transmission light source 913 and transmitted through the inspection object T (hereinafter referred to as transmission). The object T is photographed by light). The spectroscopic prism splits the light incident on the camera into at least light having a wavelength in the visible light region and infrared rays and causes each corresponding sensor to receive the light. As the sensor, for example, a CCD or CMOS sensor can be used.

Further, although not shown, the back surface camera 922 includes a signal output unit, a light receiving sensor capable of detecting light in the visible light region, and a lens, and is arranged so as to image the back surface of the inspection object T. Specifically, the inspection object T is photographed by light irradiated from the back surface reflection light source 912 and reflected by the back surface of the inspection object T (hereinafter referred to as back surface reflection light). For example, a CCD or CMOS sensor can be used as the light receiving sensor.

  The control terminal 93 controls the illumination system, the measurement system, and the transport mechanism, and processes various information related to the inspection. The hardware configuration includes various input / output devices, processors, storage devices, and the like. As functional modules, a defect detection unit 931, a defect position specifying unit 932, a defect image storage unit 933, an identical defect image integration unit 934, A defect counting unit 935, a defect number warning unit 936, and a defect type classification unit 937 are provided.

  The defect detection unit 931 detects a defect included in the inspection object T based on the image signal input from each camera of the measurement system. The detection of the defect is performed, for example, by determining whether or not a feature amount obtained from the photographed image deviates from a predetermined threshold value. For example, luminance can be used as the feature amount, and brightness, saturation, hue, or the like may be used.

  In this application example, the image by the surface photographing camera 921 and the image by the surface reflected light (hereinafter referred to as the surface reflected image) and the image by the transmitted light (hereinafter referred to as the transmitted image) are photographed by the back surface photographing camera 922. The feature amount is determined for each of the images of the back surface reflected light (hereinafter referred to as the back surface reflected image).

  When a defect is detected, the defect position specifying unit 932 specifies in which part of the inspection object T the defect is located. The position can be specified based on, for example, a predetermined conveyance speed of the inspection object T, a camera installation position, a size of the inspection object T, and an elapsed time from the start of inspection.

  When a defect is detected, the defect image storage unit 933 associates the defect image captured by each camera (hereinafter referred to as a defect image) with the defect position information identified by the defect position identification unit 932. To the storage device. Further, not only the image of the photographing method in which the defect is detected but also the image of the portion corresponding to the other photographing method is recorded in the storage device.

  When a plurality of different defect images having the same defect position specified by the defect position specifying unit 932 are recorded in the storage device, the same defect image integration unit 934 displays a plurality of images having the same defect position. , As a set of identical defect images. The plurality of defect images set in this way are then integrally processed during display on the display device and various data analysis. In addition, the same defect image set may be newly recorded in the storage device as a set.

  The defect counting unit 935 counts the number of identical defect image sets for each predetermined inspection unit, and calculates the number of defects per the inspection unit. Here, the predetermined inspection unit may be a predetermined number (for example, one roll, one lot, several lots) or a predetermined time (for example, one day, one week, one month). . Since the number of defects is calculated in this way, a plurality of defect image data indicating the same defect in the inspection object T is not counted repeatedly, and an accurate number of defects for each predetermined inspection unit is obtained. I can do things.

  The defect number warning unit 936 determines whether or not the number of defects calculated by the defect counting unit 935 exceeds a predetermined value, and when it is determined that the number of defects exceeds the predetermined value, this is not illustrated. The user is notified by outputting from an output device (for example, a liquid crystal display, a speaker, etc.).

The defect type classification unit 937 classifies the defect image data integrated as the same defect image set according to the type of defect, and records the classified type and the defect image data in association with each other. The type of defect to be classified can be arbitrarily set by the user. For example, a type such as foreign matter contamination, dirt, wrinkle, or hole may be provided, or a finer type (eg, insect, wood piece, metal foreign matter, Oil stains, water stains, large holes, small holes, etc.).

  As a method of classifying the defect type, it may be performed based on the feature amount of any one of the front reflection image, the back reflection image, and the transmission image, or by comparing the feature amount of the defect portion of each image. Alternatively, it may be performed depending on whether or not a defect is detected by each inspection method and a combination thereof. Moreover, it can also classify using these methods in combination. In this way, by classifying the defect type by setting a plurality of defect image data indicating the same defect as compared to the defect type classification based on the single image in which the defect is detected. Therefore, it is possible to accurately classify the defect types.

  FIG. 2 is a flowchart showing a flow of processing performed in the inspection management system 9 according to this application example. First, the inspection management system 9 acquires a front-surface reflection image, a back-surface reflection image, and a transmission image by the measurement system (step S101), and detects a defect by the defect detection unit 931 based on the image (step S102). Next, the defect image storage unit 933 records the defect image in the storage device (step S103). Subsequently, the same defect image integration unit 934 integrates a plurality of images having the same defect position on the inspection object as a set of the same defect image from the recorded defect images (step S104). Next, the defect counting unit 935 counts the number of sets of the same defect image set for each predetermined inspection unit, and calculates the number of defects per the inspection unit (step S105). Here, the defect number warning unit 936 determines whether or not the number of defects calculated in step S105 exceeds a predetermined value (threshold value) (step S106). (Step S107), the process proceeds to Step S108. On the other hand, if it is determined that the number of defects does not exceed the predetermined value, the process directly proceeds to step S108. In step S108, the defect type classification unit 937 classifies and records the defect types of the defect image data integrated as the same defect image set, and ends the series of processes. Note that the order from the measurement of the number of defects to the notification when the number of defects exceeds a predetermined value (step S105 to step S107) and the processing of step S108 may be switched.

  With the configuration of the inspection management system 9 according to this application example as described above, a plurality of defect image data indicating the same defect can be set, counted, and classified by defect type, so that a plurality of defect images indicating the same defect can be obtained. It is possible to reduce harmful effects caused by managing data individually (for example, counting the same defects repeatedly, classifying defect types with low accuracy, etc.).

<Example>
Below, an example of the form for implementing this invention is demonstrated in more detail. However, the dimensions, materials, shapes, relative arrangements, and the like of the components described in this embodiment are not intended to limit the scope of the present invention only to those unless otherwise specified.

(System configuration)
FIG. 3 is a diagram schematically illustrating a configuration example of the inspection management system 1 according to the present embodiment. As shown in FIG. 3, the inspection management system 1 according to the present embodiment includes an appearance inspection device 2 and an inspection management device 3 as main components.

(Appearance inspection device)
The appearance inspection apparatus 2 is an apparatus for acquiring an appearance image of a sheet-like article and detecting a defect based on the image, and mainly includes an illumination system, a measurement system, and a transport mechanism (not shown). The control terminal 23 is provided.

  The inspection object T is conveyed in the horizontal direction (arrow direction) by a conveyance mechanism (not shown), and during the conveyance, appearance images of the inspection object T are continuously acquired by the measurement system, and an inspection is performed based on this. Is done. The inspection object T is formed in a sheet shape, and examples thereof include paper, cloth, and film. Further, the sheet body is not limited to a single material, and may be a sheet body having a plurality of layers such as a wrapping paper in which a film and a nonwoven fabric are bonded together. It may also be a food such as dried seaweed.

  The illumination system includes a surface reflection light source 211 that irradiates the surface of the inspection object T with visible light (for example, white light), a back surface reflection light source 212 that irradiates the surface of the inspection object T with visible light, and a back surface of the inspection object T. A transmissive light source 213 that emits visible light is provided. For each of these light sources, for example, LED illumination may be used.

  The measurement system includes a front-surface reflected light camera 221 that captures light (hereinafter referred to as front-surface reflected light) irradiated from the front-surface reflection light source 211 and reflected from the surface of the inspection object T, and a rear-surface reflection light source 212 and irradiated from the inspection object T. A back-surface reflected light camera 222 that captures light reflected on the surface (hereinafter referred to as surface-reflected light), and a transmitted-light camera 223 that captures light that has been irradiated from the transmissive light source 213 and transmitted through the inspection object T. ing. Each camera constituting the measurement system corresponds to the photographing means in the present invention.

  Each camera includes a light receiving sensor capable of detecting light captured by each camera, a lens, and a signal output unit, and outputs light detected by the light receiving sensor as an electric signal via the lens. For example, a CCD or CMOS sensor can be used as the sensor.

  The control terminal 23 controls the illumination system, the measurement system, and the transport mechanism, and processes various types of information. The hardware configuration includes an input / output device, a processor, a storage device, and the like. The functional module includes a defect detection unit 231, a defect position specifying unit 232, and a defect image storage unit 233.

  The defect detection unit 231 detects a defect included in the inspection object T based on the image signal input from each camera of the measurement system. The detection of the defect is performed, for example, by determining whether or not a feature amount obtained from the photographed image deviates from a predetermined threshold value. For example, luminance can be used as the feature amount, and brightness, saturation, hue, or the like may be used.

  In this embodiment, an image captured by the front surface reflected light camera 221 (hereinafter referred to as a front surface reflected image), an image captured by the back surface reflected light camera 222 (hereinafter referred to as a back surface reflected image), and an image captured by the transmitted light camera 223. The feature amount is determined for each (hereinafter referred to as a transparent image).

  When a defect is detected from the inspection object T, the defect position specifying unit 232 specifies at which position of the inspection object T the defect is located. The position can be specified based on, for example, a predetermined conveyance speed of the inspection object T, a camera installation position, a size of the inspection object T, and an elapsed time from the start of inspection.

  When a defect is detected, the defect image storage unit 233 associates an image of the defect taken by each camera (hereinafter referred to as a defect image) with information on the position of the defect specified by the defect position specifying unit 232. To the storage device (hereinafter, the recorded data is referred to as defect image data). Further, not only the image of the photographing method in which the defect is detected, but also an image of a portion corresponding to another photographing method may be recorded in the storage device.

(Inspection management device)
The above-described appearance inspection apparatus 2 is connected to the inspection management apparatus 3 via a network (LAN). The inspection management device 3 acquires information related to the inspection from the appearance inspection device 2 and processes the information. The CPU (processor), the main storage device (memory), the auxiliary storage device (hard disk, etc.), the input It is configured by a general-purpose computer system including devices (keyboard, mouse, controller, touch panel, etc.), output devices (liquid crystal display, speakers, printers, etc.) and the like.

  The inspection management apparatus 3 may be configured by a single computer or a plurality of computers. Alternatively, all or part of the functions of the inspection management apparatus 3 can be mounted on the control terminal 23 of the appearance inspection apparatus 2. Or you may implement | achieve a part of function of the test | inspection management apparatus 3 with the server (cloud server etc.) on a network.

  The CPU of the inspection management apparatus 3 of this embodiment includes, as functional modules, the same defect image integration unit 31, a defect type classification unit 32, a teacher data registration unit 33, a defect type counting unit 34, and a defect type defect count warning. Part 35.

  The same defect image integration unit 31 acquires defect image data from the appearance inspection apparatus 2 and integrates a plurality of defect images having the same defect position on the inspection object T as a set of identical defect images. The plurality of defect images set in this way are recorded in the storage device as one set, and are integrally processed at the time of display on the display device and various data analysis.

  The defect type classification unit 32 classifies defect image data integrated as the same defect image set into defect types. The type of defect to be classified can be arbitrarily set by the user. For example, a type such as foreign matter contamination, dirt, wrinkle, or hole may be provided, or a finer type (eg, insect, wood piece, metal foreign matter, Oil stains, water stains, large holes, small holes, etc.).

  As a method of classifying the defect type, it may be performed based on the feature amount of any one of the front reflection image, the back reflection image, and the transmission image, or by comparing the feature amount of the defect portion of each image. Alternatively, it may be performed depending on whether or not a defect is detected by each inspection method, a combination of detection methods, and the like. These methods can be combined for classification.

  In this embodiment, the defect type classification unit 32 classifies defect types by combining inference processing using a learned model generated by a deep learning method. The flow of the defect type classification process will be described later.

  The teacher data registration unit 33 has a function of accepting registration of teacher data for causing the artificial intelligence of the defect type classification unit 32 to perform deep learning, and a plurality of defect images constituting the same defect image set on the display device. Is displayed, and the user is requested to input a defect type corresponding to the same defect image set.

  The defect type counting unit 34 counts the number of identical defect image sets after the defect type classification for each type, and calculates the number of defects for each defect type per predetermined inspection unit. Here, the predetermined inspection unit may be a predetermined number (for example, one roll, one lot, several lots) or a predetermined time (for example, one day, one week, one month). .

The defect type defect number warning unit 35 determines whether or not the number of defects for each defect type calculated by the defect type counting unit exceeds a predetermined value determined for each defect type, and exceeds the predetermined value. When it is determined that there is a defect type for which the number of defects is calculated, the fact is output from the output device to notify the user. Here, the output by the output device may be, for example, a display by a display device, a warning sound from a speaker, or a print by a printer. Moreover, you may notify using these together. Further, the defect type defect count warning unit 35 may also notify the defect type exceeding a predetermined value.

(Defect type classification process flow)
Next, the flow of processing when the defect type classification unit 32 classifies the defect type will be described with reference to FIG. FIG. 4 is a flowchart showing the flow of defect type classification processing in this embodiment.

  First, the defect type classification unit 32 classifies the defect type based on the combination of presence / absence of defect detection of the front surface reflection image, the back surface reflection image, and the transmission image in the same target defect image set (step S201). Depending on the type of defect, there are cases where it is always detected by a specific inspection method or not detected by a specific detection method, so the presence or absence of an image of each inspection method included in the set (that is, a defect is detected during inspection). The defect type can be determined based on the combination condition. In addition, when recording defect image data, not only the image of the imaging method in which the defect is detected, but also the image of the corresponding part of the other imaging method is recorded in the storage device. If the presence / absence identification information is added, the same processing can be performed by a combination of the identification information. If the defect type can be classified in step S201, the process ends. If the defect type cannot be classified, the process proceeds to step S203 (step S202).

In step S203, the defect type classification unit 32 performs defect type classification based on a predetermined feature amount. Specifically, if at least one of the defect images included in the same defect image set satisfies the feature amount determination condition, the same defect image set is classified as corresponding to the defect type. For example, when the peak level of luminance is 100 or more (the maximum value is 255) and the area indicating the defect is 1 mm ² or more for a transmission image, the defect type of the same defect image set is a hole. Classify. If the defect type can be classified in step S203, the process ends. If the defect type cannot be classified, the process proceeds to step S205 (step S204).

  In step S205, the defect type classification unit 32 performs defect type classification on the same defect image set using a learned inference model generated in advance by a deep learning method for each inspection method. Specifically, inference is performed using a learned model of an inspection method corresponding to each of the front reflection image, the back reflection image, and the transmission image, and the determination probability of each defect type (that is, a predetermined defect type) Probability). Then, the defect type having the largest value among the determination probabilities of the defect types of all the images included in the set is classified as the defect type of the same defect image set. Note that if there is no defect type exceeding a predetermined determination probability (for example, 50%) in the image included in the set, the defect type is classified as “unknown”. If the defect type is classified as “unknown” in step S205, the process proceeds to step S207. If the defect type is classified as any other defect type, the process ends (step S206).

  In step S207, an image of the same defect image set classified as an unknown defect type is displayed on the display device, a warning is given to the user for confirmation, and the classification process ends. With the configuration as described above, the user can determine an image to be preferentially confirmed from among a large number of images.

Moreover, the result of the above classification can also be used as teacher data for an inference model. Even the same defect image set finally classified as “unknown” can be used as teacher data. Specifically, after the warning in step S207, if the user confirms the set and the defect type having the highest determination probability matches the actual defect, the same defect image is used as the teacher data for the defect. Add images included in the set. On the other hand, when a defect type different from the defect type with the highest determination probability is an actual defect, an image included in the same defect image set is registered as teacher data of the other defect. In this way, more correct teacher data can be accumulated.

  In addition, because of the function of the teacher data registration unit 33, when the teacher data is registered, images by a plurality of inspection methods showing the same defect are displayed in parallel at the same time, and input of the defect type is accepted at once, so that the accuracy and efficiency are improved. Defect image data can often be registered as teacher data. A learned model with higher determination accuracy can be created by learning based on the correct teacher data registered in this way.

<Modification>
In the defect type classification process of the above embodiment, the defect type classification based on the feature amount is performed (step S203), and then the defect type classification based on the learned inference model is performed (step S205). The order may be changed.

  In the above embodiment, the inference model generated by the deep learning method is used. However, a model generated by another machine learning method may be used. Further, classification processing may be performed by combining models generated by a plurality of machine learning techniques.

  In the above embodiment, the inspection management device 3 has the defect type counting unit 34 that counts the number of the same defect image sets after the defect type classification. A defect number measurement unit that counts the same defect image set may be provided. Further, in this case, it is determined whether or not the number of defects calculated by the defect number measuring unit exceeds a predetermined value, and if it exceeds, a notification to that effect is given to the user via the output device. A defect number warning unit may be further provided.

  Further, in the above embodiment, all the light sources of the illumination system irradiate visible light, and the measurement system includes the same number of cameras as the number of light sources, but the configuration of the illumination system and the measurement system is as follows. It is not necessarily limited to this. For example, a part or all of the light source may be infrared rays, or a spectroscopic prism may be provided in a measurement system camera so that a plurality of different wavelengths of light can be detected by a single camera. Moreover, it is good also as a structure which does not acquire any of a surface reflection image, a back surface reflection image, and a transmission image.

<Others>
The description of the above-described embodiments is merely illustrative of the present invention, and the present invention is not limited to the above specific modes. The present invention can be variously modified within the scope of its technical idea. For example, in each of the above examples, the illumination system and the measurement system are fixed and the inspection object T is moved. Alternatively, the inspection object T may be fixed and the illumination system and the measurement system may be moved. Good.

One aspect of the present invention is an inspection management system (1) for inspecting a sheet-like object to be inspected, and a plurality of imaging means for imaging the appearance of the object to be inspected (T) by different methods ( 221; 222; 223) and a detection means (231) for detecting a defect of the object to be inspected based on respective images photographed by the plurality of photographing means, For each of the plurality of different types of photographing means, a storage unit in which defect image data in which a defect detected by the detection unit is photographed is recorded, and a set of the defect image data recorded in the storage unit An inspection management unit (3) comprising the same defect image integration means (31) for processing a plurality of defect image data in which the same defect in the inspection object is photographed as one same defect image set; Yes That is an inspection management system.

  According to another aspect of the present invention, there is provided a method for managing the appearance inspection of a sheet-like inspection object, the first step (S101) for photographing the inspection object by two or more different methods. A second step (S102) for detecting a defect of the inspection object based on images of the inspection object taken by a plurality of different imaging methods imaged in the first step; and a defect detected in the second step. A third step (S103) for recording the defect image data in which the image is photographed, and a plurality of images in which the same defect in the inspection object is photographed among the set of defect image data recorded in the third step. In the fourth step (S104) of integrating defect image data as one identical defect image set, the number of identical defect image sets integrated in the fourth step is counted for each predetermined inspection unit. A test management method having a fifth step of calculating the number of defects (S105), the.

DESCRIPTION OF SYMBOLS 1, 9 ... Inspection management system 2 ... Appearance inspection apparatus 211, 911 ... Surface reflection light source 212, 912 ... Back surface reflection light source 213, 913 ... Transmission light source 221 ... Surface reflection light camera 222 ... surface reflected light camera 223 ... transmitted light camera 23, 93 ... control terminal 3 ... inspection management device 921 ... surface imaging camera T ... inspection object

Claims (16)

An inspection management system for managing inspection of a sheet-like inspection object,
A plurality of photographing means for photographing the appearance of the inspection object by different methods; and a detection means for detecting defects of the inspection object based on respective images photographed by the plurality of photographing means. An appearance inspection section;
For each of the plurality of different types of photographing means, a storage unit in which defect image data in which the defect detected by the detecting means is photographed is recorded;
Same defect image integration means for processing a plurality of defect image data in which the same defect in the inspection object is photographed from one set of the defect image data recorded in the storage unit as one same defect image set An inspection management department comprising:
Having an inspection management system. The inspection management unit
It further comprises defect counting means for counting the number of the same defect image sets and calculating the number of defects for each predetermined inspection unit.
The inspection management system according to claim 1, wherein: It further has an output part,
The inspection management unit
In the case where the calculated number of defects exceeds a predetermined value, it further comprises defect number warning means for notifying that via the output unit.
The inspection management system according to claim 2, wherein: The inspection management unit
For each of the same defect image set, further comprising a defect type classification means for classifying the type of the defect,
The inspection management system according to any one of claims 1 to 3, wherein The inspection management unit
A defect type counting unit that counts the number of the same defect image sets for each defect type classified by the defect type classification unit, and calculates the number of defects for each defect type for each predetermined inspection unit;
The examination management system according to claim 4 characterized by things. Has an output section,
When the calculated defect number for each defect type exceeds a predetermined value determined for each defect type, the inspection management unit notifies the defect type defect number via the output unit. Further comprising a warning means,
The inspection management system according to claim 5, wherein:   The inspection management system according to claim 4, wherein the defect type classification means includes inference means that has been learned by a deep learning technique. A display unit;
The inspection management unit
A deep learning teacher that simultaneously displays the defect image data constituting the set for each of the same defect image set on the display unit, and requests the user to register the defect type corresponding to the displayed defect image data set. A data registration means;
The inspection management system according to claim 7, wherein: The defect type classification means includes
If the defect type classification of the same defect image set can not be performed with an accuracy of a predetermined accuracy or more, classify the defect type for the same defect image set as unknown,
The inspection management system according to any one of claims 4 to 8, wherein The defect type classification means includes
For the same defect image set that classifies the defect type as unknown, determine the classification of the defect type by the user,
The inspection management system according to claim 9, wherein: Two or more photographing means for photographing the appearance of the inspection object by different methods,
Surface photographing means for photographing an image by reflected light reflected by the first surface of the inspection object, and an image by reflected light reflected by the second surface opposite to the first surface of the inspection object. Including any two or more of a back surface photographing means and a transmitted light photographing means for photographing an image by transmitted light that has passed through the inspection object,
The inspection management system according to any one of claims 1 to 10, wherein Two or more photographing means for photographing the appearance of the inspection object by different methods,
First wavelength imaging means for imaging the inspection object with light of a first wavelength, and second wavelength imaging means for imaging the inspection object with light of a wavelength different from the first wavelength,
The inspection management system according to any one of claims 1 to 11, wherein: The same defect image integration means includes:
Integrating a plurality of different defect image data having the same position of the defect in the inspection object as one same defect image set;
The inspection management system according to any one of claims 1 to 12, wherein the inspection management system is characterized by that. An inspection management apparatus for processing defect image data acquired by photographing the appearance of a sheet-like inspection object using a plurality of different photographing methods,
An inspection management apparatus comprising: an identical defect image integration unit that processes a plurality of defect image data in which an identical defect in the inspection object is photographed as one identical defect image set from the set of defect image data. An inspection management method for managing the appearance inspection of a sheet-like inspection object,
A first step of photographing the inspection object by two or more different methods;
A second step of detecting defects of the inspection object based on images of the inspection object by a plurality of different imaging methods imaged in the first step;
A third step of recording defect image data in which the defect detected in the second step is photographed;
A fourth step of integrating a plurality of defect image data in which the same defect in the inspection object is photographed out of the set of defect image data recorded in the third step as one same defect image set;
And a fifth step of calculating the number of defects by counting the number of identical defect image sets integrated in the fourth step for each predetermined inspection unit. When the number of defects calculated in the fifth step exceeds a predetermined value, the method further includes a sixth step for notifying that effect.
The inspection management method according to claim 15, wherein:

Images