TWI693397B - Inspection management system, inspection management device and inspection management method - Google Patents

Abstract

The present invention provides an inspection management system, an inspection management device, and an inspection management method. In the appearance inspection of a sheet-like article in which the object to be inspected is photographed in a variety of different ways and the defects are detected separately in accordance with each shooting method, the number of different To manage the defects caused by multiple defective image materials representing the same defect. The inspection management system is used to inspect the sheet-shaped object to be inspected. The inspection management system includes an appearance inspection unit including a plurality of imaging elements that image the appearance of the object to be inspected T in different ways, and based on A detection element that detects the defects of the object under inspection by each image captured by the plurality of imaging elements; the storage section records, for each of the imaging elements of the various modes, the images detected by the detection element Defect image data of the defect; and the inspection management unit, including the same defect image merging element, the same defect image merging element is the same as that recorded in the inspection object recorded in the storage unit Multiple defective image materials of a defect are treated as a same defective image group.

Description

Inspection management system, inspection management device and inspection management method

The present invention relates to a technique for detecting an abnormal part of a sheet-shaped object to be inspected, and particularly relates to a technique for analyzing a detected abnormality.

In a production line for manufacturing or processing sheet-like articles, an image is used to detect an abnormality (mixing of foreign matter, dirt, wrinkles, etc. in the sheet) in the sheet, the image It is obtained by irradiating visible light or ultraviolet light onto a sheet, and photographing the transmitted light or reflected light with a camera (for example, Patent Literature 1, Patent Literature 2, etc.).

In this type of device, some devices are such as a surface reflection image, a back reflection image, a transmission image, a visible light image, an infrared light image, etc., using a variety of different shooting methods to photograph the object to be inspected, using each image To conduct defect detection (good or bad judgment), which can improve the accuracy of the inspection (reducing the leakage of defects).

Moreover, not only the above-mentioned device is used to detect defects of products to remove defective products, but it is also expected that by classifying the detected defects by each production factor (hereinafter also referred to as category), the defects of each defect The generated frequency is made into data and analyzed, which is used to grasp the quality status of the manufactured goods, find abnormal parts of the manufacturing process in advance, and maintain in advance.

As mentioned above, when effectively using the data classified by defect type, it is important that the number of production and classification are accurate, but when using a variety of different shooting methods (hereinafter also referred to as inspection methods) as described above, there are also It will cause the disadvantage of destroying the accuracy of the information obtained.

For example, even if you want to find the exact number of defects for each defect type, when multiple inspection methods are used, there may be cases where a defect is detected in multiple inspection methods for a defect, so when the defects obtained in each inspection method are If the quantity is simply summed up, it will cause problems such as the quantity is inconsistent with the original defect quantity on the product.

In contrast, Patent Document 2 has proposed an inspection system with a function of comparing defect information about the same defect captured by different camera devices, based on one of the different camera devices determined according to the decision rule The defect information extracted from the image data captured by the camera device is determined to represent the defect information and displayed. However, in this method, the defective image data other than the representative defect information acquired by multiple cameras is discarded and cannot be effectively used.

In addition, when classifying defect categories, artificial intelligence (AI) that uses defect images as teacher materials for in-depth learning can also be used. However, when registering teacher information, there may be a problem that is difficult to identify defect categories. The defect image is a human error that is registered in association with a defect type different from the actual defect type. In addition, at this time, it is also possible to register different defect types for each inspection method for multiple defect image data that should originally display the same defect. Moreover, the materials classified by the AI learned by using the wrong teacher data registered in the above-mentioned manner will cause doubts in accuracy.

In addition, in a method of acquiring defect images in various inspection methods and associating the defect images with defect types and registering them, the more types of inspection methods, the easier it is for teachers to register their teacher materials. The increase is also inefficient. [Prior Art Literature] [Patent Literature]

[Patent Document 1] Japanese Patent Laid-Open No. 2015-172519 [Patent Document 2] Japanese Patent No. 5305002

[Problems to be solved by the invention] The present invention has been completed in view of the actual situation as described above, and an object of the present invention is to provide a method for photographing an object to be inspected by a variety of different methods, and inspecting the appearance of a sheet-shaped article for defects by each imaging method , Can reduce the defects caused by separately managing multiple defective image data indicating the same defect. [Means to solve the problem]

In order to solve the above problem, the inspection management system of the present invention is an inspection management system for inspecting a sheet-shaped object to be inspected, which includes: an appearance inspection unit, including photographing the appearance of the object to be inspected by different methods More than two imaging elements, and a detection element that detects the defect of the inspection object based on each image captured by the plurality of imaging elements; the storage section, for the imaging elements of the various modes, Separately record the defect image data of the defect detected by the detection element; and the inspection management part includes the same defect image merging element, the same defect image merging element Among the set of recorded defect image materials, a plurality of defect image materials that have captured the same defect in the inspection object are processed as a same defect image set.

Here, "shooting by different methods" may be, for example, a method in which the surface of the object to be photographed is different, a method in which the wavelength of the detected light is different, or a method for shooting reflected light The way is different from the way of shooting through light. In addition, as for "the same defect in the inspection object was photographed", as described later, it may be determined as long as the position of the defect in the inspection object is the same. Furthermore, each element constituting the system does not need to be a separate body, and all of them may be housed in an integrated frame, or a part of them may be integrated.

According to the configuration as described above, even if the object to be inspected is photographed by multiple methods, if a plurality of defective image materials with different photographing methods are recorded for one defect in the object to be inspected, the same defect can be expressed Manage multiple data as a group. With this, it is possible to reduce the disadvantages caused by separately managing a plurality of defective image data indicating the same defect.

Furthermore, the inspection management unit may further include a defect count element that counts the number of the same defective image group and calculates the number of defects for each predetermined inspection unit. Here, the "prescribed inspection unit" may be set and changed by the user at any timing. With the above-mentioned structure, instead of separately counting the plurality of defect images taken for the same defect, the number of defects is counted for each defect in the inspection object, so it is possible to calculate an accurate Number of defects.

In addition, the inspection management system may further include an output unit, and the inspection management unit may further include a defect number warning element that notifies the situation via the output unit when the calculated number of defects exceeds a predetermined value. By adopting the structure as described above, when the number of defects exceeds a predetermined value, the user can easily know the situation, so inspection management can be performed efficiently.

In addition, the output unit may be a member that notifies the user in a perceptible manner, for example, it may be a display device such as a liquid crystal display, a voice output device such as a speaker, or a printing device. In addition, these devices may be used in combination for notification.

Furthermore, the inspection management unit may further include a defect type classification element that classifies the type of the defect for each of the same defect image groups. With such a structure, it is possible to prevent a plurality of different image elements that should originally represent the same defect from being classified into different defect categories according to a plurality of different imaging elements.

Furthermore, the inspection management unit may further include counting the number of the same defect image group for each defect category classified by the defect category classification element, and calculating for each predetermined inspection unit The defect category counts the number of defects per defect category. If this structure is used, the exact number of defects can be determined for each defect type.

Furthermore, the inspection management system may include an output unit, and the inspection management unit may further include, when the calculated number of defects for each defect category exceeds a predetermined value determined for each of the defect categories, via The output unit notifies the defect type of the number of defects in the situation, and a warning element. In addition, the defect type defect quantity warning component may also be a component that also notifies the defect category that has exceeded the specified value. With this structure, the user can know the situation when the number of defects exceeds the specified value for each defect type, so that more detailed inspection management can be performed.

Furthermore, when the inspection system is also configured to include the number-of-defects warning element, the output unit may also have a structure common to the number-of-defects warning element.

In addition, the defect category classification element may also include an inference element that has been learned through a deep learning method. In recent years, image recognition technology by artificial intelligence using a deep learning method has achieved remarkable results, and by using this technology, it is possible to automatically classify defect categories efficiently.

In addition, the inspection management system may further include a display unit, and the inspection management unit may further include the display unit simultaneously displaying the defect image data constituting the group for each of the same defect image groups. The user requests to register a teacher data registration component for deep learning of a defect category corresponding to the set of displayed defect image data.

Here, the display unit generally refers to a display device such as a liquid crystal display, but it may also be a device that uses other display elements such as display using a projector. With such a structure, it is possible to uniformly register the defective image data indicating the same defect, so that the registration of teacher materials can be efficiently performed. In addition, the user can observe and compare the defect images obtained by multiple inspection methods contained in the same defect image group to determine and register the defect type, so it is done with only the individual defect image data as the object Compared with the case of registering a defect category, it is possible to reduce the number of cases where the defect category is incorrectly selected or it takes time to judge.

In addition, the defect type classification element may classify the defect type of the same defect image group when the defect type of the same defect image group cannot be classified with an accuracy higher than a predetermined accuracy. Is unknown. With such a structure, it is possible to prevent the situation that the reliability of the finally obtained data is reduced by reluctantly classifying the defect type with low accuracy.

In addition, the defect type classification element may be an element that requests the user to classify the defect type for the same defect image group that classifies the defect type as unknown. With such a structure, it is possible to prevent the same defect image group whose defect category is classified as unknown from being left directly as the information of unknown classification.

In addition, the plurality of imaging elements that image the appearance of the object to be inspected by different methods may be surface imaging elements that include images formed by reflected light reflected by the first surface of the object to be inspected, A backside imaging element that captures an image formed by reflected light reflected by a second surface of the object to be inspected opposite to the first surface, and a image formed by transmitted light that has passed through the object to be inspected Any two or more of the image-transmitted light imaging device. With such a structure, it is possible to detect and/or classify defects corresponding to various types of defects.

In addition, the plurality of imaging elements that image the appearance of the object to be inspected in different ways may include a first wavelength imaging element that images the object to be inspected with light of the first wavelength, and The second wavelength imaging element of the object to be imaged by using light of a wavelength different from the first wavelength. With such a structure, it is possible to detect and/or classify defects corresponding to various types of defects.

In addition, the same defect image merging element may also merge a plurality of different defect image data in the same range where the position of the defect in the inspection object is the same defect image group.

Here, the meaning of "within the range of the same position" is not limited to the range in which the positions are exactly the same, but also includes the case of being within a predetermined allowable range. As a method of identifying whether the photographed defect is the same defect among a plurality of defective image materials, a method of identifying the material with the same position of the defect in the product as the image material that captured the same defect. When determining the position of the defect in the test object, for example, the position of the defect may be calculated based on the predetermined placement location of each imaging element, imaging range, conveyance speed of the test object, and the like.

In addition, in order to solve the above-mentioned problems, the inspection management device of the present invention is an inspection management device that photographs the appearance of a sheet-like object to be inspected and processes the acquired defective image data by using various shooting methods. It includes the same defect image merging element that processes multiple defect image materials that have captured the same defect in the inspection object as one same defect image group from the collection of the defect image materials.

In addition, in order to solve the above problem, the inspection management method of the present invention is a method for managing the appearance inspection of a sheet-shaped object to be inspected, which includes a first step of photographing the object to be inspected by two or more different methods The second step, based on the image of the inspection object formed by using a variety of different shooting methods taken in the first step, to detect the defects of the inspection object; the third step, recording the Defect image data of the defect detected in the second step; in the fourth step, the same defect in the object to be inspected is captured from the set of defect image data recorded in the third step Multiple defective image data are merged into a same defective image group; and in the fifth step, the number of the same defective image group merged in the fourth step is counted for each prescribed inspection unit, And calculate the number of defects.

In addition, the inspection management method may also include a sixth step, that is, when the number of defects calculated in the fifth step exceeds a prescribed value, notify the situation.

Furthermore, as long as there is no technical contradiction, the processes or components can be freely combined and implemented. [Effect of invention]

According to the present invention, in the visual inspection of a sheet-like article in which the object to be inspected is photographed in a variety of different ways and the defects are separately detected according to each shooting method, it is possible to reduce the number of defective images representing the same defect by separately managing Defects caused by information.

Hereinafter, an example of an embodiment of the present invention will be described with reference to the drawings.

<Application example> The present invention can be applied as the inspection management system 9 shown in FIG. 1, for example. FIG. 1 is a diagram schematically showing a configuration example of an inspection management system 9 of this application example. The inspection management system 9 is a system for detecting defects of sheet-shaped articles and managing information on inspections. The main structural elements include a surface reflection light source 911, a back reflection light source 912, a transmission light source 913 of a lighting system, and measurement The system has a front camera 921, a rear camera 922, a control terminal 93, and a transport mechanism (not shown).

As shown in FIG. 1, the test object T is transported in a horizontal direction (arrow direction) by a transport mechanism (not shown), and the appearance of the test object T is continuously acquired by the measurement system during the transport process Like, and implement inspection based on this. The test object T is formed into a sheet shape, and for example, paper, cloth, film, resin, cellulose, etc. can be exemplified. In addition, it is not limited to a single raw material, but may be a packaging paper such as a film formed by bonding a film and a nonwoven fabric, and has a multi-layer sheet body. In addition, it can also be dried seaweed and other foods.

The surface reflection light source 911 of the illumination system is arranged to irradiate visible light (for example, white light) onto the surface (first surface) of the object T, and the rear surface reflection light source 912 similarly irradiates visible light to the object T The back side (second side). In addition, the transmission light source 913 is arranged to irradiate infrared rays to the back surface (second surface) of the test object T.

The surface imaging camera 921 of the measurement system, although not shown, includes a signal output unit, a visible light receiving sensor, an infrared receiving sensor, a beam splitter, and a lens, and is arranged to photograph the surface of the object T . Specifically, the light irradiated from the surface reflection light source 911 and reflected by the surface of the object T (hereinafter referred to as surface reflected light), and the infrared light irradiated from the transmission light source 913 and transmitted through the object T (hereinafter referred to as transmission) Light) while shooting the object T. The beam splitter divides the light incident on the camera into at least wavelength light and infrared light in the visible light region, and the corresponding sensors receive the light. In the sensor, for example, a charge coupled device (Charge Coupled Device, CCD) or a complementary metal oxide semiconductor (CMOS) sensor may be used.

In addition, although not shown, the backside camera 922 includes a signal output unit, a light-receiving sensor capable of detecting light in the visible light region, and a lens, and is arranged so as to image the backside of the object T to be inspected. Specifically, the object T is photographed by the light irradiated from the back surface reflection light source 912 and reflected by the back surface of the object T (hereinafter referred to as back surface reflected light). As the light receiving sensor, for example, a CCD sensor or a CMOS sensor can be used.

The control terminal 93 performs control of the lighting system, the measurement system, and the transport mechanism, and also processes various information for inspection. As a hardware structure, it includes various input/output devices, processors, storage devices, etc. As a functional module, it includes a defect detection unit 931, a defect position determination unit 932, a defect image storage unit 933, a same defect image merge unit 934, Defect counting section 935, defect number warning section 936, and defect category classification section 937.

The defect detection unit 931 detects defects included in the object T based on the image signals input from the cameras of the measurement system. Defect detection is performed by, for example, determining whether the feature quantity obtained from the captured image has exceeded a predetermined threshold value. In the feature quantity, for example, brightness and the like, brightness, saturation, and color can be used.

In this application example, the image formed by the surface reflected light (hereinafter referred to as the surface reflected image) and the image formed by the transmitted light (hereinafter referred to as the transmitted image) captured by the surface shooting camera 921, and the back surface The image formed by the back reflected light (hereinafter referred to as the back reflected image) captured by the shooting camera 922 performs the determination of the feature amount.

When a defect is determined, the defect position determination unit 932 determines where the defect T is located in the object T. The position can be determined based on, for example, a predetermined transport speed of the object T to be inspected, the installation position of the camera, the size of the object T to be inspected, and the elapsed time from the start of the inspection.

When the defect image storage unit 933 detects a defect, the defect image (hereinafter referred to as a defect image) captured by each camera is compared with the position of the defect determined by the defect position determination unit 932 The information is related and recorded in the storage device. In addition, not only the image of the imaging mode in which the defect is detected, but also the image of the corresponding part of the other imaging mode is recorded in the storage device.

The same defect image merging unit 934, when a plurality of different defect images having the same defect position determined by the defect position determining unit 932 have been recorded in the storage device, use the plurality of images having the same defect position as a Associate the same defective image group. The plurality of defective images formed as a group as described above will be processed integrally when the display to the display device, various data analysis, and the like are performed later. In addition, the same defective image group may be re-recorded in the storage device as a group.

The defect counting unit 935 counts the number of the same defective image group for each predetermined inspection unit, and calculates the number of defects for each inspection unit. Here, the predetermined inspection unit may be either a predetermined number (for example, one roll, one batch, several batches) or a predetermined time (for example, one day, one week, one month). Since the number of defects is calculated in the manner described above, the exact number of defects per predetermined inspection unit can be obtained without repeatedly counting a plurality of defective image materials representing the same defect in the object T under inspection.

The defect number warning unit 936 determines whether the number of defects calculated by the defect counting unit 935 exceeds a predetermined value, and when it is determined that the number of defects exceeds the predetermined value, the situation is output from an unillustrated output device (for example, a liquid crystal display, a speaker, etc.) ) Output and notify the user.

The defect category classification unit 937 classifies the defect image data merged into the same defect image group according to the type of defect, and associates the classified category with the defect image data to record. The classification of the classified defects can be arbitrarily set by the user, for example, it can be set up with foreign matter mixing, dirt, wrinkles, holes and the like, or it can be classified into finer categories (for example, insects, wood chips, metal foreign objects, Grease, scale, large holes, small holes, etc.).

As a method of classifying the defect type, it can be based on the feature amount of any one of the surface reflection image, the back reflection image, and the transmission image, or by the feature of the defect portion of each image The comparison of the quantities can also be carried out by whether or not defects are detected in each inspection method, and combinations thereof. Also, these methods can be used in combination for classification. By classifying a plurality of defect image data indicating the same defect as a group and classifying the defect type as described above, it is comparable to the case of classifying the defect type based on a single image where the defect is detected To classify the defect category more accurately.

2 is a flowchart showing the flow of processing performed in the inspection management system 9 of this application example. The inspection management system 9 first acquires the surface reflection image, the back reflection image, and the transmission image by the measurement system (step S101), and based on the image, the defect detection unit 931 performs defect detection (step S102). Next, the defective image storage unit 933 records the defective image in the storage device (step S103). Then, the same defect image merging unit 934 merges a plurality of images having the same position of the defect in the inspection object from the recorded defect image into a group of the same defect image group (step S104). Next, the defect counting unit 935 counts the number of groups of the same defective image group for each predetermined inspection unit, and calculates the number of defects for each inspection unit (step S105). Here, the defect number warning unit 936 determines whether the number of defects calculated in step S105 exceeds a predetermined value (threshold value) (step S106), and when it is determined to be exceeded, notifies this (step S107) and proceeds to step S108 . On the other hand, when it is determined that the number of defects does not exceed the predetermined value, the process directly proceeds to step S108. Then, in step S108, the defect category classification unit 937 classifies the defect categories of the defect image materials merged into the same defect image group, and records the series of processes to end. Furthermore, the processing from the measurement of the number of defects to the notification when the number of defects exceeds a predetermined value (steps S105 to S107) and the processing of step S108 may be reversed.

With the configuration of the inspection management system 9 of this application example as described above, a plurality of defect image data indicating the same defect can be grouped to count and classify the defect type, so it is possible to reduce the number of individual management indicators. Defects caused by multiple defective image materials of the same defect (for example, the classification of defect types with low accuracy that repeatedly counts the same defect, etc.).

<Example> Hereinafter, an example of a form for implementing the present invention will be described in further detail. However, the size, material, shape, relative arrangement, etc. of the component parts described in the above-mentioned embodiments are not intended to limit the scope of the present invention to these unless otherwise disclosed.

(system structure) FIG. 3 is a diagram schematically showing a configuration example of the inspection management system 1 of this embodiment. As shown in FIG. 3, the inspection management system 1 of this embodiment includes an appearance inspection device 2 and an inspection management device 3 as main structures.

(Appearance inspection device) The appearance inspection apparatus 2 is an apparatus that acquires an appearance image of a sheet-like article and detects defects based on the image, and includes an illumination system, a measurement system, a transport mechanism (not shown), and a control terminal 23 as The main structure.

The test object T is transported in a horizontal direction (arrow direction) by a transport mechanism not shown. During the transport process, the appearance image of the test object T is continuously acquired by the measurement system, and based on this Carry out inspections. The object to be inspected T is formed into a sheet shape, and for example, paper, cloth, film, etc. can be exemplified. In addition, it is not limited to a single raw material, but may be a packaging paper such as a film formed by bonding a film and a nonwoven fabric, and has a multi-layer sheet body. It can also be dried seaweed and other foods.

The illumination system includes a surface reflection light source 211 that irradiates the surface of the object T with visible light (for example, white light), a back surface reflection light source 212 that irradiates the surface of the object T with visible light, and a transmission that irradiates the back surface of the object T with visible light Light source 213. Among these light sources, for example, light-emitting diode (LED) lighting can also be used.

The measurement system includes: a surface reflection light camera 221 that captures light irradiated from the surface reflection light source 211 and reflected by the surface of the object T (hereinafter referred to as surface reflection light); a rear reflection light camera 222 that captures the light reflected from the rear reflection light source 212 The light reflected by the back surface of the test object T (hereinafter referred to as back-reflected light); and the light transmitted through the light camera 223 and transmitted through the light source 213 and transmitted through the test object T. In addition, each camera constituting the measurement system corresponds to the imaging element in the present invention.

Each camera may use a light-receiving sensor, a lens that can detect the light captured by it, and a signal output portion sensor such as a CCD sensor or a CMOS sensor, for example.

The control terminal 23 controls the lighting system, the measurement system, and the transport mechanism, and performs various information processing. As a hardware structure, it includes an input/output device, a processor, a storage device, etc., and as a functional module, it includes a defect detection unit 231, a defect position determination unit 232, and a defect image storage unit 233.

The defect detection unit 231 detects defects contained in the object T based on the image signals input from the cameras of the measurement system. The detection of the defect is performed by determining whether, for example, the feature quantity obtained from the captured image exceeds a predetermined threshold value. In the feature quantity, for example, brightness and the like, brightness, saturation, and color can be used.

In this embodiment, for the image captured by the surface reflection light camera 221 (hereinafter referred to as the surface reflection image), the image captured by the back reflection light camera 222 (hereinafter referred to as the back reflection image) and the transmission light camera The image captured by 223 (hereinafter referred to as a transmission image) is subjected to feature value determination.

The defect position determination unit 232 determines where the defect T is located when the defect is detected from the object T. The position can be determined based on, for example, a predetermined transport speed of the object T to be inspected, a camera installation position, a size of the object T to be inspected, and an elapsed time from the start of the inspection.

When the defect image storage unit 233 detects a defect, the defect image (hereinafter referred to as a defect image) captured by each camera and the position of the defect determined by the defect position determination unit 232 The information is correlated and recorded in the storage device (hereinafter the recorded data is referred to as defective image data). In addition, it may be configured that not only the image of the imaging method in which the defect is detected, but also the image of the part corresponding to the other imaging method is recorded in the storage device.

(Check management device) The appearance inspection device 2 is connected to the inspection management device 3 via a network (local area network (LAN)). The inspection management device 3 is a device that acquires inspection information from the appearance inspection device 2 and processes the information, including a central processing unit (CPU) (processor), a main storage device (memory), General-purpose storage devices (hard drives, etc.), input devices (keyboards, mice, controllers, touch panels, etc.), output devices (liquid crystal displays, speakers, printers, etc.), etc. computer system.

Furthermore, the inspection management device 3 may include either one computer or multiple computers. Alternatively, all or part of the functions of the inspection management device 3 may be installed on the control terminal 23 of the visual inspection device 2. Alternatively, a part of the functions of the inspection management device 3 may be realized by a server on the network (cloud server, etc.).

The CPU of the inspection management device 3 of this embodiment includes the same defect image merging unit 31, defect category classification unit 32, teacher profile registration unit 33, defect category counting unit 34, and defect category defect number warning unit 35 as functional modules.

The same defect image merging unit 31 acquires defect image data from the appearance inspection device 2 and merges a plurality of defect images having the same position of the defect in the inspection object T into a group of the same defect image group. The plurality of defective images formed as a group as described above are recorded in the storage device as a group, and are processed integrally during display to the display device, various data analysis, and the like.

The defect type classification unit 32 classifies the defect image data merged into the same defect image group according to the defect type. The classification of the classified defects can be arbitrarily set by the user. For example, it is possible to set the classification of foreign matter mixing, dirt, wrinkles, holes, etc., or it can be classified into finer categories (for example, insects, wood chips, metal foreign bodies, grease, Scale, large holes, small holes, etc.).

As a method of classifying the defect type, it can be based on the feature amount of any one of the surface reflection image, the back reflection image, and the transmission image, or by the feature amount of the defect portion of each image The comparison is carried out, and it can also be performed by whether a defect is detected in each inspection method, a combination of inspection methods, and the like. In addition, these methods may be combined for classification.

In the present embodiment, the defect category classification unit 32 performs defect category classification in conjunction with inference processing using a learned model, which is generated by a deep learning method. In addition, the flow of defect classification processing will be described later.

The teacher profile registration unit 33 has a function of accepting the registration of teacher profiles for deep learning of the artificial intelligence of the defect category classification unit 32, and after displaying multiple defect images constituting the same defect image group on the display device, the The requester inputs the defect category corresponding to the same defect image group.

The defect type counting unit 34 counts the number of the same defect image group classified by the defect type for each type, and calculates the number of defects of each defect type for each predetermined inspection unit. Here, the predetermined inspection unit may be either a predetermined number (for example, one roll, one batch, several batches) or a predetermined time (for example, one day, one week, one month).

The defect category defect number warning unit 35 determines whether the number of defects for each defect category calculated by the defect category counting unit exceeds a predetermined value prescribed for each of the defect categories, and when it is determined that there is a defect exceeding the prescribed value, the number of defects is calculated In the case of a defect type, the situation is output from the output device and notified to the user. Here, the output by the output device may be displayed by a display device, or may sound an alarm from a speaker, or may be printed by a printer. Furthermore, these devices may be used in combination for notification. In addition, the defect type defect number warning unit 35 may also be set to notify the type of defects exceeding a predetermined value at the same time.

(Processing flow for defect category classification) Next, based on FIG. 4, the flow of processing when the defect type classification unit 32 classifies the defect type will be described. FIG. 4 is a flowchart showing a flow of processing for classifying defects in the present embodiment.

The defect type classification unit 32 first classifies the defect type by a combination of the presence or absence of defect detection in the same defect image group as the target defect image group (step S201 ). Depending on the type of defect, there may be cases where it is necessary to detect by a specific inspection method, or it cannot be detected by a specific inspection method, so the presence or absence of images of each inspection method included in the group (ie, whether to detect during inspection Defects) under the combination of conditions, to determine the type of defect. Furthermore, when recording defective image data, when not only the image of the imaging mode in which the defect is detected, but also the image of the corresponding part of the other imaging mode is recorded in the storage device, as long as If identification information for defect detection is given, the same processing can be performed using the combination of the identification information. When the classification of the defect type has been completed in step S201, the process ends, and when the classification has not been completed, the process proceeds to step S203 (step S202).

In step S203, the defect type classification unit 32 performs defect type classification using a predetermined feature amount. Specifically, when there is an image that meets the condition for determining the feature amount among the defect images included in the same defect image group, the same defect image group is classified as belonging to the defect category. For example, when the peak level of the brightness of the transmitted image is 100 or more (the highest value is 255), and the area of the defect is 1 mm ² or more, the defect category of the same defect image group is classified For holes. When the classification of the defect category has been completed in step S203, the process ends, and when the classification has not been completed, the process proceeds to step S205 (step S204).

In step S205, the defect type classification unit 32 classifies the defect type using the learned inference model generated by the deep learning method for each inspection method in advance for the same defect image group. Specifically, using the learned model of the inspection method corresponding to each of the surface reflection image, the back reflection image, and the transmission image to make an inference, the judgment accuracy rate of each defect type is calculated (that is, as a predetermined The accuracy rate of the defect category). Then, the defect type having the largest value among the determination accuracy rates of the defect types of all images included in the group is classified as the defect type of the same defect image group. Furthermore, when there is no defect category that exceeds the specified judgment accuracy rate (for example, 50%) in the images contained in the group, the defect category is classified as "unknown". When the defect category is classified as "unknown" in step S205, the process proceeds to step S207, and when the defect category is classified as other than this, the process is terminated (step S206).

In step S207, after displaying the images of the same defect image group whose defect category is classified as unknown to the display device, the user is warned to prompt confirmation, and the classification process ends. With the structure as described above, the user can judge the image to be confirmed preferentially from the large number of existing images.

In addition, the results of the classification can be further used as teacher data of the inference model. Even the same defective image group whose final category is classified as "unknown" can be used as teacher information. Specifically, after the warning in step S207, the user confirms the group, and if it is determined that the defect type with the highest accuracy rate matches the actual defect, the images included in the same defect image group are added Teacher information as the defect. On the other hand, when the defect category different from the defect category with the highest determination accuracy rate is an actual defect, the images included in the same defect image group are registered as the teacher's profile of the other defects. By the way described above, more accurate teacher information can be saved.

Also, with the function of the teacher data registration unit 33, when registering teacher data, images obtained by multiple inspection methods representing the same defect are displayed side by side at once, and input of defect types is accepted uniformly, so it can be accurate and efficient Register defective image materials as teacher materials. By learning based on the correct teacher data registered as described above, it is possible to create a completed learning model with higher determination accuracy.

<Variation> Furthermore, in the processing of the defect category classification of the above embodiment, after the defect category classification by the feature amount (step S203), the defect category classification by the inference model after learning is completed (step S205), but also The order of these processes can be changed.

In addition, in the above-mentioned embodiment, the inference model generated by the deep learning method is used, but a model generated by other mechanical learning methods may also be used. In addition, classification processing can also be performed in combination with models generated by multiple machine learning methods.

In addition, in the above-mentioned embodiment, the inspection management device 3 is a structure including a defect type counting unit 34 that counts the number of the same defect image group after the defect type is classified, but this structure may be replaced and include a defect A defect number measuring section that counts the same defect image group before category classification. In addition, at this time, a defect number warning unit may be further included, and the defect number warning unit determines whether the number of defects calculated by the defect number measurement unit exceeds a predetermined value, and when it exceeds, informs the use of the situation via an output device By.

In the above-mentioned embodiment, the light source of the illumination system is a member that irradiates all visible light, and the measurement system includes the same number of cameras as the number of light sources. However, the configurations of the illumination system and the measurement system are not necessarily limited to this. For example, a part or all of the light source may be infrared light, or a spectroscopic lens may be provided in the camera of the measurement system, and a single camera may be used to detect a plurality of lights of different wavelengths. In addition, it may be configured not to acquire any of the front reflection image, the back reflection image, and the transmission image.

<Others> The descriptions of the above embodiments are merely illustrative for explaining the present invention, and the present invention is not limited to the specific forms. The present invention can be variously modified within the scope of its technical idea. For example, in the above examples, the illumination system and the measurement system are fixed to move the test object T, but instead of this structure, the test object T may be fixed and the illumination system and measurement system may be moved.

An aspect of the present invention is an inspection management system (1) for inspecting a sheet-shaped object to be inspected, which includes an appearance inspection unit (2) including photographing the object to be inspected (T) by different methods A plurality of imaging elements (221, 222, 223) of the external appearance, and a detection element (231) that detects defects of the object to be inspected based on the images captured by the plurality of imaging elements; the storage section, For the imaging elements of the various different modes, respectively record the defect image data of the defects detected by the detection element; and the inspection management part (3), including the same defect image merging element (31), The same defect image merging element (31) is a plurality of defect image materials that have captured the same defect in the inspection object from the collection of the defect image data recorded in the storage unit , As a group of the same defective image.

In addition, another aspect of the present invention is a method for managing the appearance inspection of a sheet-shaped object to be inspected, which includes: a first step (S101) of photographing the object to be inspected by two or more different methods; The second step (S102), based on the images of the object to be inspected obtained by the multiple different shooting methods captured in the first step, detects defects of the object to be inspected; the third step (S103), Record the defect image data of the defect detected in the second step; the fourth step (S104), among the collection of the defect image data recorded in the third step, the Multiple defect image materials of the same defect in the inspection object are merged into one same defect image group; and the fifth step (S105), according to each specified inspection unit, the merged in the fourth step The number of the same defective image group is counted, and the number of defects is calculated.

1, 9‧‧‧ inspection management system 2‧‧‧Appearance inspection device (Appearance Inspection Department) 3‧‧‧ Inspection management device (Inspection Management Department) 23.93‧‧‧Control terminal 31‧‧‧ The same defect image merging unit (the same defect image merging component) 934‧‧‧Defective Image Merging Department 32、937‧‧‧Defect classification department 33‧‧‧ Teacher Information Registration Department 34‧‧‧Defect category counting department 35‧‧‧ Defect Type Defect Quantity Warning Department 211、911‧‧‧Surface reflection light source 212, 912‧‧‧ Rear reflective light source 213、913‧‧‧Through the light source 221‧‧‧Surface reflection light camera (photographic element) 222‧‧‧Backside reflected light camera (photographic element) 223‧‧‧Through the light camera (photographic element) 231‧‧‧Defect Detection Department (Detection Element) 931‧‧‧Defect Inspection Department 232、932‧‧‧Defect position determination department 233, 933‧‧‧ Defective Image Storage Department 922‧‧‧rear camera 935‧‧‧Defect counting department 936‧‧‧ Defect Quantity Warning Department 921‧‧‧Surface shooting camera S101～S108, S201～S207 ‧‧‧ steps T‧‧‧ subject to be inspected

FIG. 1 is a diagram schematically showing a configuration example of an inspection management system of an application example. 2 is a flowchart showing the flow of processing performed in the inspection management system of the application example. FIG. 3 is a diagram schematically showing a configuration example of the inspection management system of the embodiment. FIG. 4 is a flowchart showing a flow of processing for classification of defect types in the embodiment.

1‧‧‧ Inspection management system

2‧‧‧Appearance inspection device (appearance inspection department)

3‧‧‧ Inspection management device (Inspection Management Department)

23‧‧‧Control terminal

31‧‧‧ The same defect image merging unit (the same defect image merging element)

32‧‧‧Defect Category Classification Department

33‧‧‧ Teacher Information Registration Department

34‧‧‧Defect category counting department

35‧‧‧ Defect Type Defect Quantity Warning Department

211‧‧‧surface reflection light source

212‧‧‧Back reflection light source

213‧‧‧Through the light source

221‧‧‧Surface reflection light camera (photographic element)

222‧‧‧Back light camera (imaging element)

223‧‧‧Through the light camera (imaging element)

231‧‧‧Defect Inspection Department (Inspection Element)

232‧‧‧Defect location determination department

233‧‧‧Defective image storage

T‧‧‧ subject to be inspected

Claims (16)

An inspection management system for managing inspection of a sheet-shaped object to be inspected, the inspection management system includes: an appearance inspection unit including a plurality of imaging elements that photograph the appearance of the object to be inspected in different ways, and A detection element that detects the defect of the object to be inspected based on each image captured by the plurality of imaging elements; a storage unit, for the plurality of imaging elements of the different modes, records the Defect image data of the defect detected by the detection element; and an inspection management section including the same defect image merging element, the same defect image merging element is to record the Among the set of defective image materials, a plurality of the defective image materials in which the same defect in the object is photographed are treated as a same defective image group. The inspection management system according to item 1 of the patent application scope, wherein the inspection management section further includes: a defect counting element that counts the number of the same defective image group and calculates for each prescribed inspection unit Number of defects. The inspection management system as described in item 2 of the patent application scope, which further includes: an output unit; and the inspection management unit further includes: a defect number warning element, when the calculated number of defects exceeds a prescribed value, This situation is notified via the output section. The inspection management system according to item 1 of the patent application scope, wherein the inspection management section further includes: a defect category classification element that classifies the defect category for each of the same defect image groups. The inspection management system as described in item 4 of the patent application scope, wherein the inspection management section further includes: a defect category counting element for each defect category classified by the defect category classification element to the same defect map Count the number of image groups, and calculate the number of defects for each defect type according to each specified inspection unit. The inspection management system as described in item 5 of the patent application scope, which includes: an output section; and the inspection management section further includes a defect category defect quantity warning element, the defect category defect quantity warning element is calculated for each defect When the number of defects in a category exceeds a predetermined value defined for each defect category, this is notified via the output unit. The inspection management system according to any one of the items 4 to 6 of the patent application scope, wherein the defect classification component includes an inference component that has been learned through deep learning methods. The inspection management system as described in item 7 of the patent application scope, which further includes: A display unit; and the inspection management unit further includes: a deep learning teacher data registration element that causes the display unit to simultaneously display the defects constituting the same defect image group for each of the same defect image group Image data, and a request to the user to register a defect category corresponding to the group of displayed defect image data. The inspection management system according to item 4 of the patent application scope, wherein when the classification of the defect category of the same defect image group cannot be performed with an accuracy higher than a prescribed accuracy, the defect category classification element The defect category of the same defect image group is classified as unknown. The inspection management system according to item 9 of the patent application scope, wherein the defect category classification element requests the user to classify the defect category for the same defect image group that classifies the defect category as unknown. The inspection management system according to item 1 of the patent application scope, wherein the plurality of imaging elements that photograph the appearance of the object to be inspected by the different methods include: Surface imaging element for an image formed by reflected light reflected from a surface, and a rear imaging element for imaging an image formed by reflected light reflected by a second surface of the object to be inspected opposite to the first surface And any two or more of the transmitted light imaging elements that image the image formed by the transmitted light passing through the object to be inspected. The inspection management system as described in item 1 of the patent application scope, in which The plurality of imaging elements that photograph the appearance of the object to be inspected by the different methods include: a first wavelength imaging element that photographs the object to be inspected with light of the first wavelength; and a second wavelength to photograph The element photographs the object to be inspected by light having a wavelength different from the first wavelength. The inspection management system according to item 1 of the patent application scope, wherein the visual inspection section further includes: a defect position determining element that determines which of the inspection objects the defect detected by the detecting element is located in Part, the same defect image merging element is a plurality of the same position of the defect identified by the defect position determining element in the set of the defect image data recorded in the storage section The different defect image materials are taken as multiple defect image materials that have captured the same defect. An inspection management device that photographs the appearance of a sheet-like object to be inspected and processes the acquired defect image data by using various shooting methods. The inspection management device includes: the same defect image merging element, From the collection of defective image materials, a plurality of the defective image materials that have photographed the same defect in the inspection object are treated as a same defective image group. An inspection management method for managing the appearance inspection of a sheet-shaped object to be inspected. The inspection management method includes: a first step of photographing the object to be inspected by two or more different methods; In the second step, based on the images of the object to be inspected obtained by a variety of different shooting methods taken in the first step, the defects of the object to be inspected are detected; Defect image data of the defect detected in the second step; in the fourth step, the inspected image is captured from the set of the defective image data recorded in the third step A plurality of the defect image materials of the same defect in the same defect are merged into a same defect image group; and the fifth step, according to each specified inspection unit, the merged said in the fourth step The number of the same defective image group is counted to calculate the number of defects. The inspection management method as described in item 15 of the patent application scope, which further includes: a sixth step, which is notified when the number of defects calculated in the fifth step exceeds a prescribed value.

Images