WO2021054376A1 - 学習処理装置及び検査装置 - Google Patents

学習処理装置及び検査装置 Download PDF

Info

Publication number
WO2021054376A1
WO2021054376A1 PCT/JP2020/035141 JP2020035141W WO2021054376A1 WO 2021054376 A1 WO2021054376 A1 WO 2021054376A1 JP 2020035141 W JP2020035141 W JP 2020035141W WO 2021054376 A1 WO2021054376 A1 WO 2021054376A1
Authority
WO
WIPO (PCT)
Prior art keywords
inspection
neural network
learning
network model
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2020/035141
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
謙 和田
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Syntegon Technology KK
Original Assignee
Syntegon Technology KK
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Syntegon Technology KK filed Critical Syntegon Technology KK
Priority to JP2021546938A priority Critical patent/JP7212792B2/ja
Priority to US17/642,423 priority patent/US12217495B2/en
Priority to EP20864924.4A priority patent/EP4016057A4/en
Priority to CN202080065129.4A priority patent/CN114424242A/zh
Publication of WO2021054376A1 publication Critical patent/WO2021054376A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G06T7/0008—Industrial image inspection checking presence/absence
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/90—Investigating the presence of flaws or contamination in a container or its contents
    • G01N21/9018—Dirt detection in containers
    • G01N21/9027—Dirt detection in containers in containers after filling
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G06V10/7747—Organisation of the process, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/776—Validation; Performance evaluation
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G01N2021/8883—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges involving the calculation of gauges, generating models
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/90—Investigating the presence of flaws or contamination in a container or its contents
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection

Definitions

  • the present disclosure relates to a learning processing device for constructing a neural network model used for inspection and an inspection device having a learning processing function.
  • an inspection device that uses an image to inspect an inspection object.
  • an inspection device that inspects the presence or absence of cracks in a drug container and foreign substances in the container by using an image when manufacturing a drug in a container.
  • the software used for such an inspection uses, for example, rule-based image recognition processing, and has a function of managing a processing procedure for executing a predetermined inspection, parameters to be set, and the like as inspection processing conditions.
  • software is created and verified using test processing conditions during drug manufacturing to comply with the guidelines set by Good Manufacturing Practice (GMP) for manufacturers. It is necessary to be able to confirm whether the inspection processing conditions are the same and valid.
  • GMP Good Manufacturing Practice
  • Patent Document 1 a part of an inspection image obtained by capturing an image of an inspection object is cut out as an input image, and an inspection mark that affects the inspection result of the input image is input according to the cutout position of the input image in the inspection image.
  • An inspection device is disclosed in which an input image embedded in an image and an inspection mark is embedded is input to a neural network, and an inspection result of the input image is determined using the output of the neural network.
  • a neural network is trained based on a training image including an inspection object, a trained neural network that outputs features of the training image is constructed, and the trained neural network is described.
  • a discriminator for judging the quality of the inspection object based on the output feature amount of the learning image is generated by learning, and the judgment image including the inspection object is input to the trained neural network to determine the feature amount of the judgment image.
  • an image inspection device is disclosed in which the feature amount of the image for determination is input to the classifier to determine the quality of the inspection object.
  • the structure of the neural network model cannot be understood by the user. It is not easy to verify that the neural network model used is the same or that the neural network model is valid.
  • the learning processing device is a learning processing device for constructing a neural network model used for inspection of an inspection object based on image data obtained by imaging an inspection object and a neural network model. It is provided with a learning unit that constructs a neural network model by performing learning processing under predetermined learning conditions based on a list of image data including a plurality of learning images, and the learning unit is equipped with a learning unit that constructs a neural network model each time the neural network model is constructed.
  • a learning processing device that embeds model identification data unique to a network model is provided.
  • an inspection device that inspects an inspection object according to a data file of set inspection processing conditions, and is predetermined based on a list of image data including a plurality of learning images.
  • the learning unit that performs learning processing under the learning conditions of the above and builds a neural network model
  • the processing condition setting unit that generates the data file of the inspection processing condition to which the constructed neural network model is applied, and the data file of the inspection processing condition.
  • An inspection unit that determines an abnormality of the inspection object based on the image data obtained by capturing the image of the inspection object and the neural network model, and the learning unit provides unique model identification data each time the neural network model is constructed. Is added to build a neural network model, and the processing condition setting unit is characterized in that each time the applied neural network model is changed, unique condition identification data is added to generate inspection processing condition data. Inspection equipment is provided.
  • a neural network model created and verified and a neural network model used in the inspection stage are used in a learning processing device for constructing a neural network model used for inspection of an inspection object using a neural network model. It is possible to easily verify whether the neural network model to be used is the same.
  • the inspection device according to the present embodiment is used, for example, by being incorporated in a part of a work line of a drug inspection process in a container, and inspects an inspection object using a neural network.
  • the type of neural network is not particularly limited.
  • the inspection device inspects the container for damage such as cracks and cracks, and for foreign matter mixed in the container.
  • Containers are, for example, ampoules, vials, bottles, carples or syringes, but are not limited to these examples.
  • the inspection device is configured to include at least a processor and a storage device.
  • the processor includes, for example, an image processing device such as a GPU (Graphics Processing Unit) and an arithmetic processing device such as a CPU (Central Processing Unit).
  • the storage device may be, for example, one or more storage devices such as HDD (Hard Disk Drive), CD (Compact Disc), DVD (Digital Versatile Disc), SSD (Solid State Drive), USB (Universal Serial Bus) flash, and storage device. Contains a storage medium.
  • FIG. 1 is a schematic view showing a configuration example of the inspection device 1 according to the present embodiment.
  • the inspection device 1 includes an image processing unit 40, an inspection unit 10, a processing condition setting unit 20, a learning unit 30, and a storage unit 50.
  • a part or all of the image processing unit 40, the inspection unit 10, the processing condition setting unit 20, and the learning unit 30 are functions that can be realized by executing a program by the above processor.
  • a part or all of the image processing unit 40, the inspection unit 10, the processing condition setting unit 20, and the learning unit 30 may be composed of updatable firmware or the like, and is a program module executed by a command from the processor. And so on.
  • the storage unit 50 is a function realized by the above-mentioned storage device.
  • the learning unit 30 constituting the inspection device 1 is a component having a function as a learning processing device, and may be configured as an independent processing device having a part of the functions including the learning unit 30.
  • the inspection device 1 is connected to image pickup devices 5a and 5b for photographing a container containing a drug that is continuously conveyed.
  • the image pickup devices 5a and 5b for example, a wide-angle CCD (Charge Coupled Device) camera, a line sensor camera, or the like is used. Although two image pickup devices 5a and 5b are shown in FIG. 1, the number of image pickup devices may be one or three or more.
  • the inspection device 1 may include a lighting device that irradiates the inspection object with light, and a microscope that visually magnifies the inspection object.
  • the inspection device 1 may include an image display monitor, a speaker, and the like.
  • the image display monitor displays, for example, image information or text for the operator to confirm the inspection status.
  • the image display monitor may be a touch panel that accepts operation input by an operator.
  • the speaker is provided, for example, to give guidance on the inspection procedure or to give an alarm when a defect is detected in the drug-containing container.
  • the state of the container can be detected by taking a picture while rotating the container containing the drug that has reached the inspection station, or the state of the drug inside can be taken by taking a picture of the state of the drug in the rotating state to detect the inclusion of foreign matter.
  • each of a plurality of imaging devices inspects a container containing a drug.
  • the image processing unit 40 acquires the captured images taken by the imaging devices 5a and 5b.
  • the image processing unit 40 stores the acquired captured image in the storage unit 50.
  • the image processing unit 40 may perform appropriate image processing suitable for the inspection.
  • the image processing unit 40 may perform a process of cutting out a part of the captured image corresponding to the inspection target range.
  • the image processing unit 40 may perform processing such as compression, normalization, and expansion of captured image data.
  • the learning unit 30 performs neural network learning (deep learning: deep learning) using captured images. That is, a neural network model is constructed by learning images without abnormalities obtained under predetermined inspection conditions and images with various abnormalities.
  • the processing condition setting unit 20 generates a data file of inspection processing conditions for the inspection object.
  • the inspection processing condition data file is a data file that summarizes inspection processing procedures, inspection condition parameters, and the like.
  • the inspection is performed based on the data file of the predetermined inspection conditions, and the inspection conditions in each inspection are stored together with the image data taken for each imaging device, for example. Will be done.
  • an inspection condition data file corresponding to the constructed neural network model is generated from the inspection conditions of the image used for the learning.
  • the user executes the inspection according to the set inspection processing conditions.
  • the inspection unit 10 inspects the inspection target using the trained neural network model. For example, the inspection unit 10 inputs an image of the inspection target into the neural network model, and inspects the inspection target based on the output.
  • the neural network model used is formed by learning under the set inspection processing conditions.
  • the inspection unit 10 may be configured to be able to perform an inspection based on the conventional rule-based image recognition process in addition to the inspection using the neural network model.
  • the inspection device 1 according to the present embodiment described below is configured to be capable of performing inspections using conventional rule-based image recognition processing in addition to inspections using a neural network model.
  • FIG. 2 shows an example of the inspection procedure.
  • steps S11 to S19 after performing inspection processing by the inspection units vsn_PC1 to vsn_PC4 under the conditions set respectively, a determination using a neural network model is performed, and then in step S21, Determine defects in the inspection object.
  • steps S11 to S17 are inspection processes performed by rule-based image recognition processing and the like
  • step S19 is inspection processing using a neural network model.
  • the object to be inspected is a bottle-shaped container
  • steps S11 to S17 there are no large scratches on the shoulders or neck of the container, which are mainly prone to cracks, or foreign matter is mixed in the container.
  • a condition is set so that it can be optically determined whether or not the test is performed, and the inspection is performed, and the judgment is performed based on the image obtained as the inspection result.
  • the imaging devices 5a and 5b that image the container to be inspected are changed according to the setting of the data file of the inspection processing conditions, and the exposure amount, spectrum, exposure time, etc. of the light emitted from the lighting device are changed.
  • a predetermined inspection is performed while changing the temperature.
  • steps S11 to S17 based on the obtained image data, the appearance of the container based on the conventional rule or the presence or absence of an abnormality such as foreign matter mixed in the container is detected.
  • the inspection processing conditions and the images obtained in each inspection are stored in the storage unit 50 in association with the inspections in steps S11 to S17.
  • step S19 the appearance of the container or the presence or absence of defects in the container is determined using the neural network model. Also in step S19, images of a plurality of containers to be inspected are input to the neural network model according to the setting of the data file of the inspection processing conditions, and based on the output, the presence or absence of scratches in the container and the inside of the container are introduced. Inspected for foreign matter. That is, by inputting the container image obtained by one inspection into the neural network model prepared for the inspection, a judgment result about an abnormality (defect) such as a scratch which is an inspection result for the inspection can be obtained. .. Then, for each inspection, the data file of the inspection processing conditions used and the neural network model used are associated and stored.
  • step S21 the pass / fail of the inspection target is determined from the determination results of steps S11 to S19.
  • the container determined to be defective in either the determination result in steps S11 to S17 and the determination result in step S19 can be rejected, or the container determined to be defective in both can be rejected. ..
  • the contents of the processes performed in each of these steps S11 to S21 conventionally known techniques can be adopted, and therefore detailed description thereof will be omitted.
  • the model identification data of the neural network model and the inspection processing condition thereof are stored.
  • the model identification data of the neural network model is given when the neural network model is constructed by learning. That is, each time the data set used for learning the neural network is different, identification data that is difficult to rewrite is added.
  • the model identification data of the neural network model used for the inspection process is stored corresponding to the inspection. Therefore, after the inspection is completed, the model identification data of the neural network model used for each inspection is stored, and the neural network model used for the inspection processing can be verified.
  • a list of image data and setting parameters used when constructing the model are stored. Therefore, when the user discovers some problem in image recognition, the neural network model can be identified from the model identification data, thereby tracking the list of image data and setting parameters used when learning the neural network. Can be done. Therefore, the validity of the neural network model for the inspection performed can be verified.
  • FIG. 3 is a flowchart showing an operation example of the neural network learning process and the inspection process condition setting process.
  • the left column of FIG. 3 shows the operation related to the setting of the inspection processing condition (recipe: Rcp.), And the right column shows the operation related to the data set (Dtst.) Used for generating the neural network model.
  • the processing condition setting unit 20 of the inspection device 1 detects that the neural network model is applied to a specific inspection processing condition based on the input of the user (YES in step S31). That is, a neural network model is constructed by learning, and a data file of inspection processing conditions to be applied to the constructed neural network model is generated.
  • the learning unit 30 sets a data set for learning the neural network (step S33).
  • the data set for learning is a list of image data used for learning a neural network and a data group of setting parameters used for learning.
  • the setting parameters are, for example, parameters such as the depth for deep learning (deep learning) of the neural network, the learning rate, and the error function.
  • the list of image data may be randomly selected from the image data stored in the storage unit 50, or may be specified by the user. That is, image data about the results of inspections performed under specific inspection conditions are stored as examples of abnormalities and no abnormalities, and a neural network model is constructed based on these data sets.
  • the image data may be manually added as image data for learning.
  • the image data obtained by imaging the container is newly registered in the data set and the image is obtained. The data may be trained.
  • the learning unit 30 adds model identification data to the set data set (step S35).
  • the model identification data to be given is identification data that is difficult to rewrite and is given to the data set, and is given individually when even a part of the data set is different.
  • the learning unit 30 always assigns different model identification data so that they can be distinguished as different data sets.
  • the model identification data identifies the neural network model and the dataset used to build it.
  • the model identification data may be, for example, binary data. By using binary data, identification data that can be distinguished from other data sets can be easily incorporated into the data set as identification data that is difficult to rewrite.
  • the learning unit 30 learns the neural network and generates a neural network model (step S37). For example, the learning unit 30 learns the neural network while classifying the image data.
  • the classification may be performed according to, for example, a state in which the container is cracked, a state in which the container is chipped, a state in which foreign matter is mixed in the container, and the degree thereof.
  • the type of neural network and learning method that can be applied are not particularly limited, and an appropriate neural network or learning method is selected according to the content of the inspection.
  • the learning unit 30 adds model identification data to the generated neural network model (step S39).
  • the same identification data as the model identification data given to the data set is given to the neural network model. Even if the neural network model is generated without changing the data set, the hash values will be different, but as long as the data sets are the same, the same identification data (the present embodiment) will be given as the model identification data. Then binary data) is given. On the other hand, different model identification data is added to the neural network model as long as at least part of the dataset has been modified.
  • the processing condition setting unit 20 applies the generated neural network model to the inspection processing condition data file (step S41).
  • the generated neural network model is applied as the neural network model used for the inspection process using the neural network. That is, a data file of the inspection processing conditions corresponding to the constructed neural network model is generated from the inspection processing conditions corresponding to the data set used when the neural network model was constructed.
  • the inspection unit 10 may verify the effectiveness of the neural network model before applying it to the data file of the inspection processing conditions. As a result, the reliability of the inspection using the neural network model can be improved.
  • the processing condition setting unit 20 adds condition identification data to the generated inspection processing condition data file, saves the inspection processing condition data file (step S43), and ends the processing.
  • condition identification data different values are given to each of the data files of the inspection processing conditions in which the model identification data of the neural network model is different.
  • the condition identification data like the model identification data, is identification data that cannot be easily rewritten.
  • condition identification data for example, binary data is used.
  • step S45 determines whether or not the data set has been changed. Specifically, the learning unit 30 determines whether or not the image data for learning has been added and the setting parameters have been added or changed by the user. When the data set has not been changed (S45 / No), the learning unit 30 holds the stored neural network model and the data file of the inspection processing conditions, and ends the processing. On the other hand, when the data set is changed (S45 / Yes), the learning unit 30 returns to step S35 to generate a data file of a new neural network model and inspection processing conditions. At that time, the learning unit 30 newly adds the model identification data and the condition identification data to the data file of the neural network model and the inspection processing condition.
  • steps S31 to S45 Since the processes of steps S31 to S45 are repeated at any time, a new neural network model and a data file of inspection processing conditions are generated when there is a user input or when the data set is changed. ..
  • FIGS. 4 and 5 are diagrams for explaining the model identification data and the condition identification data before and after the change of the data set.
  • FIG. 4 shows a data file of model identification data and condition identification data before changing the data set
  • FIG. 5 shows a data file of model identification data and condition identification data after changing the data set.
  • the data files of FIGS. 4 and 5 are given "111" and "112" as project file numbers (Prj.), Respectively.
  • the first inspection unit vsn_PC1 was photographed by the first imaging camera mr1, the second imaging camera mr2, the third imaging camera mr3, and the fourth imaging camera cm4, respectively.
  • An inspection using a neural network model is performed using a list of image data.
  • the tenth inspection unit vsn_PC10 is at least one of the first image pickup camera mr1, the second image pickup camera mr2, the third image pickup camera mr3, and the fourth image pickup camera cm4, respectively.
  • the inspection by the conventional rule-based image recognition process is performed using the list of the image data taken by each camera.
  • a data file of inspection processing conditions is generated for each inspection performed.
  • the condition identification data (RUN) of the inspection processing condition data file is "010", "003", “021”, “004", "018", "013”. ... "066" is added to each data file.
  • MUN model identification data
  • the model identification data (MUN) of the generated neural network model is updated to "003" and "018", respectively.
  • the condition identification data (RUN) which is the identification data of the data file of the inspection processing condition to which the neural network model is applied, is updated to "012" and "022", respectively.
  • the learning unit 30 adds model identification data that is difficult to rewrite to the neural network every time a part of the data set for generating the neural network model is changed. Generate a network model. Further, the processing condition setting unit 20 adds condition identification data that is difficult to change to the inspection processing condition to which the neural network model having different model identification data is applied, and generates a data file of the inspection processing condition. Therefore, when the user feels a defect in the inspection result by image recognition using the neural network model, the user can track the list of image data used for generating the neural network model used, the setting parameters, and the generation time. it can.
  • condition identification data and the model identification data when the neural network model is used are stored for each camera of each inspection department. Therefore, the network model used for the determination and its inspection conditions can be easily verified. Further, when the network model is changed, the condition identification data is also changed, so that the data set corresponding to the network model can be reliably verified.
  • an inspection processing condition data file is generated corresponding to each inspection, and this is stored together with the condition identification data. In this case, even if the inspection processing conditions up to imaging are the same, condition identification data different from that when the neural network model is used is added.
  • condition identification data is added to the inspection processing condition data file at the time of inspection between the case where the neural network model is not used and the case where the neural network model is used, and the neural network model used is different.
  • another condition identification data is assigned to the corresponding inspection processing condition data file.
  • condition identification data is assigned to each inspection processing condition data file of each inspection, and in vsn_PC2, a determination using a neural network model is performed. If so, another condition identification data (RUN) corresponding to the model is given. ..
  • the neural network model used for the inspection is the model used for generating and verifying the neural network model based on the model identification data or the condition identification data. It is possible to verify the validity such as whether it is the same as or valid.
  • model identification data and the condition identification data to be given are identification data that is difficult to rewrite such as binary data, there is a low possibility that the neural network model or the data file of the inspection processing condition will be tampered with, and the neural network model. It is possible to increase the reliability of the validity of.
  • model identification data is assigned to each generated neural network model and condition identification data is assigned to each inspection processing condition data file, for example, when the container is changed or the same container is used. However, it becomes easy to select the neural network model to be used according to the inspection environment such as when the contents are changed.
  • the inspection unit 10, the processing condition setting unit 20, the learning unit 30, and the image processing unit 40 of the inspection device 1 are composed of one computer device, but the present disclosure is not limited to such an example.
  • Each of the inspection unit 10, the processing condition setting unit 20, the learning unit 30, and the image processing unit 40, or two or more of them may be composed of a plurality of computer devices capable of communicating with each other.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Multimedia (AREA)
  • Quality & Reliability (AREA)
  • Biomedical Technology (AREA)
  • Biochemistry (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Pathology (AREA)
  • Chemical & Material Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Biophysics (AREA)
  • Immunology (AREA)
  • Signal Processing (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)
  • Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
PCT/JP2020/035141 2019-09-17 2020-09-16 学習処理装置及び検査装置 Ceased WO2021054376A1 (ja)

Priority Applications (4)

Application Number Priority Date Filing Date Title
JP2021546938A JP7212792B2 (ja) 2019-09-17 2020-09-16 学習処理装置及び検査装置
US17/642,423 US12217495B2 (en) 2019-09-17 2020-09-16 Learning process device and inspection device
EP20864924.4A EP4016057A4 (en) 2019-09-17 2020-09-16 LEARNING APPARATUS AND TESTING APPARATUS
CN202080065129.4A CN114424242A (zh) 2019-09-17 2020-09-16 学习处理装置及检查装置

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2019168127 2019-09-17
JP2019-168127 2019-09-17

Publications (1)

Publication Number Publication Date
WO2021054376A1 true WO2021054376A1 (ja) 2021-03-25

Family

ID=74883108

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/035141 Ceased WO2021054376A1 (ja) 2019-09-17 2020-09-16 学習処理装置及び検査装置

Country Status (5)

Country Link
US (1) US12217495B2 (https=)
EP (1) EP4016057A4 (https=)
JP (1) JP7212792B2 (https=)
CN (1) CN114424242A (https=)
WO (1) WO2021054376A1 (https=)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102021112479A1 (de) 2021-05-12 2022-11-17 Espera-Werke Gmbh Verfahren zum Betrieb eines Etikettiersystems
WO2023176393A1 (ja) * 2022-03-14 2023-09-21 キヤノン株式会社 認識装置、認識処理方法、及びプログラム
JP2023134183A (ja) * 2022-03-14 2023-09-27 キヤノン株式会社 認識装置、認識処理方法、及びプログラム
US20240386444A1 (en) * 2023-04-17 2024-11-21 Synergie Medicale Brg Inc. Methods, systems, and computer program product for validating a drug product while being held by a drug product packaging system prior to packaging
US12170849B2 (en) 2022-02-04 2024-12-17 Applied Materials, Inc. Pulsed illumination for fluid inspection

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2018198053A (ja) * 2017-05-22 2018-12-13 キヤノン株式会社 情報処理装置、情報処理方法、及びプログラム
JP2019505802A (ja) 2016-01-28 2019-02-28 シーメンス・ヘルスケア・ダイアグノスティックス・インコーポレーテッドＳｉｅｍｅｎｓ Ｈｅａｌｔｈｃａｒｅ Ｄｉａｇｎｏｓｔｉｃｓ Ｉｎｃ． 試料容器と試料の特徴付けのための方法及び装置
JP2019061578A (ja) * 2017-09-27 2019-04-18 富士フイルム株式会社 学習支援装置、学習支援装置の作動方法、学習支援プログラム、学習支援システム、および端末装置
JP2019087181A (ja) 2017-11-10 2019-06-06 アズビル株式会社 画像検査装置および方法

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7676062B2 (en) * 2002-09-03 2010-03-09 Automotive Technologies International Inc. Image processing for vehicular applications applying image comparisons
EP1886257A1 (en) * 2005-05-11 2008-02-13 Optosecurity Inc. Method and system for screening luggage items, cargo containers or persons
WO2018136262A1 (en) * 2017-01-20 2018-07-26 Aquifi, Inc. Systems and methods for defect detection
WO2018216629A1 (ja) 2017-05-22 2018-11-29 キヤノン株式会社 情報処理装置、情報処理方法、及びプログラム
JP6705777B2 (ja) 2017-07-10 2020-06-03 ファナック株式会社 機械学習装置、検査装置及び機械学習方法
US20190096135A1 (en) * 2017-09-26 2019-03-28 Aquifi, Inc. Systems and methods for visual inspection based on augmented reality
JP7002949B2 (ja) * 2018-01-22 2022-01-20 株式会社日立ハイテク 画像評価方法及び画像評価装置
CN108710941A (zh) 2018-04-11 2018-10-26 杭州菲数科技有限公司 用于电子设备的神经网络模型的硬加速方法和装置
CN108805259A (zh) * 2018-05-23 2018-11-13 北京达佳互联信息技术有限公司 神经网络模型训练方法、装置、存储介质及终端设备
KR20230135069A (ko) * 2020-12-18 2023-09-22 스트롱 포스 브이씨엔 포트폴리오 2019, 엘엘씨 밸류 체인 네트워크를 위한 로봇 플릿 관리 및 적층제조

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2019505802A (ja) 2016-01-28 2019-02-28 シーメンス・ヘルスケア・ダイアグノスティックス・インコーポレーテッドＳｉｅｍｅｎｓ Ｈｅａｌｔｈｃａｒｅ Ｄｉａｇｎｏｓｔｉｃｓ Ｉｎｃ． 試料容器と試料の特徴付けのための方法及び装置
JP2018198053A (ja) * 2017-05-22 2018-12-13 キヤノン株式会社 情報処理装置、情報処理方法、及びプログラム
JP2019061578A (ja) * 2017-09-27 2019-04-18 富士フイルム株式会社 学習支援装置、学習支援装置の作動方法、学習支援プログラム、学習支援システム、および端末装置
JP2019087181A (ja) 2017-11-10 2019-06-06 アズビル株式会社 画像検査装置および方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4016057A4

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102021112479A1 (de) 2021-05-12 2022-11-17 Espera-Werke Gmbh Verfahren zum Betrieb eines Etikettiersystems
US12170849B2 (en) 2022-02-04 2024-12-17 Applied Materials, Inc. Pulsed illumination for fluid inspection
WO2023176393A1 (ja) * 2022-03-14 2023-09-21 キヤノン株式会社 認識装置、認識処理方法、及びプログラム
JP2023134183A (ja) * 2022-03-14 2023-09-27 キヤノン株式会社 認識装置、認識処理方法、及びプログラム
JP7799518B2 (ja) 2022-03-14 2026-01-15 キヤノン株式会社 認識装置、認識処理方法、及びプログラム
US20240386444A1 (en) * 2023-04-17 2024-11-21 Synergie Medicale Brg Inc. Methods, systems, and computer program product for validating a drug product while being held by a drug product packaging system prior to packaging

Also Published As

Publication number Publication date
EP4016057A4 (en) 2023-09-06
US20220343640A1 (en) 2022-10-27
CN114424242A (zh) 2022-04-29
EP4016057A1 (en) 2022-06-22
US12217495B2 (en) 2025-02-04
JP7212792B2 (ja) 2023-01-25
JPWO2021054376A1 (https=) 2021-03-25

Similar Documents

Publication Publication Date Title
WO2021054376A1 (ja) 学習処理装置及び検査装置
CN109949305B (zh) 产品表面缺陷检测方法、装置及计算机设备
CN109961421B (zh) 数据生成装置、数据生成方法以及数据生成记录介质
JP7054436B2 (ja) 検出システム、情報処理装置、評価方法及びプログラム
TW202001699A (zh) 利用人工智慧的瑕疵檢測系統及其方法
US11574400B2 (en) System and method for automated visual inspection
KR20080003719A (ko) 결함 검사 장치 및 결함 검사 방법
CN116245808B (zh) 工件缺陷检测方法、装置、电子设备及存储介质
CN114894801B (zh) 检查系统、检查方法及检查程序
TW201500731A (zh) 光學薄膜缺陷鑑別方法
KR102864461B1 (ko) 얼룩 자동 검사 장치 및 이를 이용한 얼룩 검사 방법
CN101408521A (zh) 用于缺陷增强的方法
WO2022044150A1 (ja) 画像生成装置、画像生成方法およびプログラム
KR20160064058A (ko) 영역 지정에 따른 지능형 판정 기준의 자동 설정 방법 및 그에 의한 스마트 학습 방식의 엑스레이 검사 방법
CN114730176A (zh) 自动视觉检查站的离线故障排除和开发
KR101910069B1 (ko) 영상 분석 기반의 결함 검사 시스템 및 방법
JP2026510261A (ja) 自動外観検査のための固定位置撮像システム
JP7489436B2 (ja) 検査装置、および、検査方法
CN113205110B (zh) 一种面板缺陷分类模型的建立方法及面板缺陷分类方法
CN119178769A (zh) 用于区域导向修复范围优化的系统和方法
KR101898843B1 (ko) AF(Anti-Finger)코팅 검사시스템 및 방법
WO2023021209A1 (en) System and method for inspecting an object
TWI662251B (zh) 三維線性檢測方法及系統
Agamirov et al. Application of neural networks for detecting defects and damage in metal structures
JP2017156517A (ja) 表示装置の欠陥モード分類システム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20864924

Country of ref document: EP

Kind code of ref document: A1

DPE1 Request for preliminary examination filed after expiration of 19th month from priority date (pct application filed from 20040101)
ENP Entry into the national phase

Ref document number: 2021546938

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2020864924

Country of ref document: EP

Effective date: 20220316