WO2022196003A1 - Quality evaluation device and inspection management system - Google Patents

Quality evaluation device and inspection management system Download PDF

Info

Publication number
WO2022196003A1
WO2022196003A1 PCT/JP2021/047106 JP2021047106W WO2022196003A1 WO 2022196003 A1 WO2022196003 A1 WO 2022196003A1 JP 2021047106 W JP2021047106 W JP 2021047106W WO 2022196003 A1 WO2022196003 A1 WO 2022196003A1
Authority
WO
WIPO (PCT)
Prior art keywords
inspection
learning
quality evaluation
quality
unit
Prior art date
Application number
PCT/JP2021/047106
Other languages
French (fr)
Japanese (ja)
Inventor
葵 望月
心平 藤井
Original Assignee
オムロン株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by オムロン株式会社 filed Critical オムロン株式会社
Publication of WO2022196003A1 publication Critical patent/WO2022196003A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/95—Investigating the presence of flaws or contamination characterised by the material or shape of the object to be examined
    • G01N21/956—Inspecting patterns on the surface of objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis

Definitions

  • the present invention relates to a quality evaluation device and an inspection management system for evaluating the quality of inspection objects such as printed circuit boards.
  • a process of inspecting the quality of the work is realized by capturing an image of the inspection target such as the work and inputting the image into a pre-learned AI model with a teacher.
  • the quality of mass-produced workpieces at production sites is constantly changing, and if the change becomes large, the supervised AI model will not be able to make correct judgments, increasing the risk of oversight or oversight. Therefore, it is necessary to relearn the supervised AI model using newly obtained learning data and use the optimal supervised AI model according to the change in quality.
  • the quality of the work includes the appearance of solder and the performance of appearance inspection of component positions and the like.
  • Patent Document 1 it is possible to generate a model that appropriately judges even data with new features that have never existed before by having an image judgment model learn new non-defective product images and defective product images. .
  • the characteristics of the workpiece can change due to various factors such as temperature and material variations at that time, and the state of the inspection equipment and equipment used in the previous process. It is difficult for the user to decide whether to relearn or not.
  • the present invention has been made in view of the above problems.
  • the purpose is to realize highly reliable inspection by promoting re-learning of the learner.
  • the present invention for solving the above problems is A non-defective product that is determined as a non-defective product in an inspection process using a trained first learning device that is generated by machine-learning an inspection image generated by capturing an image of an inspection target as learning data and using an inspection result as teacher data.
  • a non-defective product image acquisition unit that acquires an image;
  • a quality change evaluation unit that evaluates a change in quality of the inspection target;
  • a quality evaluation information generating unit that generates quality evaluation information including the result of the evaluation; It is a quality evaluation device with
  • the user can recognize the change in the quality of the object to be inspected by the quality evaluation information including the evaluation result by the quality change evaluating unit that evaluates the change in the quality of the object to be inspected. Based on the quality evaluation information, the user can detect changes in the quality of the inspection object and determine the necessity of re-learning the first learning device used in the inspection process. It is possible to promote and achieve highly reliable inspections.
  • the non-defective product image is an inspection image generated by imaging an inspection object determined to be a non-defective product by the trained first learning device, and an inspection image generated by imaging an inspection target determined to be a non-defective product by the trained first learning device. and an inspection image generated by capturing an image of an inspection object determined to be non-defective.
  • the quality change evaluation unit a quality evaluation unit that outputs a quality evaluation index for evaluating the quality based on the non-defective product image; a quality evaluation index accumulation unit that accumulates the quality evaluation index; a change determination unit that determines a change in the quality based on a change in the quality evaluation index accumulated in the quality evaluation index accumulation unit over a predetermined period; may have
  • the need for re-learning of the first learning device used in the inspection process is determined by grasping the change in the quality of the inspection target based on the change over a predetermined period of time-varying quality evaluation index. Therefore, it is possible to prompt the user to re-learn the first learning device and realize highly reliable inspection.
  • the quality evaluation index is an abnormality degree output by a trained second learner generated by machine learning by unsupervised learning using the good product image as learning data
  • the change determination unit calculates the average value and standard deviation of the degree of abnormality over a predetermined period, compares the average value and the standard deviation with the first threshold and the second threshold, respectively, and determines the change in quality You may do so.
  • the degree of abnormality output by the second learning device that has been trained based on the non-defective product image catches the change in the quality of the inspection object, and the necessity of re-learning the first learning device used for the inspection process. can be determined, the user is urged to re-learn the first learning device, and highly reliable inspection can be realized.
  • a setting unit that automatically sets at least one of the first threshold and the second threshold may be provided.
  • the first threshold and the second threshold are automatically set by the setting unit, setting by the user is not required, and appropriate setting using machine learning or the like is possible.
  • the quality change evaluation unit Clustering is performed by a trained third learner generated by machine learning by unsupervised learning using the non-defective image as learning data, and the number of outliers is compared with a third threshold to evaluate the change in quality. You may do so.
  • a third threshold value setting unit that automatically sets the third threshold value may be provided.
  • the third threshold is automatically set by the third threshold setting unit, it is not necessary for the user to set it, and it is possible to set it appropriately using machine learning or the like.
  • the quality evaluation information may include information recommending re-learning of the first learner.
  • the user can recognize the evaluation result of the quality of the object to be inspected, and the information recommending the re-learning of the first learning device simply prompts the re-learning of the first learning device. More reliable relearning of the first learner can be expected, and highly reliable inspection can be realized.
  • the quality evaluation information may include information recommending improvement of the previous process for the inspection target.
  • the user can recognize the result of the evaluation of the quality of the object to be inspected, and is recommended to improve the previous process related to the object to be inspected. .
  • the user can visually recognize the contents of the quality evaluation information via the display unit.
  • the present invention the quality evaluation device; An inspection processing unit that performs inspection processing using the learned first learning device on the inspection object, and stores the inspection image that is provided for the inspection processing and the inspection result that is the result of the inspection processing. and an inspection apparatus including a storage unit.
  • the user can grasp the change in the quality of the inspection object from the quality evaluation information of the quality evaluation device and judge the necessity of re-learning of the first learning device used in the inspection process. It is possible to configure an inspection management system that promotes re-learning and realizes highly reliable inspections.
  • a learning device for re-learning the first learning device may be included.
  • the present invention it is possible to achieve highly reliable inspection by recognizing changes in the quality of the inspection object and promoting re-learning of the learned learner by supervised machine learning used for inspection processing.
  • FIG. 1 is a functional block diagram of an inspection apparatus according to Example 1 of the present invention
  • FIG. 3 is a functional block diagram of a management device according to Example 1 of the present invention
  • FIG. It is a figure which shows the example of the quality evaluation information which concerns on Example 1 of this invention. It is a figure which shows the other example of the quality evaluation information which concerns on Example 1 of this invention. It is a figure which shows the other example of the quality evaluation information which concerns on Example 1 of this invention. It is a figure which shows the other example of the quality evaluation information which concerns on Example 1 of this invention.
  • 1 is a functional block diagram of a supervised learning device according to Example 1 of the present invention; FIG.
  • FIG. 1 is a functional block diagram of an unsupervised learning device according to Example 1 of the present invention
  • FIG. FIG. 8 is a functional block diagram of a management device according to Example 2 of the present invention
  • FIG. 4 is a functional block diagram of an unsupervised learning device according to Example 2 of the present invention
  • It is a figure explaining the quality evaluation based on Example 2 of this invention.
  • FIG. 10 is a diagram showing an example of quality evaluation information according to application example 2 of the present invention.
  • FIG. 1 shows a schematic configuration of a management system 1 including a management device 10 to which the present invention is applied.
  • FIG. 1 shows a management system 1, which comprises a solder printing device X1, a mounter X2, and a reflow furnace X3, which constitute manufacturing equipment in a printed circuit board surface mounting line, a solder printing inspection device Y1, a component inspection device Y2, and an external view. It includes an inspection device Y3 and an X-ray inspection device Y4.
  • the functions related to the inspection processing of the appearance inspection device Y3 are realized by the camera 31, the information processing device 32, and the display device 33 as shown in FIG.
  • the inspection program read from the inspection program storage unit 324 is executed by the CPU, so that the inspection unit 323 executes inspection processing on the inspection image.
  • a learner 215 (see FIG. 8) which is machine-learned by supervised learning using inspection images as learning data and inspection results as teacher data is used.
  • the quality of mass-produced printed circuit boards, which are subject to inspection is constantly changing, so if the quality changes significantly, the learner using supervised learning will not be able to correctly judge the quality, leading to the risk of over-observing or overlooking. growing. For this reason, it is desirable to relearn the learning device using newly obtained inspection images and carry out inspection using an inspection program corresponding to changing quality.
  • the inspection unit 323 corresponds to the inspection processing unit of the present invention.
  • quality evaluation is performed by the quality evaluation unit 103 using a quality evaluation program for non-defective product images, which are inspection images for inspection objects determined to be non-defective products among the inspection images acquired from the inspection device. Furthermore, the quality change determination unit 105 evaluates the quality change of the inspection target based on the accumulated quality evaluation results. Then, the quality evaluation information generation unit 106 generates quality evaluation information including the evaluation result regarding the change in quality, and displays it on the display unit 107 .
  • the quality evaluation program is a learning device that is machine-learned by unsupervised learning by a learning device 22 as shown in FIG. is a model.
  • the quality evaluation information 71 includes, for example, information as shown in FIG.
  • the control chart 711 plots the average value of the anomaly degree output over a predetermined period of time by inputting the non-defective product image, which is mass-production data, into the quality evaluation program, and plots it with a broken line. It shows the standard deviation of The threshold value (abnormality degree) displayed in the display area 712 of the quality evaluation information 71 and the threshold value (variation) displayed in the display area 714 are set to “0.5” and “0.4”, respectively. ing.
  • the value of the degree of abnormality exceeds the threshold (degree of abnormality) indicated by the solid line parallel to the horizontal axis, and the standard deviation indicated by the vertical bar also exceeds the threshold (variation). .
  • the quality evaluation information 71 displays a comprehensive evaluation that comprehensively evaluates the quality of the inspection target based on the set threshold (abnormality) and threshold (variation) and changes in the abnormality shown in the control chart 711. is doing.
  • a message is displayed that says, "There are variations in quality and there are changes. We recommend reviewing the process and re-learning the AI model.” Therefore, the user can recognize that the quality of the mass-produced printed circuit board is greatly changed, and that it is necessary to review the process and re-learn the learning device of the inspection program. Therefore, by re-learning the learning device of the inspection program in a timely manner, highly reliable inspection becomes possible.
  • the above-mentioned message corresponds to the information recommending the improvement of the previous process regarding the inspection object of the present invention.
  • FIG. 1 schematically shows a configuration example of manufacturing equipment in a surface mounting line for printed circuit boards according to the present embodiment.
  • Surface mount technology is a technology for soldering electronic components to the surface of a printed circuit board, and the surface mount line mainly consists of three processes: solder printing, component mounting, and reflow (solder welding). consists of
  • a solder printing device X1, a mounter X2, and a reflow furnace X3 are provided as manufacturing devices in this order from the upstream side.
  • the solder printing device X1 is a device that prints paste-like solder on electrode portions (called lands) on a printed circuit board by screen printing.
  • the mounter X2 is a device for picking up the electronic component to be mounted on the substrate and placing the component on the solder paste of the corresponding portion, and is also called a chip mounter.
  • the reflow furnace X3 is a heating device for heating and melting the solder paste, cooling it, and soldering the electronic component onto the substrate.
  • a plurality of mounters X2 may be provided in the surface mounting line.
  • the surface mounting line is equipped with a system that inspects the state of the board at the exit of each process from solder printing to component mounting to reflow, and automatically detects defects or potential defects.
  • this system also has a function to feed back the operation of each manufacturing apparatus based on the inspection results and analysis results (for example, change of the mounting program, etc.).
  • the solder printing inspection device Y1 is a device for inspecting the printed state of the solder paste on the board carried out from the solder printing device X1.
  • the solder print inspection apparatus Y1 measures the solder paste printed on the board two-dimensionally or three-dimensionally, and determines whether or not various inspection items are normal values (allowable range) from the measurement results. Inspection items include, for example, solder volume, area, height, misalignment, and shape.
  • An image sensor (camera) or the like can be used for two-dimensional measurement of solder paste, and a laser displacement meter, phase shift method, spatial encoding method, light section method, etc. can be used for three-dimensional measurement. can.
  • the component inspection device Y2 is a device for inspecting the arrangement state of electronic components on the board carried out from the mounter X2.
  • the component inspection device Y2 measures the component placed on the solder paste (or part of the component such as the component body or the electrode) two-dimensionally or three-dimensionally. Determines whether or not the value (allowable range). Inspection items include, for example, misalignment of parts, misalignment of angles (rotation), missing parts (parts are not placed), wrong parts (different parts are placed), wrong polarity (part side and board the polarity of the electrode on the side is different), the front/back inversion (the part is placed face down), the height of the part, etc.
  • image sensors can be used for two-dimensional measurement of electronic components
  • laser displacement meters, phase shift methods, spatial encoding methods, and light section methods can be used for three-dimensional measurement. etc. can be used.
  • the appearance inspection device Y3 is a device for inspecting the soldering quality of the board carried out from the reflow furnace X3.
  • the appearance inspection apparatus Y3 measures the solder portion after reflow two-dimensionally or three-dimensionally, and determines whether or not the various inspection items are normal values (allowable range) based on the measurement results. Inspection items include, in addition to the same items as the component inspection, the quality of the solder fillet shape.
  • the so-called color highlight method R, G, and B illumination is applied to the solder surface at different angles of incidence
  • a method of detecting the three-dimensional shape of the solder as two-dimensional hue information by photographing the reflected light of each color with a zenith camera) can be used.
  • FIG. 2 is a block diagram showing a schematic configuration of functions related to inspection processing of the visual inspection apparatus Y3.
  • the inspection processing function of the visual inspection apparatus Y3 is mainly realized by the camera 31, the information processing device 32, and the display device 33.
  • the information processing device 32 is configured by a general-purpose computer system including a CPU (processor), a main storage device (memory), an auxiliary storage device (hard disk, etc.), an input device (keyboard, mouse, controller, touch panel, etc.), and the like. be.
  • the image acquisition unit 321 acquires an image of the board to be inspected, which is captured by the camera 31 .
  • the inspection image generation unit 322 generates an inspection image by performing predetermined processing on the image acquired by the image acquisition unit 321 .
  • the inspection unit 323 measures (calculates) a predetermined index based on the inspection image, inspects the state of the inspection object using these measured values, and determines the quality.
  • the inspection program to be executed is a program including a learner trained by supervised learning.
  • the inspection program storage unit 324 stores inspection programs executed by the inspection unit 323 .
  • the result output unit 325 outputs the inspection result by the inspection unit 323 on the screen and causes the display device 33 to display it.
  • the inspection result storage unit 326 stores inspection images and inspection results in association with each other.
  • the communication interface 327 is an interface for communicating with the management device 10, the program management server 20, etc., which are connected via a network (LAN).
  • the inspection images and inspection results stored in the inspection result storage unit 326 include inspection images that were determined to be defective by the inspection program but were determined to be non-defective in the subsequent visual inspection process (over-detected inspection images). Also includes those for
  • the inspection images and inspection results stored in the inspection result storage unit 326 are transmitted to the management device 10 via the communication interface 327. Further, the inspection program is re-learned by the program management server 20, transmitted from the program management server 20 via the communication interface 327 to the inspection program storage unit 324, and stored.
  • the outline of the functions related to the inspection process has been described for the visual inspection apparatus Y3, but other inspection apparatuses have the same functional configuration.
  • the X-ray inspection device Y4 is a device for inspecting the soldering state of the board using an X-ray image. For example, in the case of package parts such as BGA (Ball Grid Array) and CSP (Chip Size Package) and multi-layer boards, the solder joints are hidden under the parts and boards. In the image) it is not possible to inspect the state of the solder.
  • the X-ray inspection apparatus Y4 is an apparatus for compensating for such weaknesses of appearance inspection. Items to be inspected by the X-ray inspection apparatus Y4 include, for example, component misalignment, solder height, solder volume, solder ball diameter, backfillet length, and solder joint quality.
  • As the X-ray image an X-ray transmission image may be used, and it is also preferable to use a CT (Computed Tomography) image.
  • the manufacturing apparatuses X1 to X3 and the inspection apparatuses Y1 to Y4 described above are connected to the management apparatus 10 via a network (LAN).
  • the management apparatus 10 is a system responsible for managing and controlling the manufacturing apparatuses X1 to X3 and the inspection apparatuses Y1 to Y4. It is composed of a general-purpose computer system equipped with an input device (keyboard, mouse, controller, touch panel, etc.), a display device, and the like. Functions of the management device 10, which will be described later, are realized by the CPU reading and executing a program stored in the auxiliary storage device.
  • the management device 10 may be composed of one computer, or may be composed of a plurality of computers. Alternatively, all or part of the functions of the management device 10 can be implemented in a computer built in any one of the manufacturing devices X1 to X3 and the inspection devices Y1 to Y4. Alternatively, part of the functions of the management device 10 may be realized by a server (such as a cloud server) on the network.
  • a server such as a cloud server
  • a program management server 20 is connected to the management device 10 via a network (LAN).
  • the program management server 20 is a server that manages inspection programs and quality evaluation programs.
  • the program management server 20 is a general-purpose computer system including a CPU (processor), a main storage device (memory), an auxiliary storage device (hard disk, etc.), an input device (keyboard, mouse, controller, touch panel, etc.), a display device, and the like.
  • Consists of The inspection program is a program that implements inspection processing in the inspection apparatuses Y1 to Y4, is stored in a predetermined storage area of the program management server 20, and is downloaded to each of the inspection apparatuses Y1 to Y4 as necessary, It is stored in a predetermined storage area of each device and executed in each device.
  • the quality evaluation program is a program that implements the quality evaluation process in the management device 10, and is stored in a predetermined storage area of the program management server 20. It is also downloaded to the management device 10 as necessary and stored in the quality evaluation program. It is stored in the unit 109 and executed in the management device.
  • the program management server 20 may manage not only the inspection program and the quality evaluation program but also various programs for controlling the manufacturing apparatuses X1 to X3 and the inspection apparatuses Y1 to Y4.
  • the management device 10 of the present embodiment has a functional unit for implementing functions for the manager of manufacturing equipment to efficiently perform maintenance and quality control of the equipment.
  • FIG. 2 shows a block diagram of each functional unit related to quality evaluation that the management device 10 has.
  • the management device 10 corresponds to the quality evaluation device of the invention
  • the management system 1 corresponds to the inspection management system of the invention.
  • the management device 10 includes an inspection image/inspection result acquisition unit 101, an inspection image/inspection result database (DB), a quality evaluation unit 103, an evaluation result storage unit 104, a quality change determination unit 105, a quality evaluation It has an information generation unit 106 , a display unit 107 , an input unit 108 and a quality evaluation program storage unit 109 .
  • the quality evaluation section 103, evaluation result accumulation section 104, and quality change determination section 105 correspond to the quality change evaluation section of the present invention.
  • the quality evaluation section 103, the evaluation result storage section 104, and the quality change determination section 105 correspond to the quality evaluation section, quality evaluation storage section, and change determination section of the present invention, respectively.
  • the quality evaluation information generation unit 106 corresponds to the quality evaluation information generation unit of the present invention.
  • the inspection image/inspection result acquisition unit 101 acquires inspection images from the inspection devices Y1 to Y4 and inspection results associated therewith from the inspection devices Y1 to Y4.
  • the inspection images and inspection results acquired by the inspection image/inspection result acquiring unit 101 are stored in the inspection image/inspection result DB.
  • the inspection image is associated with information such as the date and time of inspection and the product number of line symmetry, and this information is also acquired and stored.
  • the quality evaluation unit 103 has a function of evaluating the quality of the printed circuit board based on the output obtained by inputting the inspection image to a learned device such as an autoencoder that has been learned by unsupervised learning.
  • a learned device such as an autoencoder that has been learned by unsupervised learning.
  • an inspection image of a printed circuit board determined to be non-defective is input to the quality evaluation unit 103.
  • the inspection images determined to be non-defective products also include inspection images of printed circuit boards that were determined to be non-defective by the inspection program in the inspection apparatus but were determined to be non-defective products in the subsequent visual inspection process.
  • a learned learner used here outputs a one-dimensional numerical value, which is called an anomaly degree. Inspection images used for quality evaluation can be appropriately sampled. For example, an hourly sampling result is accumulated for a predetermined period to calculate the degree of abnormality.
  • the degree of abnormality corresponds to the quality evaluation index of the present invention.
  • the degree of anomaly calculated by the quality evaluation unit 103 is accumulated in the evaluation result accumulation unit 104.
  • the evaluation result storage section 104 corresponds to the quality evaluation index storage section of the present invention.
  • the quality change determination unit 105 analyzes changes over time in the degree of abnormality accumulated in the evaluation result accumulation unit 104, and evaluates changes in the quality of the printed circuit board. Specifically, the determination is made by comparing the change in the degree of anomaly with a set threshold value.
  • the degree of anomaly as described above, the average value of the degrees of anomaly accumulated over a predetermined period is the degree of anomaly for that period, and the standard deviation is the variation in the degree of anomaly.
  • two thresholds are set: a threshold for the value of the degree of abnormality (abnormality) and a threshold for variation in the degree of abnormality (variation).
  • the threshold value (abnormality degree) and threshold value (variation) are inputted in advance by the user via the input unit 108 .
  • the threshold (abnormality) and threshold (variation) correspond to the first and second thresholds of the present invention, respectively.
  • the quality evaluation information generation unit 106 generates quality evaluation information based on the evaluation result by the quality change determination unit 105 and causes the display unit 107 to display the information.
  • FIG. 4 shows a display example of the quality evaluation information 71. As shown in FIG. Frame lines and lead lines indicating each piece of information in the quality evaluation information 71 are indicated by dotted lines to distinguish them from the elements of the quality evaluation information 71 .
  • a control chart 711 is displayed in the center of the quality evaluation information 71.
  • the control chart for product number A is displayed.
  • the horizontal axis represents the period during which inspection images were acquired, and the vertical axis represents the degree of abnormality.
  • the degree of anomaly is shown by plotting the average value of the target period with a dashed line, and the standard deviation is shown with a solid vertical bar.
  • the threshold value of the degree of abnormality is indicated by a solid line on the vertical axis.
  • a threshold (abnormality) display area 712 is arranged below the control chart 711, and the currently set "0.5" is displayed.
  • the threshold (abnormality) display area 712 is an input field, and the threshold (abnormality) can be changed by entering a numerical value and specifying it by clicking a setting button 713 on the right side.
  • a threshold (variation) display area 714 is arranged below the threshold (abnormality) display area 712, and the currently set "0.4" is displayed.
  • the threshold (variation) display area 712 is an input field, and the threshold (variation) can be changed by inputting a numerical value and pressing the setting button 713 on the right side by clicking.
  • the threshold (abnormality) and threshold (variation) may be input and set by the user.
  • Appropriate thresholds (abnormalities) and thresholds (variations) are determined by machine learning or the like based on the history of thresholds (abnormalities) and thresholds (variations) and the history of evaluation results accumulated in the evaluation result accumulation unit 104. It may be set by the quality change determination unit 105, or a recommended value may be taught to the user. The user may input only one of the threshold (abnormality) and the threshold (variation), and the other may be automatically set or recommended values may be taught.
  • the quality change determination section 105 corresponds to the setting section of the present invention.
  • a comprehensive evaluation display area 716 is arranged above the control chart 711 .
  • a comprehensive evaluation display area 716 displays a message regarding quality. This overall evaluation is based on changes in the degree of abnormality displayed in the control chart 711 .
  • the degree of abnormality exceeds the threshold (degree of abnormality) and the variation also exceeds the threshold (variation).
  • a message is displayed as a comprehensive evaluation of quality, stating, "Quality varies and changes. It is recommended to review the process and relearn the AI model.” , the user is informed that it is time to re-learn the learning device of the inspection program used for the inspection.
  • the quality evaluation information 71 and the like are displayed on the display unit 107 by capturing the change in the quality of the inspection object, and a message is displayed to encourage the user to relearn the learning device of the inspection program, thereby prompting the user to relearn. Therefore, a highly reliable inspection can be realized.
  • a product number display area 717 is arranged on the left side of the quality evaluation information 71 .
  • the inspection program generates an AI model for each part type, and one part type includes multiple part numbers.
  • the QFP AI models used for inspection include those for product number A, product number B, and product number C. These product numbers are displayed in a color such as red that is different from white when there is an alarm for process review or re-learning of the AI model.
  • the display 717a of the product number A is displayed in a different color.
  • Such a display allows the user to clearly recognize the product number that needs to be handled.
  • each product number display 717a and the like are buttons, and by clicking and pressing these buttons, the display is switched to the control chart of the corresponding product number. Further, by clicking and pressing the display 717b of the AI model for QFP, the entire control chart can be confirmed.
  • a representative data display area 718 is arranged on the right side of the quality evaluation information 71 .
  • representative data for each period of the inspection images sampled by the quality change determination unit 105 are displayed in chronological order. This makes it possible to visually confirm the validity of the comprehensive evaluation.
  • clicking, for example, pressing a reselection button 719 arranged below the display area 718 of the representative data the representative data can be switched to another inspection image in the relevant period.
  • FIG. 5 shows a display example of other quality evaluation information 72. Description of information common to the quality evaluation information 71 is omitted.
  • the control chart 721 and comprehensive evaluation display area 716 are different from the quality evaluation information 71 .
  • the degree of abnormality does not exceed the threshold (degree of abnormality), but does exceed the threshold (variation).
  • the comprehensive evaluation display area 726 a message is displayed as a comprehensive evaluation of quality, stating that "there is variation in quality. We recommend that you review the process.” is notified to
  • FIG. 6 shows a display example of other quality evaluation information 73. Description of information common to the quality evaluation information 71 is omitted. Here, description of information common to the quality evaluation information 71 is omitted.
  • the control chart 731 and comprehensive evaluation display area 736 are different from the quality evaluation information 71 .
  • the degree of abnormality exceeds the threshold (degree of abnormality), but does not exceed the threshold (variation). For this reason, in the overall evaluation display area 736, a message "The quality has changed. It is recommended to re-learn the AI model.” The user is notified that it is time to learn.
  • FIG. 7 shows a display example of other quality evaluation information 74. Description of information common to the quality evaluation information 71 is omitted. Here, description of information common to the quality evaluation information 71 is omitted.
  • the control chart 731 , comprehensive evaluation display area 736 and product number display area 747 are different from the quality evaluation information 71 .
  • the degree of abnormality does not exceed the threshold (degree of abnormality) nor the threshold (variation). Therefore, in the overall evaluation display area 746, the message "Quality is stable" is displayed as the overall quality evaluation. Also, the display 747a of the product number A in the product number display area 747 is displayed in white, indicating that no alarm has been issued.
  • Re-learning of the inspection program learning device is recommended by the quality change determination unit 105, and when the user instructs re-learning of the inspection program learning device, the managing program management server 20 re-learns the inspection program. At this time, the program management server 20 also re-learns the learning device of the quality evaluation program. In this way, by re-learning the quality evaluation program together with the re-learning of the inspection program, it is possible to perform inspection and quality evaluation immediately responding to quality changes.
  • the learning device 21 for learning the inspection program will be described with reference to FIG.
  • This learning device 21 is configured in the program management server 20 .
  • the learning device 21 includes a learning inspection image acquisition unit 211 , a learning inspection result acquisition unit 212 , a learning data storage unit 213 , a learning processing unit 214 and a learning device 215 .
  • the object of learning data is the inspection images and inspection results in the most recent period up to the time of re-learning.
  • the learning device 215 corresponds to the first learning device of the present invention.
  • the learning device 21 corresponds to the learning device of the present invention.
  • the learning inspection image acquisition unit 211 acquires inspection images for re-learning from the inspection image/inspection result DB 102 .
  • the learning inspection result acquisition unit 212 acquires the inspection result of the inspection image from the inspection image/inspection result DB 102 .
  • the learning test images and the learning test results acquired through the learning test image acquisition unit 211 and the learning test result acquisition unit 212 are stored in the learning data storage unit, which is a predetermined area of the auxiliary storage device of the program management server 20. 213.
  • the learning processing unit 214 performs machine learning of the learning device so that when the learning inspection image stored in the learning data storage unit 213 is input, the inspection result is output.
  • the learning processing unit 214 is implemented by the CPU of the program management server 20 reading and executing a learning model generation program stored in a predetermined area of the auxiliary storage device.
  • the learning device is, for example, a program that calculates a model using a neural network using inspection images for learning as learning data and inspection results for learning as teacher data, but is not limited to this.
  • a learned learner 215 is obtained by repeating the machine learning of the learner in the learning processing unit 214 for a large number of learning data.
  • the learning device 215 thus obtained is stored in the learning result data storage unit 216, which is a predetermined area of the auxiliary storage device.
  • the learned learning device 215 stored in the learning result data storage unit 216 is transmitted to and stored in the inspection program storage unit 324 of the inspection machine Y3, for example.
  • the learning device 22 for learning the quality evaluation program will be described with reference to FIG.
  • This learning device 22 is configured in the program management server 20 .
  • the learning device 22 includes a good product image acquisition unit 221 for learning, a learning data storage unit 222 , a learning processing unit 214 and a learning device 224 .
  • the target of the learning data is the non-defective product image in the most recent period up to the time of re-learning.
  • the learner 224 corresponds to the second learner of the present invention.
  • the learning-use non-defective product image acquisition unit 221 acquires inspection images determined to be non-defective products from the inspection image/inspection result DB 102 .
  • the non-defective product image for learning acquired via the non-defective product image acquisition unit 221 for learning is stored in the learning data storage unit 222 which is a predetermined area of the auxiliary storage device of the program management server 20 .
  • the learning processing unit 223 performs machine learning of the learning device so that when the non-defective product image for learning stored in the learning data storage unit 222 is input, the degree of abnormality is output.
  • the learning processing unit 223 is implemented by the CPU of the program management server 20 reading and executing a learning model generation program stored in a predetermined area of the auxiliary storage device.
  • the learning device is an unsupervised AI model such as VAE that uses non-defective images for learning as learning data.
  • a learned learner 224 is obtained by repeating the machine learning of the learner in the learning processing unit 223 for a large number of learning data.
  • the learning device 224 thus obtained is stored in the learning result data storage unit 225, which is a predetermined area of the auxiliary storage device.
  • the trained learner 224 stored in the learning result data storage unit 225 is transmitted to the quality evaluation program storage unit 109 and stored.
  • Example 2 A management system 2 according to a second embodiment of the present invention will be described below. Configurations common to the first embodiment are denoted by the same reference numerals, and detailed description thereof is omitted.
  • FIG. 10 shows a block diagram of each functional unit related to quality evaluation included in the management device 11 according to the second embodiment.
  • the functions of the quality evaluation unit 203, the evaluation result storage unit 204, the quality change determination unit 205, and the quality evaluation information generation unit 206 of the management device 11 are different from those of the first embodiment.
  • the management device 11 corresponds to the quality evaluation device of the invention
  • the management system 2 corresponds to the inspection management system of the invention.
  • Fig. 11 shows a schematic functional configuration of the learning device 23 for performing machine learning on the quality evaluation program according to the second embodiment.
  • the learning device 23 is configured in the program management server 20 .
  • the learning device 23 includes a good product image acquisition unit 231 for learning, a learning data storage unit 232 , a learning processing unit 233 and a learning device 234 .
  • the target of the learning data is the non-defective product image in the most recent period up to the time of re-learning.
  • the learner 234 corresponds to the third learner of the present invention.
  • the learning-use non-defective product image acquisition unit 231 acquires inspection images determined to be non-defective products from the inspection image/inspection result DB 102 .
  • the non-defective product image for learning acquired via the non-defective product image acquisition unit 231 for learning is stored in the learning data storage unit 232 which is a predetermined area of the auxiliary storage device of the program management server 20 .
  • the learning processing unit 233 inputs the learning good product images stored in the learning data storage unit 232, dimensionally compresses the feature amounts of the good product images, maps them on a two-dimensional plane, and clusters the mapped good product images. Machine learning of the learner is performed so that The learning processing unit 223 is implemented by the CPU of the program management server 20 reading and executing a learning model generation program stored in a predetermined area of the auxiliary storage device.
  • the learning device is an unsupervised AI model such as principal component analysis (PCA) that uses non-defective images for learning as learning data.
  • PCA principal component analysis
  • a learned learner 234 is obtained by repeating the machine learning of the learner in the learning processing unit 233 for a large number of learning data.
  • the learning device 234 thus obtained is stored in the learning result data storage unit 235, which is a predetermined area of the auxiliary storage device.
  • the trained learner 234 stored in the learning result data storage unit 235 is transmitted to the quality evaluation program storage unit 109 and stored.
  • FIG. 12A shows an output example of the learned learner 234 .
  • a point L1, a point L2, etc. representing the learned non-defective product image are mapped on the two-dimensional surface S, and clustered by a circle C surrounding these points L1, etc.
  • the quality evaluation unit 203 of the second embodiment maps non-defective product images of mass production data acquired from inspection images/inspection results using the quality evaluation program generated in this way.
  • FIG. 12B shows a two-dimensional map mapped on the two-dimensional surface S by inputting a non-defective product image, which is mass production data, to the learned learner 234 .
  • the non-defective product image, which is learning data is displayed with black points
  • the non-defective product image, which is mass production data is displayed with gray points.
  • points P1, P2, etc. indicating a non-defective product image, which are mass production data are mapped on the two-dimensional surface S. , are included in the cluster indicated by the circle C1.
  • the quality evaluation unit 203 of the second embodiment samples and maps non-defective product images, which are mass production data obtained from the inspection image/inspection result DB, and outputs a two-dimensional map.
  • the two-dimensional map is accumulated in the evaluation result accumulation unit 204.
  • the quality change determination unit 205 of the second embodiment counts the number of points indicating non-defective images that are outliers outside the cluster in the two-dimensional map generated by the quality evaluation unit 203 .
  • FIG. 11B when non-defective product images, which are mass production data, are sequentially mapped, data mapped at positions outside cluster C appear as outliers when the quality changes.
  • FIG. 12C a point O1 indicating a non-defective product image that is mass production data is mapped at a position outside a cluster C that includes a non-defective product image L1 that is mass production data, such as a point L1 that indicates a non-defective product image for learning. .
  • the quality change determination unit 205 counts the number of such outliers, compares the number of outliers with a threshold, and determines whether or not the threshold is exceeded.
  • the quality evaluation information generation unit 206 generates quality evaluation information based on the evaluation result by the quality change determination unit 205, and causes the display unit 107 to display the information.
  • FIG. 13 shows a display example of the quality evaluation information 75 according to the second embodiment. Frame lines and lead lines that indicate each piece of information in the quality evaluation information 75 are indicated by dotted lines to distinguish them from the elements of the quality evaluation information 75 .
  • a 2D map 751 is displayed on the right side of the center of the quality evaluation information 75 .
  • the 2D map 751 also displays the product number subject to quality evaluation (here, “product number A”) and the date (evaluation date) indicating when the evaluation data was obtained.
  • the evaluation date of the displayed quality evaluation information 75 can be appropriately specified by the user.
  • the 2D map is obtained by inputting a non-defective product image, which is mass production data, into the quality evaluation program and mapping it on a 2D map generated from the non-defective product image for learning.
  • a cluster C of the 2D map 751 includes a point L1 indicating a good product image for learning and a point P1 indicating a good product image that is mass production data.
  • a threshold (number of alarms) display area 752 is arranged below the 2D map 751 of the quality evaluation information 75, and the currently set "4" is displayed.
  • the display area 752 for the threshold (the number of alarms) is an input field, and the threshold (the number of alarms) can be changed by entering a numerical value and specifying it by clicking a setting button 753 on the right side.
  • This threshold (number of alarms) instructs the user to review the process or re-learn the inspection program learner when the number of outliers in the mass production data arranged on the 2D map exceeds the threshold (number of alarms). It is for issuing a recommendation alarm.
  • the threshold (number of alarms) corresponds to the third threshold of the present invention.
  • the threshold (the number of alarms) may be input and set by the user. Based on the history of the number of alarms) and the history of the evaluation results accumulated in the evaluation result accumulation unit 204, an appropriate threshold value (the number of alarms) is set by the quality change determination unit 205 by machine learning or the like, and a recommended value is given to the user. may be taught.
  • the quality change determination section 205 corresponds to the threshold setting section of the present invention.
  • a comprehensive evaluation display area 754 is arranged above the quality evaluation information 75 .
  • a message regarding quality is displayed in the comprehensive evaluation display area 754 .
  • This overall evaluation is based on the number of outliers in the 2D map 751 .
  • the threshold (number of alarms) display area 752 the current threshold (number of alarms) is four.
  • the quality change determination unit 105 evaluates that the quality has changed, and displays a message in the overall evaluation display area 754 stating, "The quality has changed. It is recommended to review the process or re-learn the AI model.” is displayed to inform the user that the process should be reviewed or the learner of the inspection program used for the inspection should be retrained.
  • the quality evaluation information 75 is displayed on the display unit 107 in response to changes in the quality of the inspection object, and a message is displayed to encourage the user to re-learn the learning device of the inspection program. , a highly reliable inspection can be realized.
  • a product number display area 755 is arranged on the left side of the quality evaluation information 75 .
  • the inspection program generates an AI model for each part type, and one part type includes multiple part numbers.
  • the QFP AI models used for inspection include those for product number A, product number B, and product number C. These product numbers are displayed in a color such as red that is different from white when there is an alarm for process review or re-learning of the AI model.
  • the display 755a of the product number A is displayed in a different color.
  • Such a display allows the user to clearly recognize the product number that needs to be handled.
  • each product number display 755a and the like are buttons, and by clicking and pressing these buttons, the display is switched to the 2D map of the corresponding product number. Also, by clicking and pressing the display 755b of the AI model for QFP, the entire 2D map can be confirmed.
  • ⁇ Appendix 1> The inspection image determined to be non-defective in the inspection process using the trained first learner (215) generated by performing machine learning using the inspection image generated by imaging the inspection object as learning data and using the inspection result as teacher data.
  • a quality evaluation information generation unit (106) for generating quality evaluation information including the result of the evaluation
  • a quality evaluation device (10) comprising:
  • management device 103 quality evaluation unit 104: evaluation result accumulation unit 105: quality change evaluation unit 106: quality evaluation information generation unit 215: learning device

Landscapes

  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Immunology (AREA)
  • Pathology (AREA)
  • Analytical Chemistry (AREA)
  • Biochemistry (AREA)
  • General Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Chemical & Material Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • General Factory Administration (AREA)
  • Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
  • Image Analysis (AREA)

Abstract

This quality evaluation device comprising: a non-defective image acquisition unit which acquires a non-defective image that is an inspection image determined as non-defective in an inspection process using a first trained learning apparatus generated by machine-learning with, as learning data, an inspection image generated by capturing an image of a subject to be inspected and, as teaching data, inspection results; a quality change evaluation unit which evaluates a change in quality of the subject to be inspected; and a quality evaluation information generation unit which generates quality evaluation information including the results of the evaluation.

Description

品質評価装置及び検査管理システムQuality evaluation equipment and inspection management system
　本発明は、プリント基板等の検査対象の品質を評価する品質評価装置及び検査管理システムに関する。 The present invention relates to a quality evaluation device and an inspection management system for evaluating the quality of inspection objects such as printed circuit boards.
　ＦＡ（Ｆａｃｔｏｒｙ　Ａｕｔｏｍａｔｉｏｎ）分野ではワーク等の検査対象を撮像し、事前に学習させた教師ありＡＩモデルにその画像を入力することで、当該ワークについての良否を検査するような工程が実現される。生産現場で量産されるワークの品質は絶えず変化しており、その変化が大きくなると教師ありＡＩモデルが正しく判定することができなくなるため、見過ぎや見逃しのリスクが大きくなる。ゆえに新しく得られた学習データを用いて教師ありＡＩモデルを再学習させ、品質の変化に応じた最適な教師ありＡＩモデルを使用する必要がある。ワークの品質としては、例えば、ワークがプリント基板である場合には、はんだの外観や、部品位置等の外観検査における出来栄え等が挙げられる。 In the field of FA (Factory Automation), a process of inspecting the quality of the work is realized by capturing an image of the inspection target such as the work and inputting the image into a pre-learned AI model with a teacher. The quality of mass-produced workpieces at production sites is constantly changing, and if the change becomes large, the supervised AI model will not be able to make correct judgments, increasing the risk of oversight or oversight. Therefore, it is necessary to relearn the supervised AI model using newly obtained learning data and use the optimal supervised AI model according to the change in quality. For example, when the work is a printed circuit board, the quality of the work includes the appearance of solder and the performance of appearance inspection of component positions and the like.
　例えば、特許文献１では画像判定用のモデルに新たな良品画像、不良品画像を学習させることでこれまでになかった新たな特徴を持つデータに対しても適切に判定するモデルを生成可能としている。 For example, in Patent Document 1, it is possible to generate a model that appropriately judges even data with new features that have never existed before by having an image judgment model learn new non-defective product images and defective product images. .
　しかしながら、ワークの特徴はその時の気温や材料のバラつき、さらに検査装置及び前工程で使用される装置の状態など様々な要因によって変化しうるため、どのタイミングでワークの特徴が変わり、教師ありＡＩモデルを再学習すべきかをユーザが判断することは困難である。 However, the characteristics of the workpiece can change due to various factors such as temperature and material variations at that time, and the state of the inspection equipment and equipment used in the previous process. It is difficult for the user to decide whether to relearn or not.
特開２０２０－１０７１０４号公報JP 2020-107104 A
　本発明は、上記のような問題に鑑みてなされたものであり、本発明が解決しようとする課題は、検査対象の品質の変化をとらえて、検査処理に用いられる教師あり機械学習による学習済み学習器の再学習を促すことで、信頼性の高い検査を実現することを目的とする。 The present invention has been made in view of the above problems. The purpose is to realize highly reliable inspection by promoting re-learning of the learner.
　上記の課題を解決するための本発明は、
　検査対象を撮像して生成した検査画像を学習データとし、検査結果を教師データとして機械学習させて生成した学習済み第１学習器を用いた検査処理において良品と判定された該検査画像である良品画像を取得する良品画像取得部と、
　前記検査対象の品質の変化を評価する品質変化評価部と、
　前記評価の結果を含む品質評価情報を生成する品質評価情報生成部と、
を備えた品質評価装置である。 The present invention for solving the above problems is
A non-defective product that is determined as a non-defective product in an inspection process using a trained first learning device that is generated by machine-learning an inspection image generated by capturing an image of an inspection target as learning data and using an inspection result as teacher data. a non-defective product image acquisition unit that acquires an image;
a quality change evaluation unit that evaluates a change in quality of the inspection target;
a quality evaluation information generating unit that generates quality evaluation information including the result of the evaluation;
It is a quality evaluation device with
　これによれば、ユーザは、検査対象の品質の変化を評価する品質変化評価部による評価の結果を含む品質評価情報によって、検査対象の品質の変化をユーザに認識させることができる。ユーザは、品質評価情報により、検査対象の品質の変化をとらえ、検査処理に用いられる第１学習器の再学習の必要性を判断することができるので、ユーザに第１学習器の再学習を促し、信頼性の高い検査を実現することができる。
　ここで、良品画像は、学習済み第１学習器により良品と判定された検査対象を撮像して生成した検査画像と、学習済み第１学習器により不良と判定されたものの、後の目視工程により良品と判定された検査対象を撮像して生成した検査画像とを含む。 According to this, the user can recognize the change in the quality of the object to be inspected by the quality evaluation information including the evaluation result by the quality change evaluating unit that evaluates the change in the quality of the object to be inspected. Based on the quality evaluation information, the user can detect changes in the quality of the inspection object and determine the necessity of re-learning the first learning device used in the inspection process. It is possible to promote and achieve highly reliable inspections.
Here, the non-defective product image is an inspection image generated by imaging an inspection object determined to be a non-defective product by the trained first learning device, and an inspection image generated by imaging an inspection target determined to be a non-defective product by the trained first learning device. and an inspection image generated by capturing an image of an inspection object determined to be non-defective.
　また、本発明において、
　前記品質変化評価部は、
　前記良品画像に基づいて、前記品質を評価する品質評価指標を出力する品質評価部と、
　前記品質評価指標を蓄積する品質評価指標蓄積部と、
　前記品質評価指標蓄積部に蓄積された前記品質評価指標の所定期間にわたる変化に基づいて、前記品質の変化を判定する変化判定部と、
を有するようにしてもよい。 Moreover, in the present invention,
The quality change evaluation unit
a quality evaluation unit that outputs a quality evaluation index for evaluating the quality based on the non-defective product image;
a quality evaluation index accumulation unit that accumulates the quality evaluation index;
a change determination unit that determines a change in the quality based on a change in the quality evaluation index accumulated in the quality evaluation index accumulation unit over a predetermined period;
may have
　これによれば、経時的に変化する品質評価指標の所定期間にわたる変化に基づいて、検査対象の品質の変化をとらえて、検査処理に用いられる第１学習器の再学習の必要性を判断することができるので、ユーザに第１学習器の再学習を促し、信頼性の高い検査を実現することができる。 According to this, the need for re-learning of the first learning device used in the inspection process is determined by grasping the change in the quality of the inspection target based on the change over a predetermined period of time-varying quality evaluation index. Therefore, it is possible to prompt the user to re-learn the first learning device and realize highly reliable inspection.
　また、本発明において、
　前記品質評価指標は、前記良品画像を学習データとして教師なし学習によって機械学習させて生成した学習済み第２学習器により出力される異常度であり、
　前記変化判定部は、所定期間にわたる前記異常度の平均値及び標準偏差を算出し、該平均値及び該標準偏差を、それぞれ第１閾値及び第２閾値と比較して前記品質の変化を判定するようにしてもよい。 Moreover, in the present invention,
The quality evaluation index is an abnormality degree output by a trained second learner generated by machine learning by unsupervised learning using the good product image as learning data,
The change determination unit calculates the average value and standard deviation of the degree of abnormality over a predetermined period, compares the average value and the standard deviation with the first threshold and the second threshold, respectively, and determines the change in quality You may do so.
　これによれば、良品画像に基づいて学習済み第２学習器により出力される異常度によって、検検査対象の品質の変化をとらえて、検査処理に用いられる第１学習器の再学習の必要性を判断することができるので、ユーザに第１学習器の再学習を促し、信頼性の高い検査を実現することができる。 According to this, the degree of abnormality output by the second learning device that has been trained based on the non-defective product image catches the change in the quality of the inspection object, and the necessity of re-learning the first learning device used for the inspection process. can be determined, the user is urged to re-learn the first learning device, and highly reliable inspection can be realized.
　前記第１閾値及び前記第２閾値の少なくともいずれかを自動で設定する設定部を備えるようにしてもよい。 A setting unit that automatically sets at least one of the first threshold and the second threshold may be provided.
　これによれば、第１閾値及び第２閾値が設定部によって自動で設定されるので、ユーザにより設定が不要になるとともに、機械学習等を用いた適切な設定が可能となる。 According to this, since the first threshold and the second threshold are automatically set by the setting unit, setting by the user is not required, and appropriate setting using machine learning or the like is possible.
　また、本発明において、
　前記品質変化評価部は、
　前記良品画像を学習データとして教師なし学習によって機械学習させて生成した学習済み第３学習器により、クラスタリングを行い、外れ値の個数と、第３閾値とを比較して前記品質の変化を評価するようにしてもよい。 Moreover, in the present invention,
The quality change evaluation unit
Clustering is performed by a trained third learner generated by machine learning by unsupervised learning using the non-defective image as learning data, and the number of outliers is compared with a third threshold to evaluate the change in quality. You may do so.
　これによれば、良品画像に基づいて学習済み第３学習器により行われるクラスタリングを用いて、検査対象の品質の変化をとらえて、検査処理に用いられる第１学習器の再学習の必要性を判断することができるので、ユーザに第１学習器の再学習を促し、信頼性の高い検査を実現することができる。 According to this, using the clustering performed by the third learner that has been trained based on the non-defective product image, changes in the quality of the inspection object are caught, and the necessity of re-learning the first learner used for the inspection process is determined. Since the determination can be made, it is possible to prompt the user to relearn the first learner and realize a highly reliable inspection.
　また、本発明において、
　前記第３閾値を自動で設定する第３閾値設定部を備えるようにしてもよい。 Moreover, in the present invention,
A third threshold value setting unit that automatically sets the third threshold value may be provided.
　第３閾値が第３閾値設定部によって自動で設定されるので、ユーザにより設定が不要になるとともに、機械学習等を用いた適切な設定が可能となる。 Since the third threshold is automatically set by the third threshold setting unit, it is not necessary for the user to set it, and it is possible to set it appropriately using machine learning or the like.
　また、本発明において、
　前記品質評価情報は、前記第１学習器の再学習を勧める情報を含むようにしてもよい。 Moreover, in the present invention,
The quality evaluation information may include information recommending re-learning of the first learner.
　これによれば、ユーザは、検査対象の品質の評価の結果を認識できるとともに、第１学習器の再学習を勧める情報によって、端的に第１学習器の再学習を促されることになるので、より確実な第１学習器の再学習が期待でき、信頼性の高い検査を実現することができる。 According to this, the user can recognize the evaluation result of the quality of the object to be inspected, and the information recommending the re-learning of the first learning device simply prompts the re-learning of the first learning device. More reliable relearning of the first learner can be expected, and highly reliable inspection can be realized.
　また、本発明において、
　前記品質評価情報は、前記検査対象に関する前工程の改善を勧める情報を含むようにしてもよい。 Moreover, in the present invention,
The quality evaluation information may include information recommending improvement of the previous process for the inspection target.
　これによれば、ユーザは、検査対象の品質の評価の結果を認識できるとともに、検査対象に関する前工程の改善を勧められるので、前工程の改善により検査対象の品質の改善を実現することができる。 According to this, the user can recognize the result of the evaluation of the quality of the object to be inspected, and is recommended to improve the previous process related to the object to be inspected. .
　また、本発明において、
　前記品質評価情報を表示する表示部を備えるようにしてもよい。 Moreover, in the present invention,
You may make it provide the display part which displays the said quality evaluation information.
　これによれば、ユーザは、表示部を介して、品質評価情報の内容を視覚的に認識できる。 According to this, the user can visually recognize the contents of the quality evaluation information via the display unit.
　また、本発明は、
　前記品質評価装置と、
　前記検査対象に対して前記学習済み第１学習器を用いた検査処理を実施する検査処理部と、該検査処理に供される前記検査画像と該検査処理による結果である前記検査結果を記憶する記憶部とを、備えた検査装置と
を含む検査管理システムである。 In addition, the present invention
the quality evaluation device;
An inspection processing unit that performs inspection processing using the learned first learning device on the inspection object, and stores the inspection image that is provided for the inspection processing and the inspection result that is the result of the inspection processing. and an inspection apparatus including a storage unit.
　これによれば、
ユーザは、品質評価装置の品質評価情報により、検査対象の品質の変化をとらえ、検査処理に用いられる第１学習器の再学習の必要性を判断することができるので、ユーザに第１学習器の再学習を促し、信頼性の高い検査を実現できる検査管理システムを構成することができる。 According to this,
The user can grasp the change in the quality of the inspection object from the quality evaluation information of the quality evaluation device and judge the necessity of re-learning of the first learning device used in the inspection process. It is possible to configure an inspection management system that promotes re-learning and realizes highly reliable inspections.
　また、本発明において、
　前記第１学習器を再学習させる学習装置を含むようにしてもよい。 Moreover, in the present invention,
A learning device for re-learning the first learning device may be included.
　これによれば、品質評価情報によって促された適宜のタイミングで第１学習器を再学習させることにより、信頼性の高い検査を実現できる検査管理システムを構成することができる。 According to this, it is possible to configure an inspection management system that can realize highly reliable inspection by relearning the first learning device at an appropriate timing prompted by the quality evaluation information.
　本発明によれば、検査対象の品質の変化をとらえて、検査処理に用いられる教師あり機械学習による学習済み学習器の再学習を促すことで、信頼性の高い検査を実現することができる。 According to the present invention, it is possible to achieve highly reliable inspection by recognizing changes in the quality of the inspection object and promoting re-learning of the learned learner by supervised machine learning used for inspection processing.
本発明の実施例１に係る製造設備の概略構成を示す図である。It is a figure which shows schematic structure of the manufacturing equipment which concerns on Example 1 of this invention. 本発明の実施例１に係る検査装置の機能ブロック図である。1 is a functional block diagram of an inspection apparatus according to Example 1 of the present invention; FIG. 本発明の実施例１に係る管理装置の機能ブロック図である。3 is a functional block diagram of a management device according to Example 1 of the present invention; FIG. 本発明の実施例１に係る品質評価情報の例を示す図である。It is a figure which shows the example of the quality evaluation information which concerns on Example 1 of this invention. 本発明の実施例１に係る品質評価情報の他の例を示す図である。It is a figure which shows the other example of the quality evaluation information which concerns on Example 1 of this invention. 本発明の実施例１に係る品質評価情報の他の例を示す図である。It is a figure which shows the other example of the quality evaluation information which concerns on Example 1 of this invention. 本発明の実施例１に係る品質評価情報の他の例を示す図である。It is a figure which shows the other example of the quality evaluation information which concerns on Example 1 of this invention. 本発明の実施例１に係る教師あり学習装置の機能ブロック図である。1 is a functional block diagram of a supervised learning device according to Example 1 of the present invention; FIG. 本発明の実施例１に係る教師なし学習装置の機能ブロック図である。1 is a functional block diagram of an unsupervised learning device according to Example 1 of the present invention; FIG. 本発明の実施例２に係る管理装置の機能ブロック図である。FIG. 8 is a functional block diagram of a management device according to Example 2 of the present invention; 本発明の実施例２に係る教師なし学習装置の機能ブロック図である。FIG. 4 is a functional block diagram of an unsupervised learning device according to Example 2 of the present invention; 本発明の実施例２に係る品質評価を説明する図である。It is a figure explaining the quality evaluation based on Example 2 of this invention. 本発明の適用例２に係る品質評価情報の例を示す図である。FIG. 10 is a diagram showing an example of quality evaluation information according to application example 2 of the present invention;
〔適用例〕
　以下、本発明の適用例について、図面を参照しつつ説明する。 [Example of application]
Hereinafter, application examples of the present invention will be described with reference to the drawings.
　図１は、本発明が適用される管理装置１０を含む管理システム１の概略構成を示す。図１は、管理システム１は、プリント基板の表面実装ラインにおける製造設備を構成するはんだ印刷装置Ｘ１、マウンタＸ２、及びリフロー炉Ｘ３の製造装置と、はんだ印刷検査装置Ｙ１、部品検査装置Ｙ２、外観検査装置Ｙ３、Ｘ線検査装置Ｙ４の検査装置を含む。 FIG. 1 shows a schematic configuration of a management system 1 including a management device 10 to which the present invention is applied. FIG. 1 shows a management system 1, which comprises a solder printing device X1, a mounter X2, and a reflow furnace X3, which constitute manufacturing equipment in a printed circuit board surface mounting line, a solder printing inspection device Y1, a component inspection device Y2, and an external view. It includes an inspection device Y3 and an X-ray inspection device Y4.
　外観検査装置Ｙ３の検査処理に関する機能は、図２に示すようにカメラ３１、情報処理装置３２、表示装置３３によって実現される。ここでは、検査プログラム記憶部３２４から読み出された検査プログラムがＣＰＵによって実行されることによって、検査部３２３において検査画像に対する検査処理が実行される。検査プログラムとして、検査画像を学習データ、検査結果を教師データとして教師あり学習により機械学習された学習器２１５（図８参照）を用いる。検査対象である、量産されるプリント基板の品質は絶えず変化しているため、品質の変化が大きくなると教師あり学習による学習器によって正しく良否を判断することができなくなり、見過ぎや見逃しのリスクが大きくなる。このため、新しく得られた検査画像を用いて学習器を再学習させ、変化する品質に対応した検査プログラムにより検査を実施することが望ましい。検査部３２３は、本発明の検査処理部に対応する。 The functions related to the inspection processing of the appearance inspection device Y3 are realized by the camera 31, the information processing device 32, and the display device 33 as shown in FIG. Here, the inspection program read from the inspection program storage unit 324 is executed by the CPU, so that the inspection unit 323 executes inspection processing on the inspection image. As an inspection program, a learner 215 (see FIG. 8) which is machine-learned by supervised learning using inspection images as learning data and inspection results as teacher data is used. The quality of mass-produced printed circuit boards, which are subject to inspection, is constantly changing, so if the quality changes significantly, the learner using supervised learning will not be able to correctly judge the quality, leading to the risk of over-observing or overlooking. growing. For this reason, it is desirable to relearn the learning device using newly obtained inspection images and carry out inspection using an inspection program corresponding to changing quality. The inspection unit 323 corresponds to the inspection processing unit of the present invention.
　図３に示す管理装置１０では、検査装置から取得した検査画像のうち良品と判定された検査対象に対する検査画像である良品画像について、品質評価部１０３において品質評価プログラムによって品質評価を行う。さらに、品質変化判定部１０５では、蓄積された品質評価結果に基づき、検査対象の品質の変化を評価する。そして、品質評価情報生成部１０６では、品質の変化に関する評価結果を含む品質評価情報を生成し、表示部１０７に表示させる。 In the management device 10 shown in FIG. 3, quality evaluation is performed by the quality evaluation unit 103 using a quality evaluation program for non-defective product images, which are inspection images for inspection objects determined to be non-defective products among the inspection images acquired from the inspection device. Furthermore, the quality change determination unit 105 evaluates the quality change of the inspection target based on the accumulated quality evaluation results. Then, the quality evaluation information generation unit 106 generates quality evaluation information including the evaluation result regarding the change in quality, and displays it on the display unit 107 .
　具体的には、品質評価プログラムは、図９に示すような学習装置２２により、教師なし学習により機械学習させた学習器であり、学習用の良品画像を学習データとするＶＡＥ等の教師なしＡＩモデルである。 Specifically, the quality evaluation program is a learning device that is machine-learned by unsupervised learning by a learning device 22 as shown in FIG. is a model.
　品質評価情報７１は、例えば、図４に示すような情報を含む。管理図７１１は、量産データである良品画像を品質評価プログラムに入力することにより出力された異常度について、所定期間の平均値をプロットして破線で示し、実線の縦棒により所定期間の異常度の標準偏差を示すものである。品質評価情報７１の表示領域７１２に表示された閾値（異常度）と表示領域７１４に表示された閾値（バラつき）には、それぞれの値として「０．５」と「０．４」が設定されている。管理図７１１において、異常度の値が、横軸に平行な実線で示された閾値（異常度）が超えており、また、縦棒で示された標準偏差も閾値（バラつき）を超えている。 The quality evaluation information 71 includes, for example, information as shown in FIG. The control chart 711 plots the average value of the anomaly degree output over a predetermined period of time by inputting the non-defective product image, which is mass-production data, into the quality evaluation program, and plots it with a broken line. It shows the standard deviation of The threshold value (abnormality degree) displayed in the display area 712 of the quality evaluation information 71 and the threshold value (variation) displayed in the display area 714 are set to “0.5” and “0.4”, respectively. ing. In the control chart 711, the value of the degree of abnormality exceeds the threshold (degree of abnormality) indicated by the solid line parallel to the horizontal axis, and the standard deviation indicated by the vertical bar also exceeds the threshold (variation). .
　品質評価情報７１では、設定された閾値（異常度）及び閾値（バラつき）と、管理図７１１に示された異常度の変化に基づいて、検査対象の品質を総合的に評価した総合評価を表示している。ここでは、「品質にばらつきがあり、変化もあります。工程の見直しとＡＩモデルの再学習をお勧めします。」とのメッセージが表示される。ユーザは、これにより量産されるプリント基板の品質の変化が大きく、工程の見直しと検査プログラムの学習器の再学習が必要であることを認識することができる。したがって、適時に検査プログラムの学習器を再学習させることにより、信頼性の高い検査が可能となる。ここでは、上述のメッセージが、本発明の検査対象に関する前工程の改善を勧める情報に対応する。 The quality evaluation information 71 displays a comprehensive evaluation that comprehensively evaluates the quality of the inspection target based on the set threshold (abnormality) and threshold (variation) and changes in the abnormality shown in the control chart 711. is doing. Here, a message is displayed that says, "There are variations in quality and there are changes. We recommend reviewing the process and re-learning the AI model." Therefore, the user can recognize that the quality of the mass-produced printed circuit board is greatly changed, and that it is necessary to review the process and re-learn the learning device of the inspection program. Therefore, by re-learning the learning device of the inspection program in a timely manner, highly reliable inspection becomes possible. Here, the above-mentioned message corresponds to the information recommending the improvement of the previous process regarding the inspection object of the present invention.
〔実施例１〕
　以下では、本発明の実施例１に係る管理システム１ついて、図面を用いて、より詳細に説明する。
　（システム構成）
　図１は、本実施例に係るプリント基板の表面実装ラインにおける製造設備の構成例を模式的に示している。表面実装（Ｓｕｒｆａｃｅ　Ｍｏｕｎｔ　Ｔｅｃｈｎｏｌｏｇｙ：ＳＭＴ）とはプリント基板の表面に電子部品をはんだ付けする技術であり、表面実装ラインは、主として、はんだ印刷～部品のマウント～リフロー（はんだの溶着）の三つの工程から構成される。 [Example 1]
Below, the management system 1 according to the first embodiment of the present invention will be described in more detail with reference to the drawings.
(System configuration)
FIG. 1 schematically shows a configuration example of manufacturing equipment in a surface mounting line for printed circuit boards according to the present embodiment. Surface mount technology (SMT) is a technology for soldering electronic components to the surface of a printed circuit board, and the surface mount line mainly consists of three processes: solder printing, component mounting, and reflow (solder welding). consists of
　図１に示すように、表面実装ラインでは、製造装置として、上流側から順に、はんだ印刷装置Ｘ１、マウンタＸ２、リフロー炉Ｘ３が設けられる。はんだ印刷装置Ｘ１は、スクリーン印刷によってプリント基板上の電極部（ランドと呼ばれる）にペースト状のはんだを印刷する装置である。マウンタＸ２は、基板に実装すべき電子部品をピックアップし、該当箇所のはんだペーストの上に部品を載置するための装置であり、チップマウンタとも呼ばれる。リフロー炉Ｘ３は、はんだペーストを加熱溶融した後、冷却を行い、電子部品を基板上にはんだ接合するための加熱装置である。基板に実装する電子部品の数や種類が多い場合には、表面実装ラインに複数台のマウンタＸ２が設けられることもある。 As shown in FIG. 1, in the surface mounting line, a solder printing device X1, a mounter X2, and a reflow furnace X3 are provided as manufacturing devices in this order from the upstream side. The solder printing device X1 is a device that prints paste-like solder on electrode portions (called lands) on a printed circuit board by screen printing. The mounter X2 is a device for picking up the electronic component to be mounted on the substrate and placing the component on the solder paste of the corresponding portion, and is also called a chip mounter. The reflow furnace X3 is a heating device for heating and melting the solder paste, cooling it, and soldering the electronic component onto the substrate. When there are many types and numbers of electronic components to be mounted on the board, a plurality of mounters X2 may be provided in the surface mounting line.
　また、表面実装ラインには、はんだ印刷～部品のマウント～リフローの各工程の出口で基板の状態を検査し、不良あるいは不良のおそれを自動で検出するシステムが設置されている。当該システムは、良品と不良品の自動仕分けの他、検査結果やその分析結果に基づき各製造装置の動作にフィードバックする機能（例えば、実装プログラムの変更など）も有している。 In addition, the surface mounting line is equipped with a system that inspects the state of the board at the exit of each process from solder printing to component mounting to reflow, and automatically detects defects or potential defects. In addition to automatic sorting of non-defective products and defective products, this system also has a function to feed back the operation of each manufacturing apparatus based on the inspection results and analysis results (for example, change of the mounting program, etc.).
　はんだ印刷検査装置Ｙ１は、はんだ印刷装置Ｘ１から搬出された基板に対し、はんだペーストの印刷状態を検査するための装置である。はんだ印刷検査装置Ｙ１では、基板上に印刷されたはんだペーストを２次元ないし３次元的に計測し、その計測結果から各種の検査項目について正常値（許容範囲）か否かの判定を行う。検査項目としては、例えば、はんだの体積・面積・高さ・位置ずれ・形状などがある。はんだペーストの２次元計測には、イメージセンサ（カメラ）などを用いることができ、３次元計測には、レーザ変位計や、位相シフト法、空間コード化法、光切断法などを利用することができる。 The solder printing inspection device Y1 is a device for inspecting the printed state of the solder paste on the board carried out from the solder printing device X1. The solder print inspection apparatus Y1 measures the solder paste printed on the board two-dimensionally or three-dimensionally, and determines whether or not various inspection items are normal values (allowable range) from the measurement results. Inspection items include, for example, solder volume, area, height, misalignment, and shape. An image sensor (camera) or the like can be used for two-dimensional measurement of solder paste, and a laser displacement meter, phase shift method, spatial encoding method, light section method, etc. can be used for three-dimensional measurement. can.
　部品検査装置Ｙ２は、マウンタＸ２から搬出された基板に対し、電子部品の配置状態を検査するための装置である。部品検査装置Ｙ２では、はんだペーストの上に載置された部品（部品本体、電極など部品の一部でもよい）を２次元ないし３次元的に計測し、その計測結果から各種の検査項目について正常値（許容範囲）か否かの判定を行う。検査項目としては、例えば、部品の位置ずれ、角度（回転）ずれ、欠品（部品が配置されていないこと）、部品違い（異なる部品が配置されていること）、極性違い（部品側と基板側の電極の極性が異なること）、表裏反転（部品が裏向きに配置されていること）、部品高さなどがある。はんだ印刷検査と同様、電子部品の２次元計測には、イメージセンサ（カメラ）などを用いることができ、３次元計測には、レーザ変位計や、位相シフト法、空間コード化法、光切断法などを利用することができる。 The component inspection device Y2 is a device for inspecting the arrangement state of electronic components on the board carried out from the mounter X2. The component inspection device Y2 measures the component placed on the solder paste (or part of the component such as the component body or the electrode) two-dimensionally or three-dimensionally. Determines whether or not the value (allowable range). Inspection items include, for example, misalignment of parts, misalignment of angles (rotation), missing parts (parts are not placed), wrong parts (different parts are placed), wrong polarity (part side and board the polarity of the electrode on the side is different), the front/back inversion (the part is placed face down), the height of the part, etc. As with solder printing inspection, image sensors (cameras) can be used for two-dimensional measurement of electronic components, and laser displacement meters, phase shift methods, spatial encoding methods, and light section methods can be used for three-dimensional measurement. etc. can be used.
　外観検査装置Ｙ３は、リフロー炉Ｘ３から搬出された基板に対し、はんだ付けの品質を検査するための装置である。外観検査装置Ｙ３では、リフロー後のはんだ部分を２次元ないし３次元的に計測し、その計測結果から各種の検査項目について正常値（許容範囲）か否かの判定を行う。検査項目としては、部品検査と同じ項目に加え、はんだフィレット形状の良否なども含まれる。はんだの形状計測には、上述したレーザ変位計、位相シフト法、空間コード化法、光切断法などの他、いわゆるカラーハイライト方式（Ｒ、Ｇ、Ｂの照明を異なる入射角ではんだ面に当て、各色の反射光を天頂カメラで撮影することで、はんだの３次元形状を２次元の色相情報として検出する方法）を用いることができる。 The appearance inspection device Y3 is a device for inspecting the soldering quality of the board carried out from the reflow furnace X3. The appearance inspection apparatus Y3 measures the solder portion after reflow two-dimensionally or three-dimensionally, and determines whether or not the various inspection items are normal values (allowable range) based on the measurement results. Inspection items include, in addition to the same items as the component inspection, the quality of the solder fillet shape. In addition to the above-mentioned laser displacement meter, phase shift method, spatial encoding method, and light section method, the so-called color highlight method (R, G, and B illumination is applied to the solder surface at different angles of incidence) is used to measure solder shape. A method of detecting the three-dimensional shape of the solder as two-dimensional hue information by photographing the reflected light of each color with a zenith camera) can be used.
　図２は、外観検査装置Ｙ３の、検査処理に関する機能の概略構成を示すブロック図である。外観検査装置Ｙ３の検査処理機能は、主として、カメラ３１、情報処理装置３２、表示装置３３によって実現される。情報処理装置３２は、ＣＰＵ（プロセッサ）、主記憶装置（メモリ）、補助記憶装置（ハードディスクなど）、入力装置（キーボード、マウス、コントローラ、タッチパネルなど）などを具備する汎用的なコンピュータシステムにより構成される。 FIG. 2 is a block diagram showing a schematic configuration of functions related to inspection processing of the visual inspection apparatus Y3. The inspection processing function of the visual inspection apparatus Y3 is mainly realized by the camera 31, the information processing device 32, and the display device 33. FIG. The information processing device 32 is configured by a general-purpose computer system including a CPU (processor), a main storage device (memory), an auxiliary storage device (hard disk, etc.), an input device (keyboard, mouse, controller, touch panel, etc.), and the like. be.
　検査対象である基板をカメラ３１によって撮影した画像を、画像取得部３２１が取得する。画像取得部３２１によって取得された画像に対して検査画像生成部３２２が、所定の処理を行うことにより検査画像を生成する。検査部３２３は、検査プログラムを実行することにより、検査画像をもとに、所定の指標を計測（計算）し、これらの計測値を用いて検査対象の状態を検査し、良否を判定する。ここで、実行される検査プログラムは、教師あり学習によって学習された学習器を含むプログラムである。検査プログラム記憶部３２４は、検査部３２３において実行される検査プログラムを記憶している。結果出力部３２５は、検査部３２３による検査結果を画面出力し、表示装置３３に表示させる。検査結果記憶部３２６は、検査画像と検査結果を関連付けて記憶する。通信インタフェース３２７は、ネットワーク（ＬＡＮ）を介して接続された管理装置１０、プログラム管理サーバ２０等と通信を行うためのインタフェースである。ここで、検査結果記憶部３２６に記憶される検査画像及び検査結果には、検査プログラムによって不良と判定されたものの、後の目視工程で良品と判定された検査画像（過検出された検査画像）に対するものも含まれる。 The image acquisition unit 321 acquires an image of the board to be inspected, which is captured by the camera 31 . The inspection image generation unit 322 generates an inspection image by performing predetermined processing on the image acquired by the image acquisition unit 321 . By executing the inspection program, the inspection unit 323 measures (calculates) a predetermined index based on the inspection image, inspects the state of the inspection object using these measured values, and determines the quality. Here, the inspection program to be executed is a program including a learner trained by supervised learning. The inspection program storage unit 324 stores inspection programs executed by the inspection unit 323 . The result output unit 325 outputs the inspection result by the inspection unit 323 on the screen and causes the display device 33 to display it. The inspection result storage unit 326 stores inspection images and inspection results in association with each other. The communication interface 327 is an interface for communicating with the management device 10, the program management server 20, etc., which are connected via a network (LAN). Here, the inspection images and inspection results stored in the inspection result storage unit 326 include inspection images that were determined to be defective by the inspection program but were determined to be non-defective in the subsequent visual inspection process (over-detected inspection images). Also includes those for
　検査結果記憶部３２６に記憶された検査画像及び検査結果は、通信インタフェース３２７を介して、管理装置１０に送信される。また、検査プログラムは、プログラム管理サーバ２０によって再学習され、プログラム管理サーバ２０から通信インタフェース３２７を介して検査プログラム記憶部３２４に送信され、記憶される。ここでは、外観検査装置Ｙ３について、検査処理に関する機能の概略を説明したが、他の検査装置も同様の機能構成を有する。 The inspection images and inspection results stored in the inspection result storage unit 326 are transmitted to the management device 10 via the communication interface 327. Further, the inspection program is re-learned by the program management server 20, transmitted from the program management server 20 via the communication interface 327 to the inspection program storage unit 324, and stored. Here, the outline of the functions related to the inspection process has been described for the visual inspection apparatus Y3, but other inspection apparatuses have the same functional configuration.
　Ｘ線検査装置Ｙ４は、Ｘ線像を用いて基板のはんだ付けの状態を検査するための装置である。例えば、ＢＧＡ（Ｂａｌｌ　Ｇｒｉｄ　Ａｒｒａｙ）、ＣＳＰ（Ｃｈｉｐ　Ｓｉｚｅ　Ｐａｃｋａｇｅ）などのパッケージ部品や多層基板の場合には、はんだ接合部が部品や基板の下に隠れているため、外観検査装置Ｙ３では（つまり外観画像では）はんだの状態を検査することができない。Ｘ線検査装置Ｙ４は、このような外観検査の弱点を補完するための装置である。Ｘ線検査装置Ｙ４の検査項目としては、例えば、部品の位置ずれ、はんだ高さ、はんだ体積、はんだボール径、バックフィレットの長さ、はんだ接合の良否などがある。なお、Ｘ線像としては、Ｘ線透過画像を用いてもよいし、ＣＴ（Ｃｏｍｐｕｔｅｄ　Ｔｏｍｏｇｒａｐｈｙ）画像を用いることも好ましい。 The X-ray inspection device Y4 is a device for inspecting the soldering state of the board using an X-ray image. For example, in the case of package parts such as BGA (Ball Grid Array) and CSP (Chip Size Package) and multi-layer boards, the solder joints are hidden under the parts and boards. In the image) it is not possible to inspect the state of the solder. The X-ray inspection apparatus Y4 is an apparatus for compensating for such weaknesses of appearance inspection. Items to be inspected by the X-ray inspection apparatus Y4 include, for example, component misalignment, solder height, solder volume, solder ball diameter, backfillet length, and solder joint quality. As the X-ray image, an X-ray transmission image may be used, and it is also preferable to use a CT (Computed Tomography) image.
　（管理装置）
　上述した製造装置Ｘ１～Ｘ３及び検査装置Ｙ１～Ｙ４は、ネットワーク（ＬＡＮ）を介して管理装置１０に接続されている。管理装置１０は、製造装置Ｘ１～Ｘ３および検査装置Ｙ１～Ｙ４の管理や制御を担うシステムであり、図示しないが、ＣＰＵ（プロセッサ）、主記憶装置（メモリ）、補助記憶装置（ハードディスクなど）、入力装置（キーボード、マウス、コントローラ、タッチパネルなど）、表示装置などを具備する汎用的なコンピュータシステムにより構成される。後述する管理装置１０の機能は、補助記憶装置に格納されたプログラムをＣＰＵが読み込み実行することにより実現される。 (Management device)
The manufacturing apparatuses X1 to X3 and the inspection apparatuses Y1 to Y4 described above are connected to the management apparatus 10 via a network (LAN). The management apparatus 10 is a system responsible for managing and controlling the manufacturing apparatuses X1 to X3 and the inspection apparatuses Y1 to Y4. It is composed of a general-purpose computer system equipped with an input device (keyboard, mouse, controller, touch panel, etc.), a display device, and the like. Functions of the management device 10, which will be described later, are realized by the CPU reading and executing a program stored in the auxiliary storage device.
　なお、管理装置１０は、１台のコンピュータにより構成してもよいし、複数のコンピュータにより構成してもよい。あるいは、製造装置Ｘ１～Ｘ３や検査装置Ｙ１～Ｙ４のいずれかの装置が内蔵するコンピュータに、管理装置１０の機能の全部又は一部を実装することも可能である。あるいは、管理装置１０の機能の一部をネットワーク上のサーバ（クラウドサーバなど）により実現してもよい。 It should be noted that the management device 10 may be composed of one computer, or may be composed of a plurality of computers. Alternatively, all or part of the functions of the management device 10 can be implemented in a computer built in any one of the manufacturing devices X1 to X3 and the inspection devices Y1 to Y4. Alternatively, part of the functions of the management device 10 may be realized by a server (such as a cloud server) on the network.
　管理装置１０には、ネットワーク（ＬＡＮ）を介して、プログラム管理サーバ２０が接続される。また、プログラム管理サーバ２０は、検査プログラム及び品質評価プログラムを管理するサーバである。プログラム管理サーバ２０は、ＣＰＵ（プロセッサ）、主記憶装置（メモリ）、補助記憶装置（ハードディスクなど）、入力装置（キーボード、マウス、コントローラ、タッチパネルなど）、表示装置などを具備する汎用的なコンピュータシステムにより構成される。検査プログラムは、検査装置Ｙ１～Ｙ４における検査処理を実現するプログラムであり、プログラム管理サーバ２０の所定の記憶領域に記憶される一方で、必要に応じて検査装置Ｙ１～Ｙ４のそれぞれにダウンロードされ、各装置の所定の記憶領域に記憶され、各装置において実行される。品質評価プログラムは、管理装置１０における品質評価処理を実現するプログラムであり、プログラム管理サーバ２０の所定の記憶領域に記憶される一方で、必要に応じて管理装置１０にダウンロードされ、品質評価プログラム記憶部１０９に記憶され、管理装置において実行される。プログラム管理サーバ２０は、検査プログラム、品質評価プログラムに限らず、製造装置Ｘ１～Ｘ３や検査装置Ｙ１～Ｙ４を制御する各種プログラムを管理してもよい。 A program management server 20 is connected to the management device 10 via a network (LAN). The program management server 20 is a server that manages inspection programs and quality evaluation programs. The program management server 20 is a general-purpose computer system including a CPU (processor), a main storage device (memory), an auxiliary storage device (hard disk, etc.), an input device (keyboard, mouse, controller, touch panel, etc.), a display device, and the like. Consists of The inspection program is a program that implements inspection processing in the inspection apparatuses Y1 to Y4, is stored in a predetermined storage area of the program management server 20, and is downloaded to each of the inspection apparatuses Y1 to Y4 as necessary, It is stored in a predetermined storage area of each device and executed in each device. The quality evaluation program is a program that implements the quality evaluation process in the management device 10, and is stored in a predetermined storage area of the program management server 20. It is also downloaded to the management device 10 as necessary and stored in the quality evaluation program. It is stored in the unit 109 and executed in the management device. The program management server 20 may manage not only the inspection program and the quality evaluation program but also various programs for controlling the manufacturing apparatuses X1 to X3 and the inspection apparatuses Y1 to Y4.
　本実施形態の管理装置１０は、製造設備の管理者が設備のメンテナンス及び品質管理を効率的に行うための機能を実現するための機能部を有している。図２に、管理装置１０が有する品質評価に関する各機能部のブロック図を示す。ここでは、管理装置１０が本発明の品質評価装置に対応し、管理システム１が本発明の検査管理システムに対応する。 The management device 10 of the present embodiment has a functional unit for implementing functions for the manager of manufacturing equipment to efficiently perform maintenance and quality control of the equipment. FIG. 2 shows a block diagram of each functional unit related to quality evaluation that the management device 10 has. Here, the management device 10 corresponds to the quality evaluation device of the invention, and the management system 1 corresponds to the inspection management system of the invention.
　図３に示すように、管理装置１０は、検査画像・検査結果取得部１０１、検査画像・検査結果データベース（ＤＢ）、品質評価部１０３、評価結果蓄積部１０４、品質変化判定部１０５、品質評価情報生成部１０６、表示部１０７、入力部１０８、品質評価プログラム記憶部１０９の各部を有している。ここでは、品質評価部１０３、評価結果蓄積部１０４及び品質変化判定部１０５が本発明の品質変化評価部に対応する。また、品質評価部１０３、評価結果蓄積部１０４、品質変化判定部１０５は、それぞれ本発明の品質評価部、品質評価蓄積部、変化判定部に対応する。また、品質評価情報生成部１０６が本発明の品質評価情報生成部に対応する。 As shown in FIG. 3, the management device 10 includes an inspection image/inspection result acquisition unit 101, an inspection image/inspection result database (DB), a quality evaluation unit 103, an evaluation result storage unit 104, a quality change determination unit 105, a quality evaluation It has an information generation unit 106 , a display unit 107 , an input unit 108 and a quality evaluation program storage unit 109 . Here, the quality evaluation section 103, evaluation result accumulation section 104, and quality change determination section 105 correspond to the quality change evaluation section of the present invention. Also, the quality evaluation section 103, the evaluation result storage section 104, and the quality change determination section 105 correspond to the quality evaluation section, quality evaluation storage section, and change determination section of the present invention, respectively. Also, the quality evaluation information generation unit 106 corresponds to the quality evaluation information generation unit of the present invention.
　検査画像・検査結果取得部１０１は、検査装置Ｙ１～Ｙ４による検査画像と、これに関連付けられた検査結果とが検査装置Ｙ１～Ｙ４から取得する。検査画像・検査結果取得部１０１によって取得された検査画像及び検査結果は検査画像・検査結果ＤＢに記憶される。ここでは、検査画像には、検査日時や線対称の品番等の情報も関連付けられており、これらの情報も併せて取得され、記憶される。 The inspection image/inspection result acquisition unit 101 acquires inspection images from the inspection devices Y1 to Y4 and inspection results associated therewith from the inspection devices Y1 to Y4. The inspection images and inspection results acquired by the inspection image/inspection result acquiring unit 101 are stored in the inspection image/inspection result DB. Here, the inspection image is associated with information such as the date and time of inspection and the product number of line symmetry, and this information is also acquired and stored.
　品質評価部１０３は、オートエンコーダ等の、教師なし学習によって学習された学習済みの学習器に、検査画像を入力することによって得られた出力によってプリント基板の品質を評価する機能を有する。ここで、品質評価部１０３に入力されるのは、検査画像・検査結果ＤＢ１０２に記録された検査画像のうち、良品と判定されたプリント基板の検査画像である。なお、良品と判定された検査画像には、検査装置における検査プログラムによって不良と判定されたものの、後の目視工程により良品と判定されたプリント基板に対する検査画像も含まれる。ここで用いられる学習済み学習器は、１次元の数値を出力し、これを異常度と呼ぶ。品質の評価に用いる検査画像は、適宜サンプリングすることができる。例えば、１時間に１回のサンプリング結果を所定期間蓄積し、異常度を算出する。ここで、異常度は、本発明の品質評価指標に対応する。 The quality evaluation unit 103 has a function of evaluating the quality of the printed circuit board based on the output obtained by inputting the inspection image to a learned device such as an autoencoder that has been learned by unsupervised learning. Here, among the inspection images recorded in the inspection image/inspection result DB 102, an inspection image of a printed circuit board determined to be non-defective is input to the quality evaluation unit 103. FIG. The inspection images determined to be non-defective products also include inspection images of printed circuit boards that were determined to be non-defective by the inspection program in the inspection apparatus but were determined to be non-defective products in the subsequent visual inspection process. A learned learner used here outputs a one-dimensional numerical value, which is called an anomaly degree. Inspection images used for quality evaluation can be appropriately sampled. For example, an hourly sampling result is accumulated for a predetermined period to calculate the degree of abnormality. Here, the degree of abnormality corresponds to the quality evaluation index of the present invention.
　品質評価部１０３において算出された異常度は、評価結果蓄積部１０４に蓄積される。ここでは、評価結果蓄積部１０４は、本発明の品質評価指標蓄積部に対応する。 The degree of anomaly calculated by the quality evaluation unit 103 is accumulated in the evaluation result accumulation unit 104. Here, the evaluation result storage section 104 corresponds to the quality evaluation index storage section of the present invention.
　品質変化判定部１０５では、評価結果蓄積部１０４に蓄積された異常度の経時的な変化を分析し、プリント基板の品質の変化を評価する。具体的には、異常度の変化と設定された閾値と比較して判断する。異常度は、上述のように、所定期間にわたって蓄積された異常度の平均値を当該期間の異常度とし、標準偏差を異常度のバラつきとする。ここでは、異常度の値に関する閾値（異常度）と、異常度のバラつきに関する閾値（バラつき）の２つの閾値が設定される。この閾値（異常度）と、閾値（バラつき）は、入力部１０８を介してユーザがあらかじめ入力する。閾値（異常度）及び閾値（バラつき）は、それぞれ本発明の第１閾値及び第２閾値に対応する。 The quality change determination unit 105 analyzes changes over time in the degree of abnormality accumulated in the evaluation result accumulation unit 104, and evaluates changes in the quality of the printed circuit board. Specifically, the determination is made by comparing the change in the degree of anomaly with a set threshold value. As for the degree of anomaly, as described above, the average value of the degrees of anomaly accumulated over a predetermined period is the degree of anomaly for that period, and the standard deviation is the variation in the degree of anomaly. Here, two thresholds are set: a threshold for the value of the degree of abnormality (abnormality) and a threshold for variation in the degree of abnormality (variation). The threshold value (abnormality degree) and threshold value (variation) are inputted in advance by the user via the input unit 108 . The threshold (abnormality) and threshold (variation) correspond to the first and second thresholds of the present invention, respectively.
　品質評価情報生成部１０６では、品質変化判定部１０５による評価結果に基づいて、品質評価情報を生成し、表示部１０７に表示させる。
　図４は、品質評価情報７１の表示例を示す。品質評価情報７１内の各情報を指示する枠線及び引き出し線は、品質評価情報７１の要素と区別するために点線で表示している。 The quality evaluation information generation unit 106 generates quality evaluation information based on the evaluation result by the quality change determination unit 105 and causes the display unit 107 to display the information.
FIG. 4 shows a display example of the quality evaluation information 71. As shown in FIG. Frame lines and lead lines indicating each piece of information in the quality evaluation information 71 are indicated by dotted lines to distinguish them from the elements of the quality evaluation information 71 .
　品質評価情報７１の中央には、管理図７１１が表示される。ここでは、品番Ａの管理図が表示されている。管理図７１１は、検査画像が取得された期間を横軸にとり、異常度を縦軸にとっている。異常度は、対象期間の平均値をプロットして破線で示し、実線の縦棒で標準偏差を示している。また、縦軸には、異常度の閾値が実線で表示されている。 A control chart 711 is displayed in the center of the quality evaluation information 71. Here, the control chart for product number A is displayed. In the control chart 711, the horizontal axis represents the period during which inspection images were acquired, and the vertical axis represents the degree of abnormality. The degree of anomaly is shown by plotting the average value of the target period with a dashed line, and the standard deviation is shown with a solid vertical bar. Moreover, the threshold value of the degree of abnormality is indicated by a solid line on the vertical axis.
　管理図７１１の下方には、閾値（異常度）の表示領域７１２が配置されており、現在設定されている「０．５」が表示されている。閾値（異常度）の表示領域７１２は入力フィールドになっており、数値を入力し、右横の設定ボタン７１３をクリック等により指定することにより、閾値（異常度）を変更することができる。閾値（異常度）の表示領域７１２の下方には、閾値（バラつき）の表示領域７１４が配置されており、現在設定されている「０．４」が表示されている。閾値（バラつき）の表示領域７１２は入力フィールドになっており、数値を入力し、右横の設定ボタン７１３をクリック等して押下することにより、閾値（バラつき）を変更することができる。 A threshold (abnormality) display area 712 is arranged below the control chart 711, and the currently set "0.5" is displayed. The threshold (abnormality) display area 712 is an input field, and the threshold (abnormality) can be changed by entering a numerical value and specifying it by clicking a setting button 713 on the right side. Below the threshold (abnormality) display area 712, a threshold (variation) display area 714 is arranged, and the currently set "0.4" is displayed. The threshold (variation) display area 712 is an input field, and the threshold (variation) can be changed by inputting a numerical value and pressing the setting button 713 on the right side by clicking.
　上述のように、閾値（異常度）及び閾値（バラつき）をユーザが入力し設定できるようにしてもよいが、閾値（異常度）及び閾値（バラつき）の履歴を所定の記憶領域に記憶しておき、閾値（異常度）及び閾値（バラつき）の履歴と、評価結果蓄積部１０４に蓄積された評価結果の履歴等に基づき、機械学習等によって適切な閾値（異常度）及び閾値（バラつき）を品質変化判定部１０５によって設定したり、ユーザに推奨値を教示したりしてもよい。閾値（異常度）及び閾値（バラつき）のいずれか一方のみをユーザが入力し、他方を自動設定し、又は推奨値を教示するようにしてもよい。ここでは、品質変化判定部１０５が、本発明の設定部に対応する。 As described above, the threshold (abnormality) and threshold (variation) may be input and set by the user. Appropriate thresholds (abnormalities) and thresholds (variations) are determined by machine learning or the like based on the history of thresholds (abnormalities) and thresholds (variations) and the history of evaluation results accumulated in the evaluation result accumulation unit 104. It may be set by the quality change determination unit 105, or a recommended value may be taught to the user. The user may input only one of the threshold (abnormality) and the threshold (variation), and the other may be automatically set or recommended values may be taught. Here, the quality change determination section 105 corresponds to the setting section of the present invention.
　管理図７１１の上方には、総合評価表示領域７１６が配置されている。総合評価表示領域７１６には品質に関するに関するメッセージが表示される。この総合評価は、管理図７１１に表示された異常度の変化に基づいたものである。管理図７１１では、異常度が閾値（異常度）を超えるとともに、バラつきも閾値（バラつき）を超ええている。このため、総合評価表示領域７１６には、品質の総合評価として、「品質にばらつきがあり、変化もあります。工程の見直しとＡＩモデルの再学習をお勧めします。」とのメッセージが表示され、検査に用いられている検査プログラムの学習器を再学習させるべきタイミングであることをユーザに報知している。 A comprehensive evaluation display area 716 is arranged above the control chart 711 . A comprehensive evaluation display area 716 displays a message regarding quality. This overall evaluation is based on changes in the degree of abnormality displayed in the control chart 711 . In the control chart 711, the degree of abnormality exceeds the threshold (degree of abnormality) and the variation also exceeds the threshold (variation). For this reason, in the comprehensive evaluation display area 716, a message is displayed as a comprehensive evaluation of quality, stating, "Quality varies and changes. It is recommended to review the process and relearn the AI model." , the user is informed that it is time to re-learn the learning device of the inspection program used for the inspection.
　このように、検査対象の品質の変化をとらえて、品質評価情報７１等が表示部１０７に表示され、ユーザに検査プログラムの学習器の再学習を勧めるメッセージが表示されて再学習が促されることで、信頼性の高い検査を実現することができる。 In this way, the quality evaluation information 71 and the like are displayed on the display unit 107 by capturing the change in the quality of the inspection object, and a message is displayed to encourage the user to relearn the learning device of the inspection program, thereby prompting the user to relearn. Therefore, a highly reliable inspection can be realized.
　品質評価情報７１の左側には、品番表示領域７１７が配置されている。検査プログラムは、部品種ごとにＡＩモデルを生成しており、１部品種の中には複数の品番が含まれる。ここでは、検査に用いられているＱＦＰ用ＡＩモデルには、品番Ａ、品番Ｂ、品番Ｃに対するものが含まれることが示されている。これらの品番の表示は、工程の見直しやＡＩモデルの再学習のアラームが出ている場合には、品番の表示が白色とは異なる赤色等の色で表示される。ここでは、品番Ａの表示７１７ａが異なる色で表示されている。このような表示により、ユーザは対応の必要な品番を明瞭に認識することができる。また、各品番表示７１７ａ等はボタンになっており、これらのボタンをクリック等して押下することにより、該当する品番の管理図に表示が切り替わる。また、ＱＦＰ用ＡＩモデルの表示７１７ｂをクリック等して押下することにより、全体の管理図を確認することができる。 A product number display area 717 is arranged on the left side of the quality evaluation information 71 . The inspection program generates an AI model for each part type, and one part type includes multiple part numbers. Here, it is shown that the QFP AI models used for inspection include those for product number A, product number B, and product number C. These product numbers are displayed in a color such as red that is different from white when there is an alarm for process review or re-learning of the AI model. Here, the display 717a of the product number A is displayed in a different color. Such a display allows the user to clearly recognize the product number that needs to be handled. Further, each product number display 717a and the like are buttons, and by clicking and pressing these buttons, the display is switched to the control chart of the corresponding product number. Further, by clicking and pressing the display 717b of the AI model for QFP, the entire control chart can be confirmed.
　品質評価情報７１の右側には、代表データの表示領域７１８が配置されている。ここでは、品質変化判定部１０５でサンプリングした検査画像のうち各期間の代表データを時系列で表示している。これによって、総合評価の妥当性を目視でも確認することができる。代表データの表示領域７１８の下方に配置された再選択ボタン７１９をクリック等して押下することにより、代表データが当該期間の他の検査画像に切り替えることできる。 A representative data display area 718 is arranged on the right side of the quality evaluation information 71 . Here, representative data for each period of the inspection images sampled by the quality change determination unit 105 are displayed in chronological order. This makes it possible to visually confirm the validity of the comprehensive evaluation. By clicking, for example, pressing a reselection button 719 arranged below the display area 718 of the representative data, the representative data can be switched to another inspection image in the relevant period.
　図５は、他の品質評価情報７２の表示例を示す。品質評価情報７１と共通する情報については、説明を省略する。ここでは、管理図７２１と総合評価表示領域７１６が品質評価情報７１とは異なっている。管理図７２１では、異常度は、閾値（異常度）を超えてはいないが、閾値（バラつき）を超えている。このため、総合評価表示領域７２６には、品質の総合評価として、「品質にばらつきがあります。工程の見直しをお勧めします。」とのメッセージが表示され、工程の見直しをすべきことをユーザに報知している。 FIG. 5 shows a display example of other quality evaluation information 72. Description of information common to the quality evaluation information 71 is omitted. Here, the control chart 721 and comprehensive evaluation display area 716 are different from the quality evaluation information 71 . In the control chart 721, the degree of abnormality does not exceed the threshold (degree of abnormality), but does exceed the threshold (variation). For this reason, in the comprehensive evaluation display area 726, a message is displayed as a comprehensive evaluation of quality, stating that "there is variation in quality. We recommend that you review the process." is notified to
　図６は、他の品質評価情報７３の表示例を示す。品質評価情報７１と共通する情報については、説明を省略する。ここでは、品質評価情報７１と共通する情報については、説明を省略する。ここでは、管理図７３１と総合評価表示領域７３６が品質評価情報７１とは異なっている。管理図７３１では、異常度は、閾値（異常度）を超えているが、閾値（バラつき）を超えていない。このため、総合評価表示領域７３６には、品質の総合評価として、「品質に変化があります。ＡＩモデルの再学習をお勧めします。」とのメッセージが表示され、検査プログラムの学習器を再学習させるべきタイミングであることをユーザに報知している。 FIG. 6 shows a display example of other quality evaluation information 73. Description of information common to the quality evaluation information 71 is omitted. Here, description of information common to the quality evaluation information 71 is omitted. Here, the control chart 731 and comprehensive evaluation display area 736 are different from the quality evaluation information 71 . In the control chart 731, the degree of abnormality exceeds the threshold (degree of abnormality), but does not exceed the threshold (variation). For this reason, in the overall evaluation display area 736, a message "The quality has changed. It is recommended to re-learn the AI model." The user is notified that it is time to learn.
　図７は、他の品質評価情報７４の表示例を示す。品質評価情報７１と共通する情報については、説明を省略する。ここでは、品質評価情報７１と共通する情報については、説明を省略する。ここでは、管理図７３１と総合評価表示領域７３６と品番表示領域７４７が品質評価情報７１とは異なっている。管理図７４１では、異常度は、閾値（異常度）も閾値（バラつき）を超えていない。このため、総合評価表示領域７４６には、品質の総合評価として、「品質は安定しています。」とのメッセージが表示される。また、品番表示領域７４７の品番Ａの表示７４７ａが白色で表示され、アラームが出ていないことを示している。 FIG. 7 shows a display example of other quality evaluation information 74. Description of information common to the quality evaluation information 71 is omitted. Here, description of information common to the quality evaluation information 71 is omitted. Here, the control chart 731 , comprehensive evaluation display area 736 and product number display area 747 are different from the quality evaluation information 71 . In the control chart 741, the degree of abnormality does not exceed the threshold (degree of abnormality) nor the threshold (variation). Therefore, in the overall evaluation display area 746, the message "Quality is stable" is displayed as the overall quality evaluation. Also, the display 747a of the product number A in the product number display area 747 is displayed in white, indicating that no alarm has been issued.
　このように、管理装置１０において、検査プログラムの学習器を再学習するタイミングをユーザに教示することができる。 Thus, in the management device 10, it is possible to teach the user when to relearn the learning device of the inspection program.
　品質変化判定部１０５によって、検査プログラムの学習器の再学習が勧められ、ユーザが検査プログラムの学習器の再学習を指示すると、管理するプログラム管理サーバ２０が、検査プログラムの再学習を行う。このとき、プログラム管理サーバ２０は、品質評価プログラムの学習器も再学習させる。このように、検査プログラムの再学習と併せて品質評価プログラムも再学習させることにより、品質変化に即応した検査と品質評価が可能となる。 Re-learning of the inspection program learning device is recommended by the quality change determination unit 105, and when the user instructs re-learning of the inspection program learning device, the managing program management server 20 re-learns the inspection program. At this time, the program management server 20 also re-learns the learning device of the quality evaluation program. In this way, by re-learning the quality evaluation program together with the re-learning of the inspection program, it is possible to perform inspection and quality evaluation immediately responding to quality changes.
　図８に、検査プログラムを学習させる学習装置２１について説明する。この学習装置２１は、プログラム管理サーバ２０に構成される。学習装置２１は、学習用検査画像取得部２１１、学習用検査結果取得部２１２、学習データ記憶部２１３、学習処理部２１４、学習器２１５を含む。再学習の場合には、学習データの対象が、再学習された時期に至るまでの直近の期間の検査画像及び検査結果となる。ここでは、学習器２１５が、本発明の第１学習器に対応する。学習装置２１が、本発明の学習装置に対応する。 The learning device 21 for learning the inspection program will be described with reference to FIG. This learning device 21 is configured in the program management server 20 . The learning device 21 includes a learning inspection image acquisition unit 211 , a learning inspection result acquisition unit 212 , a learning data storage unit 213 , a learning processing unit 214 and a learning device 215 . In the case of re-learning, the object of learning data is the inspection images and inspection results in the most recent period up to the time of re-learning. Here, the learning device 215 corresponds to the first learning device of the present invention. The learning device 21 corresponds to the learning device of the present invention.
　学習用検査画像取得部２１１は、検査画像・検査結果ＤＢ１０２から、再学習させるための検査画像を取得する。学習用検査結果取得部２１２は、検査画像・検査結果ＤＢ１０２から、当該検査画像の検査結果を取得する。 The learning inspection image acquisition unit 211 acquires inspection images for re-learning from the inspection image/inspection result DB 102 . The learning inspection result acquisition unit 212 acquires the inspection result of the inspection image from the inspection image/inspection result DB 102 .
　学習用検査画像取得部２１１及び学習用検査結果取得部２１２を介して取得された、学習用検査画像と学習用検査結果は、プログラム管理サーバ２０の補助記憶装置の所定領域である学習データ記憶部２１３に記憶される。 The learning test images and the learning test results acquired through the learning test image acquisition unit 211 and the learning test result acquisition unit 212 are stored in the learning data storage unit, which is a predetermined area of the auxiliary storage device of the program management server 20. 213.
　学習処理部２１４は、学習データ記憶部に２１３に記憶された学習用検査画像を入力すると検査結果を出力するように、学習器の機械学習を行う。学習処理部２１４は、プログラム管理サーバ２０のＣＰＵが補助記憶装置の所定領域に記憶された学習モデル生成プログラムを読み出して実行することにより実現される。ここで、学習器は、学習用検査画像を学習データ、学習用検査結果を教師データとして、例えば、モデルをニューラルネットワークにより計算するプログラムであるが、これに限られない。 The learning processing unit 214 performs machine learning of the learning device so that when the learning inspection image stored in the learning data storage unit 213 is input, the inspection result is output. The learning processing unit 214 is implemented by the CPU of the program management server 20 reading and executing a learning model generation program stored in a predetermined area of the auxiliary storage device. Here, the learning device is, for example, a program that calculates a model using a neural network using inspection images for learning as learning data and inspection results for learning as teacher data, but is not limited to this.
　学習処理部２１４における学習器の機械学習が、多数の学習データに対して繰り返されることにより、学習済みの学習器２１５が得られる。このようにして得られた学習器２１５は、補助記憶装置の所定領域である学習結果データ記憶部２１６に記憶される。学習結果データ記憶部２１６に記憶された学習済みの学習器２１５は、例えば、検査機Ｙ３の検査プログラム記憶部３２４に送信され、記憶される。 A learned learner 215 is obtained by repeating the machine learning of the learner in the learning processing unit 214 for a large number of learning data. The learning device 215 thus obtained is stored in the learning result data storage unit 216, which is a predetermined area of the auxiliary storage device. The learned learning device 215 stored in the learning result data storage unit 216 is transmitted to and stored in the inspection program storage unit 324 of the inspection machine Y3, for example.
　図９に、品質評価プログラムを学習させる学習装置２２について説明する。この学習装置２２は、プログラム管理サーバ２０に構成される。学習装置２２は、学習用良品画像取得部２２１、学習データ記憶部２２２、学習処理部２１４、学習器２２４を含む。再学習の場合には、学習データの対象が、再学習される時期に至るまでの直近の期間の良品画像となる。ここでは、学習器２２４が、本発明の第２学習器に対応する。 The learning device 22 for learning the quality evaluation program will be described with reference to FIG. This learning device 22 is configured in the program management server 20 . The learning device 22 includes a good product image acquisition unit 221 for learning, a learning data storage unit 222 , a learning processing unit 214 and a learning device 224 . In the case of re-learning, the target of the learning data is the non-defective product image in the most recent period up to the time of re-learning. Here, the learner 224 corresponds to the second learner of the present invention.
　学習用良品画像取得部２２１は、検査画像・検査結果ＤＢ１０２から、良品と判断された検査画像を取得する。 The learning-use non-defective product image acquisition unit 221 acquires inspection images determined to be non-defective products from the inspection image/inspection result DB 102 .
　学習用良品画像取得部２２１を介して取得された、学習用の良品画像は、プログラム管理サーバ２０の補助記憶装置の所定領域である学習データ記憶部２２２に記憶される。 The non-defective product image for learning acquired via the non-defective product image acquisition unit 221 for learning is stored in the learning data storage unit 222 which is a predetermined area of the auxiliary storage device of the program management server 20 .
　学習処理部２２３は、学習データ記憶部２２２に記憶された学習用良品画像を入力すると異常度を出力するように、学習器の機械学習を行う。学習処理部２２３は、プログラム管理サーバ２０のＣＰＵが補助記憶装置の所定領域に記憶された学習モデル生成プログラムを読み出して実行することにより実現される。ここで、学習器は、学習用の良品画像を学習データとするＶＡＥ等の教師なしＡＩモデルである。 The learning processing unit 223 performs machine learning of the learning device so that when the non-defective product image for learning stored in the learning data storage unit 222 is input, the degree of abnormality is output. The learning processing unit 223 is implemented by the CPU of the program management server 20 reading and executing a learning model generation program stored in a predetermined area of the auxiliary storage device. Here, the learning device is an unsupervised AI model such as VAE that uses non-defective images for learning as learning data.
　学習処理部２２３における学習器の機械学習が、多数の学習データに対して繰り返されることにより、学習済みの学習器２２４が得られる。このようにして得られた学習器２２４は、補助記憶装置の所定領域である学習結果データ記憶部２２５に記憶される。学習結果データ記憶部２２５に記憶された学習済み学習器２２４は、品質評価プログラム記憶部１０９に送信され、記憶される。 A learned learner 224 is obtained by repeating the machine learning of the learner in the learning processing unit 223 for a large number of learning data. The learning device 224 thus obtained is stored in the learning result data storage unit 225, which is a predetermined area of the auxiliary storage device. The trained learner 224 stored in the learning result data storage unit 225 is transmitted to the quality evaluation program storage unit 109 and stored.
〔実施例２〕
　以下では、本発明の実施例２に係る管理システム２について説明する。実施例１と共通の構成については同様の符号を用いて詳細な説明は省略する。 [Example 2]
A management system 2 according to a second embodiment of the present invention will be described below. Configurations common to the first embodiment are denoted by the same reference numerals, and detailed description thereof is omitted.
　実施例２に係る管理システム２の全体構成並びに検査装置Ｙ１～Ｙ４及び学習装置２１の機能構成は、実施例１と同様である。図１０に実施例２に係る管理装置１１が有する品質評価に関する各機能部のブロック図を示す。実施例２では、、品質評価プログラムが実施例１とは異なるのに伴い、管理装置１１の品質評価部２０３、評価結果蓄積部２０４、品質変化判定部２０５及び品質評価情報生成部２０６の機能、品質評価情報の構成、学習装置２３の構成が実施例１とは異なる。ここでは、管理装置１１が本発明の品質評価装置に対応し、管理システム２が本発明の検査管理システムに対応する。 The overall configuration of the management system 2 and the functional configurations of the inspection devices Y1 to Y4 and the learning device 21 according to the second embodiment are the same as those of the first embodiment. FIG. 10 shows a block diagram of each functional unit related to quality evaluation included in the management device 11 according to the second embodiment. In the second embodiment, since the quality evaluation program is different from that in the first embodiment, the functions of the quality evaluation unit 203, the evaluation result storage unit 204, the quality change determination unit 205, and the quality evaluation information generation unit 206 of the management device 11, The configuration of the quality evaluation information and the configuration of the learning device 23 are different from those of the first embodiment. Here, the management device 11 corresponds to the quality evaluation device of the invention, and the management system 2 corresponds to the inspection management system of the invention.
　図１１に実施例２に係る品質評価プログラムを機械学習させる学習装置２３の概略機能構成を示す。学習装置２３は、プログラム管理サーバ２０に構成される。学習装置２３は、学習用良品画像取得部２３１、学習データ記憶部２３２、学習処理部２３３、学習器２３４を含む。再学習の場合には、学習データの対象が、再学習される時期に至るまでの直近の期間の良品画像となる。ここでは、学習器２３４が、本発明の第３学習器に対応する。 　Fig. 11 shows a schematic functional configuration of the learning device 23 for performing machine learning on the quality evaluation program according to the second embodiment. The learning device 23 is configured in the program management server 20 . The learning device 23 includes a good product image acquisition unit 231 for learning, a learning data storage unit 232 , a learning processing unit 233 and a learning device 234 . In the case of re-learning, the target of the learning data is the non-defective product image in the most recent period up to the time of re-learning. Here, the learner 234 corresponds to the third learner of the present invention.
　学習用良品画像取得部２３１は、検査画像・検査結果ＤＢ１０２から、良品と判断された検査画像を取得する。 The learning-use non-defective product image acquisition unit 231 acquires inspection images determined to be non-defective products from the inspection image/inspection result DB 102 .
　学習用良品画像取得部２３１を介して取得された、学習用の良品画像は、プログラム管理サーバ２０の補助記憶装置の所定領域である学習データ記憶部２３２に記憶される。 The non-defective product image for learning acquired via the non-defective product image acquisition unit 231 for learning is stored in the learning data storage unit 232 which is a predetermined area of the auxiliary storage device of the program management server 20 .
　学習処理部２３３は、学習データ記憶部２３２に記憶された学習用良品画像を入力すると良品画像の特徴量を次元圧縮することにより、２次元平面上にマッピングし、マッピングされた各良品画像をクラスタリングするように、学習器の機械学習を行う。学習処理部２２３は、プログラム管理サーバ２０のＣＰＵが補助記憶装置の所定領域に記憶された学習モデル生成プログラムを読み出して実行することにより実現される。ここで、学習器は、学習用の良品画像を学習データとする主成分分析（ＰＣＡ）等の教師なしＡＩモデルである。 The learning processing unit 233 inputs the learning good product images stored in the learning data storage unit 232, dimensionally compresses the feature amounts of the good product images, maps them on a two-dimensional plane, and clusters the mapped good product images. Machine learning of the learner is performed so that The learning processing unit 223 is implemented by the CPU of the program management server 20 reading and executing a learning model generation program stored in a predetermined area of the auxiliary storage device. Here, the learning device is an unsupervised AI model such as principal component analysis (PCA) that uses non-defective images for learning as learning data.
　学習処理部２３３における学習器の機械学習が、多数の学習データに対して繰り返されることにより、学習済みの学習器２３４が得られる。このようにして得られた学習器２３４は、補助記憶装置の所定領域である学習結果データ記憶部２３５に記憶される。学習結果データ記憶部２３５に記憶された学習済み学習器２３４は、品質評価プログラム記憶部１０９に送信され、記憶される。図１２（Ａ）に、学習済み学習器２３４の出力例を示す。２次元面Ｓ上に学習させた良品画像を表すポイントＬ１、ポイントＬ２等がマッピングされ、これらのポイントＬ１等を囲む円Ｃによってクラスタリングされている。 A learned learner 234 is obtained by repeating the machine learning of the learner in the learning processing unit 233 for a large number of learning data. The learning device 234 thus obtained is stored in the learning result data storage unit 235, which is a predetermined area of the auxiliary storage device. The trained learner 234 stored in the learning result data storage unit 235 is transmitted to the quality evaluation program storage unit 109 and stored. FIG. 12A shows an output example of the learned learner 234 . A point L1, a point L2, etc. representing the learned non-defective product image are mapped on the two-dimensional surface S, and clustered by a circle C surrounding these points L1, etc. FIG.
　実施例２の品質評価部２０３では、このようにして生成された品質評価プログラムによって、検査画像・検査結果から取得した量産データの良品画像をマッピングする。図１２（Ｂ）が、学習済み学習器２３４に量産データである良品画像を入力し、２次元面Ｓ上にマッピングされた２次元マップを示す。ここでは、学習データである良品画像を黒色のポイントで表示し、量産データである良品画像を灰色のポイントで表示している。図１２（Ｂ）に示す状態では、量産データである良品画像を示すポイントＰ１、ポイントＰ２等が、２次元面Ｓ上にマッピングされているが、量産データである良品画像を示すポイントＰ１等は、いずれも円Ｃ１で示されるクラスタに含まれている。 The quality evaluation unit 203 of the second embodiment maps non-defective product images of mass production data acquired from inspection images/inspection results using the quality evaluation program generated in this way. FIG. 12B shows a two-dimensional map mapped on the two-dimensional surface S by inputting a non-defective product image, which is mass production data, to the learned learner 234 . Here, the non-defective product image, which is learning data, is displayed with black points, and the non-defective product image, which is mass production data, is displayed with gray points. In the state shown in FIG. 12(B), points P1, P2, etc. indicating a non-defective product image, which are mass production data, are mapped on the two-dimensional surface S. , are included in the cluster indicated by the circle C1.
　実施例２の品質評価部２０３では、上述のように、検査画像・検査結果ＤＢから取得した量産データである良品画像をサンプリングしてマッピングし、２次元マップを出力する。２次元マップは評価結果蓄積部２０４に蓄積する。 As described above, the quality evaluation unit 203 of the second embodiment samples and maps non-defective product images, which are mass production data obtained from the inspection image/inspection result DB, and outputs a two-dimensional map. The two-dimensional map is accumulated in the evaluation result accumulation unit 204. FIG.
　実施例２の品質変化判定部２０５では、品質評価部２０３によって生成された２次元マップにおいて、クラスタから外れる外れ値である良品画像を示すポイントの数をカウントする。図１１（Ｂ）に示すように、量産データである良品画像が逐次マッピングされていくと、品質が変化した場合に、外れ値として、クラスタＣから外れた位置にマッピングされるデータが現れる。図１２（Ｃ）では、学習用良品画像を示すポイントＬ１等、量産データである良品画像Ｌ１等を含むクラスタＣから外れた位置に、量産データである良品画像を示すポイントＯ１がマッピングされている。品質変化判定部２０５では、このような外れ値の個数をカウントし、外れ値の個数と閾値とを比較し、閾値を超えたか否かを判断する。 The quality change determination unit 205 of the second embodiment counts the number of points indicating non-defective images that are outliers outside the cluster in the two-dimensional map generated by the quality evaluation unit 203 . As shown in FIG. 11B, when non-defective product images, which are mass production data, are sequentially mapped, data mapped at positions outside cluster C appear as outliers when the quality changes. In FIG. 12C, a point O1 indicating a non-defective product image that is mass production data is mapped at a position outside a cluster C that includes a non-defective product image L1 that is mass production data, such as a point L1 that indicates a non-defective product image for learning. . The quality change determination unit 205 counts the number of such outliers, compares the number of outliers with a threshold, and determines whether or not the threshold is exceeded.
　実施例２においても、品質評価情報生成部２０６は、品質変化判定部２０５による評価結果に基づいて、品質評価情報を生成し、表示部１０７に表示させる。
　図１３は、実施例２に係る品質評価情報７５の表示例を示す。品質評価情報７５内の各情報を指示する枠線及び引き出し線は、品質評価情報７５の要素と区別するために点線で表示している。 Also in the second embodiment, the quality evaluation information generation unit 206 generates quality evaluation information based on the evaluation result by the quality change determination unit 205, and causes the display unit 107 to display the information.
FIG. 13 shows a display example of the quality evaluation information 75 according to the second embodiment. Frame lines and lead lines that indicate each piece of information in the quality evaluation information 75 are indicated by dotted lines to distinguish them from the elements of the quality evaluation information 75 .
　品質評価情報７５の中央右側には、２Ｄマップ７５１が表示されている。２Ｄマップ７５１には、品質評価の対象となっている品番（ここでは「品番Ａ」）と、いつの時点での評価データであるのかを示す日付（評価日）も表示される。表示される品質評価情報７５の評価日はユーザが適宜指定することができる。２Ｄマップは上述のように、品質評価プログラムに量産データである良品画像を入力し、学習用の良品画像から生成された２Ｄマップ上にマッピングしたものである。２Ｄマップ７５１のクラスタＣには学習用の良品画像を示すポイントＬ１等と、量産データである良品画像を示すポイントＰ１等が含まれている。２Ｄマップ７５１には、さらに、外れ値として、クラスタＣの外に、量産データである良品画像を示すポイントＯ１、Ｏ２、Ｏ３及びＯ４の４つのポイントが配置されている。外れ値である良品画像を示すポイントＯ４にカーソルを合わせると、当該ポイントに対応する良品画像データと、当該良品画像Ｉｍ４が検査された検査日とともにポップアップ表示される。 A 2D map 751 is displayed on the right side of the center of the quality evaluation information 75 . The 2D map 751 also displays the product number subject to quality evaluation (here, “product number A”) and the date (evaluation date) indicating when the evaluation data was obtained. The evaluation date of the displayed quality evaluation information 75 can be appropriately specified by the user. As described above, the 2D map is obtained by inputting a non-defective product image, which is mass production data, into the quality evaluation program and mapping it on a 2D map generated from the non-defective product image for learning. A cluster C of the 2D map 751 includes a point L1 indicating a good product image for learning and a point P1 indicating a good product image that is mass production data. In the 2D map 751, four points O1, O2, O3, and O4 representing non-defective images, which are mass-production data, are further arranged outside the cluster C as outliers. When a cursor is placed on a point O4 indicating a non-defective product image, which is an outlier, a pop-up display is displayed together with non-defective product image data corresponding to the point and the inspection date when the non-defective product image Im4 was inspected.
　品質評価情報７５の２Ｄマップ７５１の下方には、閾値（アラーム個数）の表示領域７５２が配置されており、現在設定されている「４」が表示されている。閾値（アラーム個数）の表示領域７５２は入力フィールドになっており、数値を入力し、右横の設定ボタン７５３をクリック等により指定することにより、閾値（アラーム個数）を変更することができる。この閾値（アラーム個数）は、２Ｄマップ上に配置される量産データの外れ値の個数が閾値（アラーム個数）以上となったときに、工程の見直し又は検査プログラムの学習器の再学習をユーザに勧めるアラームを出すためのものである。ここでは、閾値（アラーム個数）が、本発明の第３閾値に対応する。 A threshold (number of alarms) display area 752 is arranged below the 2D map 751 of the quality evaluation information 75, and the currently set "4" is displayed. The display area 752 for the threshold (the number of alarms) is an input field, and the threshold (the number of alarms) can be changed by entering a numerical value and specifying it by clicking a setting button 753 on the right side. This threshold (number of alarms) instructs the user to review the process or re-learn the inspection program learner when the number of outliers in the mass production data arranged on the 2D map exceeds the threshold (number of alarms). It is for issuing a recommendation alarm. Here, the threshold (number of alarms) corresponds to the third threshold of the present invention.
　上述のように、閾値（アラーム個数）をユーザが入力し設定できるようにしてもよいが、閾値（異常度）及び閾値（アラーム個数）の履歴を所定の記憶領域に記憶しておき、閾値（アラーム個数）の履歴と、評価結果蓄積部２０４に蓄積された評価結果の履歴等に基づき、機械学習等によって適切な閾値（アラーム個数）を品質変化判定部２０５によって設定したり、ユーザに推奨値を教示したりしてもよい。ここでは、品質変化判定部２０５が、本発明の閾値設定部に対応する。 As described above, the threshold (the number of alarms) may be input and set by the user. Based on the history of the number of alarms) and the history of the evaluation results accumulated in the evaluation result accumulation unit 204, an appropriate threshold value (the number of alarms) is set by the quality change determination unit 205 by machine learning or the like, and a recommended value is given to the user. may be taught. Here, the quality change determination section 205 corresponds to the threshold setting section of the present invention.
　品質評価情報７５の上方には、総合評価表示領域７５４が配置されている。総合評価表示領域７５４には品質に関するメッセージが表示される。この総合評価は、２Ｄマップ７５１における外れ値の個数に基づいたものである。閾値（アラーム個数）の表示領域７５２に表示されているように、現在の閾値（アラーム個数）は４である。これに対して２Ｄマップ７５１には、外れ値としてポイントＯ１、Ｏ２、Ｏ３及びＯ４の４つのポイントがマッピングされているので、外れ値の個数は、閾値（アラーム個数）と等しくなっている。そのため、品質変化判定部１０５は、品質が変化したと評価し、総合評価表示領域７５４に、「品質が変化しました。工程の見直しもしくはＡＩモデルの再学習をお勧めします。」とのメッセージが表示され、工程を見直すか、検査に用いられている検査プログラムの学習器を再学習させるべきであることをユーザに報知している。 A comprehensive evaluation display area 754 is arranged above the quality evaluation information 75 . A message regarding quality is displayed in the comprehensive evaluation display area 754 . This overall evaluation is based on the number of outliers in the 2D map 751 . As shown in the threshold (number of alarms) display area 752, the current threshold (number of alarms) is four. On the other hand, since four points O1, O2, O3 and O4 are mapped as outliers in the 2D map 751, the number of outliers is equal to the threshold (the number of alarms). Therefore, the quality change determination unit 105 evaluates that the quality has changed, and displays a message in the overall evaluation display area 754 stating, "The quality has changed. It is recommended to review the process or re-learn the AI model." is displayed to inform the user that the process should be reviewed or the learner of the inspection program used for the inspection should be retrained.
　このように、検査対象の品質の変化をとらえて、品質評価情報７５が表示部１０７に表示され、ユーザに検査プログラムの学習器の再学習を勧めるメッセージが表示されて再学習が促されることで、信頼性の高い検査を実現することができる。 In this way, the quality evaluation information 75 is displayed on the display unit 107 in response to changes in the quality of the inspection object, and a message is displayed to encourage the user to re-learn the learning device of the inspection program. , a highly reliable inspection can be realized.
　品質評価情報７５の左側には、品番表示領域７５５が配置されている。検査プログラムは、部品種ごとにＡＩモデルを生成しており、１部品種の中には複数の品番が含まれる。ここでは、検査に用いられているＱＦＰ用ＡＩモデルには、品番Ａ、品番Ｂ、品番Ｃに対するものが含まれることが示されている。これらの品番の表示は、工程の見直しやＡＩモデルの再学習のアラームが出ている場合には、品番の表示が白色とは異なる赤色等の色で表示される。ここでは、品番Ａの表示７５５ａが異なる色で表示されている。このような表示により、ユーザは対応の必要な品番を明瞭に認識することができる。また、各品番表示７５５ａ等はボタンになっており、これらのボタンをクリック等して押下することにより、該当する品番の２Ｄマップに表示が切り替わる。また、ＱＦＰ用ＡＩモデルの表示７５５ｂをクリック等して押下することにより、全体の２Ｄマップを確認することができる。 A product number display area 755 is arranged on the left side of the quality evaluation information 75 . The inspection program generates an AI model for each part type, and one part type includes multiple part numbers. Here, it is shown that the QFP AI models used for inspection include those for product number A, product number B, and product number C. These product numbers are displayed in a color such as red that is different from white when there is an alarm for process review or re-learning of the AI model. Here, the display 755a of the product number A is displayed in a different color. Such a display allows the user to clearly recognize the product number that needs to be handled. Further, each product number display 755a and the like are buttons, and by clicking and pressing these buttons, the display is switched to the 2D map of the corresponding product number. Also, by clicking and pressing the display 755b of the AI model for QFP, the entire 2D map can be confirmed.
＜付記１＞
　検査対象を撮像して生成した検査画像を学習データとし、検査結果を教師データとして機械学習させて生成した学習済み第１学習器（２１５）を用いた検査処理において良品と判定された該検査画像である良品画像を取得する良品画像取得部（１０１）と、
　前記検査対象の品質の変化を評価する品質変化評価部（１０３、１０４、１０５と、
　前記評価の結果を含む品質評価情報を生成する品質評価情報生成部（１０６）と、
を備えた品質評価装置（１０）。 <Appendix 1>
The inspection image determined to be non-defective in the inspection process using the trained first learner (215) generated by performing machine learning using the inspection image generated by imaging the inspection object as learning data and using the inspection result as teacher data. a non-defective product image acquisition unit (101) for acquiring a non-defective product image;
Quality change evaluation units (103, 104, 105) for evaluating changes in quality of the inspection object,
a quality evaluation information generation unit (106) for generating quality evaluation information including the result of the evaluation;
A quality evaluation device (10) comprising:
１０、１１　　　：管理装置
１０３　　　：品質評価部
１０４　　　：評価結果蓄積部
１０５　　　：品質変化評価部
１０６　　　：品質評価情報生成部
２１５　　　：学習器 10, 11: management device 103: quality evaluation unit 104: evaluation result accumulation unit 105: quality change evaluation unit 106: quality evaluation information generation unit 215: learning device

Claims (11)

  1. 　検査対象を撮像して生成した検査画像を学習データとし、検査結果を教師データとして機械学習させて生成した学習済み第１学習器を用いた検査処理において良品と判定された該検査画像である良品画像を取得する良品画像取得部と、
    　前記検査対象の品質の変化を評価する品質変化評価部と、
    　前記評価の結果を含む品質評価情報を生成する品質評価情報生成部と、
    を備えた品質評価装置。     A non-defective product that is determined as a non-defective product in an inspection process using a trained first learning device that is generated by machine-learning an inspection image generated by capturing an image of an inspection target as learning data and using an inspection result as teacher data. a non-defective product image acquisition unit that acquires an image;
    a quality change evaluation unit that evaluates a change in quality of the inspection target;
    a quality evaluation information generating unit that generates quality evaluation information including the result of the evaluation;
    A quality evaluation device with
  2. 　前記品質変化評価部は、
    　前記良品画像に基づいて、前記品質を評価する品質評価指標を出力する品質評価部と、
    　前記品質評価指標を蓄積する品質評価指標蓄積部と、
    　前記品質評価指標蓄積部に蓄積された前記品質評価指標の所定期間にわたる変化に基づいて、前記品質の変化を判定する変化判定部と、
    を有することを特徴とする請求項１に記載の品質評価装置。     The quality change evaluation unit
    a quality evaluation unit that outputs a quality evaluation index for evaluating the quality based on the non-defective product image;
    a quality evaluation index accumulation unit that accumulates the quality evaluation index;
    a change determination unit that determines a change in the quality based on a change in the quality evaluation index accumulated in the quality evaluation index accumulation unit over a predetermined period;
    The quality evaluation device according to claim 1, characterized by comprising:
  3. 　前記品質評価指標は、前記良品画像を学習データとして教師なし学習によって機械学習させて生成した学習済み第２学習器により出力される異常度であり、
    　前記変化判定部は、所定期間にわたる前記異常度の平均値及び標準偏差を算出し、該平均値及び該標準偏差を、それぞれ第１閾値及び第２閾値と比較して前記品質の変化を判定することを特徴とする請求項２に記載の品質評価装置。     The quality evaluation index is an abnormality degree output by a trained second learner generated by machine learning by unsupervised learning using the good product image as learning data,
    The change determination unit calculates the average value and standard deviation of the degree of abnormality over a predetermined period, compares the average value and the standard deviation with the first threshold and the second threshold, respectively, and determines the change in quality 3. The quality evaluation device according to claim 2, characterized in that:
  4. 　前記第１閾値及び前記第２閾値の少なくともいずれかを自動で設定する設定部を備えたことを特徴とする請求項３に記載の品質評価装置。 The quality evaluation device according to claim 3, further comprising a setting unit that automatically sets at least one of the first threshold and the second threshold.
  5. 　前記品質変化評価部は、
    　前記良品画像を学習データとして教師なし学習によって機械学習させて生成した学習済み第３学習器により、クラスタリングを行い、外れ値の個数と、第３閾値とを比較して前記品質の変化を評価することを特徴とする請求項１に記載の品質評価装置。     The quality change evaluation unit
    Clustering is performed by a trained third learner generated by machine learning by unsupervised learning using the non-defective image as learning data, and the number of outliers is compared with a third threshold to evaluate the change in quality. The quality evaluation device according to claim 1, characterized by:
  6. 　前記第３閾値を自動で設定する閾値設定部を備えたことを特徴とする請求項５に記載の品質評価装置。 The quality evaluation device according to claim 5, further comprising a threshold setting unit that automatically sets the third threshold.
  7. 　前記品質評価情報は、前記第１学習器の再学習を勧める情報を含むことを特徴とする請求項１乃至６のいずれか１項に記載の品質評価装置。 The quality evaluation device according to any one of claims 1 to 6, wherein the quality evaluation information includes information recommending re-learning of the first learner.
  8. 　前記品質評価情報は、前記検査対象に関する前工程の改善を勧める情報を含むことを特徴とする請求項１乃至７のいずれか１項に記載の品質評価装置。 The quality evaluation apparatus according to any one of claims 1 to 7, characterized in that the quality evaluation information includes information recommending improvement of the previous process regarding the inspection object.
  9. 　前記品質評価情報を表示する表示部を備えたことを特徴とする請求項１乃至８のいずれか１項に記載の品質評価装置。 The quality evaluation device according to any one of claims 1 to 8, further comprising a display section for displaying the quality evaluation information.
  10. 　請求項１乃至９のいずれか１項に記載の品質評価装置と、
    　前記検査対象に対して前記学習済み第１学習器を用いた検査処理を実施する検査処理部と、該検査処理に供される前記検査画像と該検査処理による結果である前記検査結果を記憶する記憶部とを、備えた検査装置と
    を含む検査管理システム。     A quality evaluation device according to any one of claims 1 to 9;
    An inspection processing unit that performs inspection processing using the learned first learning device on the inspection object, and stores the inspection image that is provided for the inspection processing and the inspection result that is the result of the inspection processing. An inspection management system comprising: a storage unit;
  11. 　前記第１学習器を再学習させる学習装置を含むことを特徴とする請求項１０に記載の検査管理システム。 The inspection management system according to claim 10, further comprising a learning device for re-learning the first learning device.
PCT/JP2021/047106 2021-03-15 2021-12-20 Quality evaluation device and inspection management system WO2022196003A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2021041048A JP2022140951A (en) 2021-03-15 2021-03-15 Quality evaluation device and inspection management system
JP2021-041048 2021-03-15

Publications (1)

Publication Number Publication Date
WO2022196003A1 true WO2022196003A1 (en) 2022-09-22

Family

ID=83320087

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/047106 WO2022196003A1 (en) 2021-03-15 2021-12-20 Quality evaluation device and inspection management system

Country Status (2)

Country Link
JP (1) JP2022140951A (en)
WO (1) WO2022196003A1 (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010140444A (en) * 2008-12-15 2010-06-24 Canon Inc Inspection system and method of controlling same, and computer program
JP2010249547A (en) * 2009-04-10 2010-11-04 Denso Corp Visual examination device and visual examination method
US20170082555A1 (en) * 2015-09-18 2017-03-23 Kla-Tencor Corporation Adaptive Automatic Defect Classification
JP2019049778A (en) * 2017-09-07 2019-03-28 日本電信電話株式会社 Detection apparatus, detection method and detection program
WO2019138655A1 (en) * 2018-01-09 2019-07-18 日本電信電話株式会社 Model learning device, model learning method, and program
JP2020187071A (en) * 2019-05-16 2020-11-19 株式会社キーエンス Image inspection device and method for setting image inspection device

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2010140444A (en) * 2008-12-15 2010-06-24 Canon Inc Inspection system and method of controlling same, and computer program
JP2010249547A (en) * 2009-04-10 2010-11-04 Denso Corp Visual examination device and visual examination method
US20170082555A1 (en) * 2015-09-18 2017-03-23 Kla-Tencor Corporation Adaptive Automatic Defect Classification
JP2019049778A (en) * 2017-09-07 2019-03-28 日本電信電話株式会社 Detection apparatus, detection method and detection program
WO2019138655A1 (en) * 2018-01-09 2019-07-18 日本電信電話株式会社 Model learning device, model learning method, and program
JP2020187071A (en) * 2019-05-16 2020-11-19 株式会社キーエンス Image inspection device and method for setting image inspection device

Also Published As

Publication number Publication date
JP2022140951A (en) 2022-09-29

Similar Documents

Publication Publication Date Title
CN107734955B (en) Inspection device for surface mounting line, quality management system, and recording medium
CN109946319B (en) Printed circuit board inspection apparatus, method of determining defect type of screen printer, and computer-readable recording medium
CN105051781B (en) NI Vision Builder for Automated Inspection and the method for the long-range quality inspection for product
JPS62219632A (en) Apparatus and method for automatic inspection of soldering state of circuit board
JP6451142B2 (en) Quality control device and control method for quality control device
CN110045688B (en) Inspection management system, inspection management device, and inspection management method
JP6277754B2 (en) Quality control system and internal inspection device
CN106168582B (en) Inspection apparatus and inspection method
CN109840856A (en) Management system, managing device, management method and computer readable recording medium
KR20200006028A (en) Printed circuit board inspecting apparatus, method for determining fault type of screen printer and computer readable recording medium
WO2022196003A1 (en) Quality evaluation device and inspection management system
JP4813071B2 (en) Macro image display system and macro image display method
EP3104169B1 (en) Quality management system
US20240085896A1 (en) Quality improvement assisting device and quality improvement assisting system
JP2012253199A (en) Inspection result display device, substrate inspection system, and inspection result display method
JP7484733B2 (en) Management system, management device, management method, and program
JP2023018016A (en) Management system and cause analysis system
WO2022190543A1 (en) Information generation device
WO2022157995A1 (en) Inspection management system, inspection management device, inspection management method, and program
WO2022162961A1 (en) Management device and management system
WO2022162962A1 (en) Quality improvement assistance device
EP4134883B1 (en) Systems and methods for automated x-ray inspection
TWI822269B (en) Image inspection method, image management method of defective parts, and image inspection device
JPH0968415A (en) Method of soldering parts to printed circuit board, soldering inspection method and apparatus therefor
TW202238447A (en) System for using image features corresponding to component identification for secondary inspection and method thereof

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21931745

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21931745

Country of ref document: EP

Kind code of ref document: A1