JP2022043134A5 - - Google Patents

Description

The present invention relates to a quality determination system, a quality determination method, a server, and a program.

Patent Document 1 describes a service providing system that provides services using machine learning using artificial intelligence. This service provision system is a service provision system that provides services using machine learning using artificial intelligence. a machine learning means for generating a model, a personalization means for personalizing the general model into a model suitable for the user based on information sent from the user; and a service providing means for providing a personalized service to the user using the model, and the information sent from the user is used for both the machine learning and the personalization.

Patent Document 2 describes a method of visualizing and displaying a sound source position in which the position of an arbitrary sound source is visualized in real time by associating it with real space. This visualization display method detects one or more sounds, locates the location of each sound source, converts information about the sound source, including at least the location of the sound source, into visual information, and superimposes it in real time with an actual image around the sound source. It is characterized by displaying.

JP2016-48417A Japanese Patent Application Publication No. 2004-77277

An object of the present invention is to provide a quality determination system, a quality determination method, a server, and a program that can determine the quality of an inspected object with higher accuracy.

The invention according to claim 1 provides a visualization device that includes a detector for measuring a physical quantity that changes due to a sign of a malfunction occurring in a monitored object, and that generates a plurality of visualized images that visualize the physical quantity. a teaching data group generation unit that classifies the visualized image including a plurality of pre-obtained defects according to parameters characterizing the defects and generates a plurality of classified teaching data groups, and the plurality of teaching data a storage unit that stores the plurality of machine learning models each constructed based on a group; an optimal model selection unit that selects an optimal model that is an optimal machine learning model from among the plurality of machine learning models; and the visualization A quality determination system comprising: a determination unit that determines whether the operating state of the monitored object is good or bad based on the plurality of visualized images generated by the device and the optimal model selected by the optimal model selection unit. It is.

The invention according to claim 2 provides a sound source visualization device that includes a microphone for identifying a sound source generated from a monitored target and that generates a plurality of sound source visualization images visualizing the sound source, and a plurality of pre-obtained defects. a teaching data group generation unit that classifies the sound source visualization image including the defect according to a parameter characterizing the defect and generates a plurality of classified teaching data groups; a storage unit that stores a plurality of the machine learning models; an optimal model selection unit that selects an optimal model that is an optimal machine learning model from the plurality of machine learning models; The quality determination system includes a determination unit that determines whether the operating state of the monitored object is good or bad based on a plurality of sound source visualization images and the optimal model selected by the optimal model selection unit.

The invention according to claim 3 includes a vibration visualization device that includes a vibration sensor for detecting vibrations generated from a monitored object and generates a plurality of vibration visualization images visualizing the vibrations, and a vibration visualization device that generates a plurality of vibration visualization images that visualize the vibrations; a teaching data group generation unit that classifies the vibration visualization image including a defect according to a parameter characterizing the defect and generates a plurality of classified teaching data groups; and a teaching data group generation unit that generates a plurality of classified teaching data groups; a storage unit that stores the plurality of machine learning models generated by the vibration visualization device; an optimal model selection unit that selects an optimal model from among the plurality of machine learning models; The quality determination system includes a determination unit that determines whether the operating state of the monitored object is good or bad based on the plurality of vibration visualization images and the optimal model selected by the optimal model selection unit.

The invention according to claim 4 is a pass/fail judgment method using the pass/fail judgment system according to claim 9, wherein the visualization device generates the visualized image, and the teaching data group generation unit creating a plurality of teaching data groups; storing the plurality of machine learning models in the storage unit; and selecting an optimum machine learning model from among the plurality of machine learning models. The method includes the steps of: selecting the machine learning model; and the determining unit determining whether the operating state of the monitored object is good or bad using the machine learning model selected by the optimal model selecting unit.

The invention according to claim 5 provides a plurality of teaching data groups in which a plurality of visualized images that visualize physical quantities that have changed due to a sign of a malfunction occurring in a monitored object are classified according to parameters characterizing the malfunction. a cloud computing service that constructs a plurality of machine learning models for determining whether the operation status of the monitored target is good or bad based on the above, and a cloud computing service that constructs a plurality of machine learning models for respectively determining the good or bad operating status of the monitored target based on the plurality of machine learning models; a determination device that makes a determination, a teaching data group generation unit that generates the plurality of teaching data groups, the server being connected to the server via a network, and a teaching data group generating unit that generates the plurality of teaching data groups, and selecting an optimal machine learning model from among the plurality of machine learning models. The server is equipped with an optimal model selection unit that selects an optimal model.

The invention according to claim 6 provides a plurality of teaching data groups in which a plurality of visualized images that visualize physical quantities that have changed due to a sign of a malfunction occurring in a monitored object are classified according to parameters characterizing the malfunction. a cloud computing service that constructs a plurality of machine learning models for determining whether the operation status of the monitored target is good or bad based on the above, and a cloud computing service that constructs a plurality of machine learning models for respectively determining the good or bad operating status of the monitored target based on the plurality of machine learning models; a teaching data group generation means for generating the plurality of teaching data groups, a teaching data group generating means for generating the plurality of teaching data groups, and a teaching data group generation means for evaluating the pass/fail results judged by the judgment device, and a computer connected to the plurality of machines via a network. This is a program that functions as an optimal model selection means that selects the optimal machine learning model from among the learning models.

The invention according to claim 7 provides a plurality of teaching data groups in which a plurality of visualized images that visualize physical quantities that have changed due to a sign of a malfunction occurring in a monitored object are classified according to parameters characterizing the malfunction. a cloud computing service that constructs a plurality of machine learning models for determining whether the operating state of the monitored object is good or bad based on the above, and an imaging unit that captures an image of the inspected object. A storage means for storing the plurality of machine learning models constructed by the cloud computing service; and a storage means for storing the plurality of machine learning models constructed by the cloud computing service. This is a program that functions as a determining means for determining the quality of the inspection object.

The invention according to claim 8 includes a teaching data group generation unit that classifies images including defects of a plurality of objects to be inspected according to parameters characterizing the defects, and generates a plurality of classified teaching data groups. , a storage unit that stores the plurality of machine learning models each constructed based on the plurality of teaching data groups; a camera unit that captures an image of the object to be inspected; A determination unit that determines the quality of the inspected object imaged by the camera unit, and a determination unit that evaluates the results of the determination by the determination unit, based on machine learning models, and select the optimal one from among the plurality of machine learning models. This is a pass/fail determination system that includes an optimal model selection unit that selects a machine learning model.

The invention according to claim 9 is a method for determining the quality of each of the objects to be inspected based on a plurality of teaching data groups in which images including defects of the plurality of objects to be inspected are classified according to the brightness that characterizes the defects. A server connected via a network to a cloud computing service that constructs a plurality of machine learning models for determination, and a determination device that determines the quality of the inspected object based on the plurality of machine learning models. A teaching data group generation unit that generates the plurality of teaching data groups, and evaluating the pass/fail results determined by the determination device and selecting an optimal machine learning model from among the plurality of machine learning models. The server is equipped with an optimal model selection unit that performs the following operations.

According to the tenth aspect of the invention, the quality of each of the objects to be inspected is determined based on a plurality of teaching data groups in which images including defects of the plurality of objects to be inspected are classified according to the brightness that characterizes the defects. A computer connected via a network to a cloud computing service that constructs a plurality of machine learning models for determination, and a determination device that determines the quality of the inspected object based on the plurality of machine learning models. a teaching data group generation unit that generates the plurality of teaching data groups; an optimal model that evaluates the pass/fail results determined by the determination device and selects an optimal machine learning model from among the plurality of machine learning models; This is a program that functions as a selection means.

The invention according to claim 11 provides a server having a teaching data group generation unit that generates a plurality of teaching data groups in which images including defects of a plurality of objects to be inspected are classified according to the brightness characterizing the defects. , connected to a cloud computing service that constructs a plurality of machine learning models, each of which determines the quality of the object to be inspected, based on the plurality of teaching data groups, and an imaging unit that captures an image of the object to be inspected. a storage means for storing the plurality of machine learning models constructed by the cloud computing service, each of which is imaged by the imaging unit based on the plurality of machine learning models stored in the storage unit; This is a program that functions as a determining means for determining the quality of the inspected object.

According to the present invention, it is possible to provide a quality determination system, a quality determination method, a server, and a program that can determine the quality of an object to be inspected with higher accuracy.

FIG. 1 is a configuration diagram of a quality determination system according to a first embodiment of the present invention. FIG. 2 is an explanatory diagram of a teaching data creation device included in the pass/fail judgment system. FIG. 2 is an explanatory diagram of a server included in the pass/fail judgment system. FIG. 2 is an explanatory diagram of a teaching data group generated by the pass/fail determination system and a plurality of machine learning models each constructed from the teaching data groups. FIG. 2 is an explanatory diagram of an image including defects in an object to be inspected. FIG. 3 is an explanatory diagram of a horizontal gray value profile graph schematically showing a part of a scanning position. FIG. 3 is a horizontal gray value profile graph of image A schematically illustrated for a portion of the scan position; FIG. FIG. 3 is a horizontal gray value profile graph of image B schematically shown for a portion of the scan position; FIG. FIG. 3 is a horizontal gray value profile graph of image C schematically shown for a portion of the scan position; FIG. FIG. 2 is an explanatory diagram of preprocessing by the pass/fail judgment system. FIG. 2 is an explanatory diagram of a terminal and an imaging unit included in the pass/fail judgment system. FIG. 2 is a flow diagram showing the operation of the same-quality determination system. It is a block diagram of the quality determination system based on the 2nd Embodiment of this invention. FIG. 2 is an explanatory diagram of a terminal and an imaging unit included in the pass/fail judgment system. It is a block diagram of the predictive maintenance system based on the 3rd Embodiment of this invention. FIG. 2 is an explanatory diagram of a server included in the predictive maintenance system. FIG. 2 is an explanatory diagram of a terminal and a sound source visualization device included in the predictive maintenance system. It is a flow diagram showing the operation of the same predictive maintenance system. It is a block diagram of the predictive maintenance system based on 4th Embodiment of this invention.

Next, embodiments embodying the present invention will be described with reference to the attached drawings to provide an understanding of the present invention. Note that in the figures, illustrations of parts not related to the explanation may be omitted.

[First embodiment]
The quality determination system 10a (see FIG. 1) according to the first embodiment of the present invention can determine the quality of the appearance of the inspection object 14, which is a product manufactured by a user, by machine learning. The quality of this appearance is determined by the presence or absence of defects such as scratches and adhesion of foreign matter.
The objects to be inspected 14 are, for example, parts of transportation machines such as automobiles and airplanes, and foods. However, the inspected object 14 is not limited to these parts and foods.
The quality determination service by this quality determination system 10a is provided to users by a service provider.

As shown in FIG. 1, the quality determination system 10a includes a teaching data creation device 20, a server 30, and an inspection system 40a.
The teaching data creation device 20, the server 30, and the inspection system 40a are connected to each other via the Internet N.

The teaching data creation device 20 is, for example, a personal computer. The teaching data creation device 20 is managed by a service provider or a user using the inspection system 40a, and has a teaching data creation unit 202, as shown in FIG.
The teaching data creation unit (an example of teaching data creation means) 202 acquires an image of the object to be inspected 14 from, for example, a camera 460 provided in the inspection system 40a, and uses the image of the object to be inspected as teaching data for constructing a machine learning model. An image of object 14 can be created.
Note that the teaching data creation device 20 functions as a teaching data creation means by a program executed by the teaching data creation device 20.
The teaching data creation device 20 may be a mobile terminal such as a smartphone.

The server 30 is managed by a service provider, and includes a teaching data group generation section 302, an optimal model selection section 306, and a management section 308, as shown in FIG.
As shown in FIG. 4, the teaching data group generation unit (an example of teaching data group generation means) 302 generates images IMG1, IMG2, IMG3, By preprocessing the teaching data group TD, a set TDg of preprocessed teaching data groups, that is, teaching data groups TD1, TD2, TD3, TD4, . . . is created.

Here, when each image IMG1, IMG2, IMG3, . As shown, its size is 1024 px vertically (in the Y-axis direction) and 1280 px horizontally (in the X-axis direction). This image is represented by a horizontal gray value profile graph as shown in FIG.
Here, the horizontal gray value profile graph is a graph that sequentially scans pixels of an image and represents the brightness corresponding to the scanning position, with the horizontal axis representing the scanning position and the vertical axis representing the brightness. The pixel scanning direction is, for example, as shown by the arrow in Figure 5, from the top left of the image to the right (positive direction of the X axis), and this is repeated downward (positive direction of the Y axis). All pixels are scanned.

The horizontal gray value profile graph shown in FIG. 6 shows that the brightness of the defect D is brighter than the non-defect portion. However, the brightness of D of the defect is not always brighter than the non-defect part, and may be darker than the non-defect part.
In this way, the brightness of the defect D specified in advance has different characteristics from the brightness of the non-defect portion, and the defect D is distinguished from the non-defect portion and characterized by the brightness (an example of a parameter).
Note that the horizontal gray value profile graph shown in FIG. 6 shows only a part of the scanning position (near the defect D), and the omitted range is shown by a broken line. The same applies to FIGS. 7A to 7C, which will be described later.

The preprocessing performed by the teaching data group generation unit 302 includes the following processing P1 and processing P2.
As shown in FIG. 6, in the process P1, the teaching data group generation unit 302 generates a plurality of images IMG1, IMG2, IMG3, . TD), the brightness of the pixel characterizing the defect D whose position has been identified in advance is determined by a plurality of threshold values (standard values that serve as standards for determining the brightness range) with different predetermined sizes. One example) This is a process of comparing TH1 to TH10 and finding a plurality of threshold values included in the range of pixel brightness that characterizes the defect D as feature values representing the brightness that characterizes the defect D.
In process P2, the teaching data group generation unit 302 converts each image IMG1, IMG2, IMG3,... This is a process of classifying into TD2, TD3, TD4, . . . However, in this case, the teaching data group generation unit 302 copies the original image instead of classifying one image into any one teaching data group TD1, TD2, TD3, TD4, etc. The same number of images as the number of feature values are prepared and classified into teaching data groups TD1, TD2, TD3, TD4, . . . corresponding to each threshold value.
Therefore, through this preprocessing, taught data groups TD1, TD2, TD3, TD4, . . . are generated from the taught data group TD, as shown in FIG.

Next, a specific example of this preprocessing will be described based on images A, B, C, . . . which are examples of images IMG1, IMG2, IMG3 of the inspection object 14, respectively.
Image A corresponding to image IMG1 includes a defect D1 whose position has been specified in advance as shown in FIG. 7A, and the brightness range that characterizes defect D1 is 155 to 205.
Image B corresponding to image IMG2 includes a defect D2 whose position has been specified in advance as shown in FIG. 7B, and the brightness range characterizing the defect D2 is 165 to 215.
The image C corresponding to the image IMG3 includes a defect D3 whose position has been specified in advance as shown in FIG. 7C, and the brightness range characterizing the defect D3 is 155 to 230.

In the process P1 described above, first, a plurality of threshold values (an example of reference values) 20, 40, 60, 80, 100, 120, 140, 160, 180, 200, 220, and 240 are set, and the image A (FIG. 7A (see), three threshold values 160, 180, and 200 included in the brightness range 155 to 205 that characterize the defect D1 are determined as characteristic values.

In the above-mentioned process P2, by copying image A, the same number of images A as the number of feature values, that is, three images A, are prepared, and each image corresponds to a feature value of 160, as shown in FIG. teaching data group TD1 corresponding to feature value 180, teaching data group TD2 corresponding to feature value 200, and teaching data group TD3 corresponding to feature value 200.

The above processing P1 and processing P2 are also performed for the remaining images B, C, and so on.
That is, for image B (see FIG. 7B), two threshold values 180 and 200 included in the brightness range 165 to 215 that characterize the defect D2 are determined as feature values, and image B corresponds to the feature value 180. The data are classified into a teaching data group TD2 and a teaching data group TD3 corresponding to the feature value 200 (see FIG. 8).
For image C (see FIG. 7C), four threshold values 160, 180, 200, and 220 included in the brightness range 155 to 230 that characterize defect D3 are obtained as feature values, and image C has a feature value of 160. The teaching data group TD1 corresponds to the characteristic value 180, the teaching data group TD2 corresponds to the characteristic value 180, the teaching data group TD3 corresponds to the characteristic value 200, and the teaching data group TD4 corresponds to the characteristic value 220 (see FIG. 8).
Furthermore, for the remaining images (images other than images A, B, and C) included in the teaching data group TD, corresponding teaching data groups TD1, TD2, TD3, TD4, . ..., and a teaching data group set TDg (see FIG. 4) is generated.

These teaching data groups TD1, TD2, TD3, TD4, . . . , which are classified according to the brightness that characterizes defects, are machine learning models M1, M2, M3, M4 that determine the quality of the inspected object 14, respectively. It becomes a teaching data group for constructing...

In the preprocessing, images IMG1, IMG2, IMG3, ... (taught data group TD) are classified into teaching data groups TD1, TD2, TD3, TD4, ... based on brightness. The classification is not limited to this, and the classification may be based on parameters other than brightness. Parameters other than brightness include, for example, hue, saturation, and value that make up the HSV color space, and R (Red) and G (Green) expressed in gradations in the RGB color model. and B (Blue). That is, the parameters may be arbitrary as long as they can characterize the defective portions by distinguishing them from non-defective portions.
Further, the pre-processing may include filter processing for making defective parts more prominent than non-defective parts.

Here, the machine learning model construction service 80 shown in FIGS. 1 and 4 is provided as a cloud computing service, and can construct a trained machine learning model based on uploaded teaching data. This machine learning model construction service 80 is, for example, Cloud AutoML Vision provided by Google Cloud Platform (GCP).

The optimal model selection unit (an example of optimal model selection means) 306 (see FIG. 3) evaluates the results of the judgment of the quality of the inspected object 14 based on the plurality of trained machine learning models constructed by the machine learning model construction service 80. , the optimal machine learning model can be selected.

The management unit (an example of management means) 308 can manage the state of the inspection system 40a used by the user. Specifically, the management unit 308 can record operating information regarding the operating state of the imaging unit 460 (see FIG. 1) included in the inspection system 40a. This operating information will be described later.

Note that the server 30 functions as a teaching data group generation means, an optimal model selection means, and a management means by a program executed inside the server 30.

As shown in FIG. 1, the inspection system 40a includes a terminal 440, a PLC (Programmable Logic Controller) 450, and an imaging unit (an example of a determination device) 460 that images the inspection object 14 and determines its quality. . The terminal 440, PLC 450, and imaging unit 460 are connected to each other by wired or wireless communication.

Terminal 440 is managed by a user. The terminal 440 is, for example, a mobile terminal such as a personal computer or a smartphone, and may be a higher-level controller of the PLC 450.
As shown in FIG. 9, the terminal 440 includes a machine learning model receiving section 440a, a control section 440b, and an operating state output section 440c.

The machine learning model receiving unit (an example of a receiving unit) 440a can download each machine learning model constructed by the machine learning model construction service 80. Note that each machine learning model is downloaded using secure communication.

The control unit (an example of a control unit) 440b can control the PLC 450 and the imaging unit 460.

The operating state output unit (an example of operating state output means) 440c can output operating information regarding the operating state of the imaging unit 460. This operation information is, for example, information about the time from when the image capturing section 460 starts capturing an image until it finishes capturing the image. The operating information may be information about the number of images captured by the imaging unit 460.

Note that the terminal 440 functions as a receiving means, a control means, and an operating state output means by a program executed inside the terminal 440.
Furthermore, one terminal 440 is not limited to having all the machine learning model receiving section 440a, control section 440b, and operating state output section 440c, but each section is divided into multiple terminals connected to each other. may exist.
Furthermore, the terminal 440 may include a teaching data creation section 202 instead of the teaching data creation device 20 shown in FIG.

The PLC 450 is a controller that is managed by a user and controls an inspection device 470 that inspects the object to be inspected 14, as shown in FIG.

The imaging unit 460 (see FIG. 9) is managed by the user and can capture an image of the object to be inspected 14. Furthermore, the imaging unit 460 can determine the quality of each imaged inspection object 14 based on a plurality of machine learning models.
The imaging unit 460 is, for example, a camera equipped with a GPU (Graphics Processing Unit). However, the imaging unit 460 may be a mobile terminal with a camera such as a smartphone.
The imaging section 460 includes a camera section 460a, a storage section 460b, and a determination section 460c.

The camera unit 460a can capture an image of the object to be inspected 14 and obtain image data.
The storage unit 460b can store a plurality of machine learning models downloaded by the machine learning model receiving unit 440a.
The determining unit 460c determines whether there is a defect in the inspected object 14, that is, whether the inspected object 14 is good or bad, based on the image data of the inspected object 14 captured by the camera unit 460a and the machine learning model stored in the storage unit 460b. can be determined. The determination unit 460c can perform arithmetic processing using a machine learning model at high speed, and is configured by, for example, a GPU.

Next, the operation of the quality determination system 10a (the method for determining quality of the inspected object 14) will be explained based on FIG. 10. The quality determination system 10a operates according to steps S1 to S9 below. Of steps S1 to S9, steps S1 to S7 are operations necessary as a preparatory stage before performing the actual pass/fail judgment, and subsequent steps S8 and S9 are operations for the actual inspection process in the pre-shipment inspection process of parts etc. This is a pass/fail judgment operation.
Note that, if possible, steps S1 to S7 may be performed in a different order or may be performed in parallel.

(Step S1)
The teaching data creation unit 202 (see FIG. 2) of the teaching data creation device 20 (see FIG. 1) creates a teaching data group TD shown in FIG. 4 based on the image data of the inspected object 14 captured by the imaging unit 460. A plurality of image data of the inspection object 14 are created.
The teaching data group TD is a plurality of image data of the inspected object 14 that includes a specified defect and a plurality of image data groups of the inspected object 14 that does not include the specified defect. Note that the types of defects include, for example, scratches, voids, dirt, and foreign matter contamination. However, the defects to be inspected differ depending on the object 14 to be inspected.
The created images of the plurality of inspected objects 14 are stored in a storage means (not shown) and transmitted to the server 30 shown in FIG. 3.
Instead of being created by the teaching data creation unit 202, the teaching data group TD may be created manually based on image data of the inspected object 14 prepared in advance.

(Step S2)
As shown in FIG. 4, the teaching data group generation unit 302 of the server 30 preprocesses the plurality of images (teaching data group TD) of the object to be inspected 14 generated by the teaching data creation device 20, and A plurality of classified teaching data groups TD1, TD2, TD3, TD4, . . . are generated. In the preprocessing, for example, a plurality of brightness thresholds TH1 to TH10 (see FIG. 6) are used.
In addition, each threshold value may be calculated|required by calculating|requiring the maximum value of the brightness of the defect D, and dividing this maximum value.
Thereafter, the server 30 uploads the preprocessed teaching data groups TD1, TD2, TD3, TD4, .
In this way, since the service provider rather than the user performs the preprocessing of the teaching data that requires know-how, the user can easily install the pass/fail determination system 10a based on the machine learning model.

(Step S3)
Based on the uploaded teaching data group, the machine learning model construction service 80 constructs machine learning models M1, M2, M3, M4, . . . , respectively.
The accuracy of the constructed and trained machine learning models M1, M2, M3, M4, . . . is verified by the optimal model selection unit 306 (see FIG. 3) of the server 30, respectively.
Note that if the accuracy is worse than a predetermined standard, the process returns to the previous step S2, and the teaching data group generation unit 302 performs preprocessing by further applying filter processing.

(Step S4)
In response to a user's operation, the machine learning model receiving unit 440a (see FIG. 9) of the terminal 440 downloads each trained machine learning model constructed by the machine learning model construction service 80. Each downloaded trained machine learning model is transmitted to the imaging unit 460 via the terminal 440.

(Step S5)
The storage unit 460b of the imaging unit 460 stores each machine learning model downloaded by the machine learning model receiving unit 440a.

(Step S6)
The control unit 440b of the terminal 440 controls the PLC 450 and the imaging unit 460, and tests the quality of the inspected object 14 manufactured by the inspection device 470 (see FIG. 1).
Specifically, as a preparatory step for making a quality determination before shipping the inspection object 14, the camera section 460a (see FIG. 9) of the imaging section 460 captures an image of the inspection object 14, and the determination section 460c captures an image of the inspection object 14. Based on the image and the plurality of machine learning models stored in the storage unit 460b, each object to be inspected 14 is inspected to determine whether or not there is a defect. If so, the product is determined to be defective.

(Step S7)
Each experimental determination result in the previous step S6 is transmitted from the imaging unit 60 to the server 30 via the terminal 440.
The optimal model selection unit 306 included in the server 30 shown in FIG. 3 evaluates the quality of each machine learning model based on each experimental determination result, and selects the optimal machine learning model (hereinafter referred to as "optimal model"). Select.
Information on the selected optimal model is transmitted from the server 30 to the terminal 440.

As described above, the steps up to step S7 are operations necessary as a preparatory stage before actual quality determination is performed.
The next step S8 and subsequent steps are the actual quality determination operations in the inspection process before shipping parts and the like.

(Step S8)
The control unit 440b (see FIG. 9) of the terminal 440 controls the PLC 450 and the imaging unit 460, and the imaging unit 60 is transported on the conveyor of the inspection device 470 (see FIG. 1) based on the selected optimal model. The quality of the incoming inspection object 14 is determined.
Specifically, the camera unit 460a (see FIG. 9) of the imaging unit 460 captures an image of the inspection object 14, and the determination unit 460c uses the captured image and the optimal model stored in the storage unit 460b to , the object to be inspected 14 is inspected for the presence of defects, and if there are no defects, it is determined to be a good product, and if there is a defect, it is determined to be a defective product.

(Step S9)
The operating state output unit 440c (see FIG. 9) of the terminal 440 transmits operating information regarding the operating state of the imaging unit 460 to the server 30. The transmitted operating information is stored in a storage unit (not shown) of the server 30, and the operating state of the quality determination system 10a is centrally managed by the server 30.

In this way, the quality determination system 10a according to the present embodiment determines the quality of the inspected object 14 using the optimal machine learning model selected from the plurality of constructed machine learning models. Highly accurate judgment results can be obtained.
Note that the quality determination system 10a can determine the quality of the inspected object 14 other than its appearance depending on the type of camera unit 460a. For example, if the camera unit 460a is an infrared camera, it is also possible to determine whether the internal state of the inspected object 14 is good or bad.

[Second embodiment]
Next, a quality determination system 10b according to a second embodiment of the present invention will be described. Components having the same functions as the quality determination system 10a according to the first embodiment may be denoted by the same reference numerals and detailed description thereof may be omitted.
As shown in FIG. 11, the quality determination system 10b includes a teaching data creation device 20, a server 30, and an inspection system 40b.
The inspection system 40b includes a terminal 442, a PLC (Programmable Logic Controller) 450, and an imaging unit 462 that captures an image of the inspection object 14.

The terminal 442 (an example of a determination device) is, for example, a personal computer, a smartphone, an MR device for realizing MR (Mixed Reality), or an AR device for realizing AR (Augmented Reality).
As shown in FIG. 12, the terminal 442 includes a machine learning model receiving section (an example of a receiving means) 440a, a control section (an example of a control means) 440b, an operating state output section (an example of an operating state output means) 440c, and a storage section. It has a determination unit (an example of a storage unit) 460b and a determination unit (an example of a determination unit) 460c, and can determine the quality of each inspected object based on a plurality of machine learning models.
Note that the terminal 442 functions as a reception means, a control means, an operating state output means, a storage means, and a determination means by a program executed inside the terminal 442.

The imaging section 462 includes a camera section 460a.

That is, in the present quality/failure determination system 10b, the terminal 442 has the storage unit 460b and determination unit 460c that the imaging unit 460 according to the first embodiment had.
Note that even if the terminal 442 has a part of the machine learning model receiving section 440a, the control section 440b, the operating state output section 440c, the storage section 460b, and the determination section 460c, and the PLC 450 has the others. good. That is, the inspection system 40b as a whole only needs to have a machine learning model receiving section 440a, a control section 440b, an operating state output section 440c, a storage section 460b, and a determination section 460c.
Furthermore, the machine learning model receiving section 440a, the storage section 460b, and the determining section 460c may be included in the server 30 shown in FIG. 11 instead of the inspection system 40b.

Comparing the quality determination system 10b according to the present embodiment and the quality determination system 10a according to the first embodiment, as shown in FIGS. 9 and 12, the storage unit provided in the imaging unit 460 The only difference is that 460b and determination unit 460c are provided in terminal 442.
Therefore, since the operation of the quality determination system 10b is substantially the same as the operation (steps S1 to S9) of the quality determination system 10a, a description thereof will be omitted.

[Third embodiment]
Next, a predictive maintenance system (an example of a quality determination system) 10c according to a third embodiment of the present invention will be described. Components having the same functions as the quality determination system 10b according to the second embodiment may be denoted by the same reference numerals and detailed description thereof may be omitted.

The predictive maintenance system 10c according to the present embodiment can determine whether the operating state of the monitored object is good or bad by measuring the sound emitted by the monitored object, and can be applied to predictive maintenance that predicts defects in the monitored object.
The monitoring target is, for example, a mechanical device, specifically a press machine. However, the monitoring target is not limited to press machines as long as it is a device or device whose malfunction can be predicted by sound.

As shown in FIG. 13, the predictive maintenance system includes a sound source visualization device (an example of a visualization device) 500, a teaching data creation device 20, a server 33, and a terminal 443.

The sound source visualization device 500 includes a camera (not shown) that images the monitored object 600 and a plurality of microphones 502 for identifying the sound source generated from the monitored object 600, and includes a sound intensity distribution in an actual image around the sound source. It is possible to output a sound source visualization image that visualizes the sound source by superimposing them in real time. This sound intensity distribution is expressed as heat map-like visualized information using different colors depending on the magnitude of the sound pressure.
Note that the sound source visualization device 500 is sometimes called an acoustic camera.

As shown in FIG. 2, the teaching data creation device 20 has a teaching data creation unit 202, and can take in a sound source visualization image and create an image as teaching data.

The server 33 is managed by a service provider, and includes a teaching data group generation section 302, a determination section 334, an optimal model selection section 306, and a management section 308, as shown in FIG.
As shown in FIG. 4, the teaching data group generation unit (an example of teaching data group generation means) 302 preprocesses the teaching data group TD created by the teaching data creation device 20, and generates a set of the preprocessed teaching data group. TDg, that is, teaching data groups TD1, TD2, TD3, TD4, . . . are created.

The determination unit (an example of determination means) 334 hypothetically determines whether the operating state of the monitored object 600 is good or bad based on the teaching data group TD and a plurality of trained machine learning models constructed by the machine learning model construction service 80. It can be judged accurately.

The optimal model selection unit (an example of optimal model selection means) 306 can evaluate the determination result by the determination unit 334 and select the optimal machine learning model.

The management unit (an example of management means) 308 can manage the state of the terminal 443 or the sound source visualization device 500. Specifically, the management unit 308 can record operating information regarding the operating state of the terminal 443 or the sound source visualization device 500.
Note that the server 33 functions as a teaching data group generation means, a determination means, an optimal model selection means, and a management means by a program executed inside the server 33.

A terminal (an example of a determination device) 443 is connected to a sound source visualization device 500, as shown in FIG. The terminal 443 includes a machine learning model receiving section 440a, a control section 443b, an operating state output section 443c, a storage section 443d, and a determining section 443e, and can determine the quality of the monitored object 600 based on the optimal model.

The machine learning model receiving unit (an example of a receiving unit) 440a can receive the optimal model selected by the optimal model selection unit 306 from the server 33. Note that the optimal model is received through secure communication.

The control unit (an example of a control means) 443b can control the PLC 450 and the sound source visualization device 500.

The operating state output unit (an example of operating state output means) 443c can output operating information regarding the operating state of the terminal 443 or the sound source visualization device 500. This operation information is, for example, information about the time from when the sound source visualization device 500 starts capturing an image until it finishes capturing the image. The operation information may be information on the number of images output from the sound source visualization device 500.

The storage unit (an example of a storage unit) 443d can store the optimal model received by the machine learning model receiving unit 400a.

The determination unit (an example of a determination unit) 443e can determine whether the operating state of the monitored target 600 is good or bad based on the plurality of sound source visualization images output by the sound source visualization device 500 and the optimal model stored in the storage unit 443d.
Note that the terminal 443 functions as a reception means, a control means, an operating state output means, a storage means, and a determination means by a program executed inside the terminal 443.

Next, the operation of the predictive maintenance system 10c (method for determining whether the operating state of the monitored object 600 is good or bad) will be described based on FIG. 16. The predictive maintenance system 10c operates according to steps S3-1 to S3-9 below. Of steps S3-1 to S3-9, steps S3-1 to S3-7 are operations as a preparatory stage, and subsequent steps S3-8 and S3-9 are performed to determine the actual operating state of the monitored target 600. This is the operation for determining the quality of the product.
Note that, if possible, steps S3-1 to S3-7 may be performed in a different order or may be performed in parallel.

(Step S3-1)
The teaching data creation unit 202 (see FIG. 2) of the teaching data creation device 20 (see FIG. 13) takes in the sound source visualization image generated by the sound source visualization device 500, uses it as teaching data, and creates the teaching data group TD shown in FIG. 4. do.
The plurality of sound source visualization images (teaching data group TD) are stored in a storage means (not shown) and transmitted to the server 33 shown in FIG. 14.
Instead of being created by the teaching data creation unit 202, the teaching data group TD may be created manually based on a sound source visualization image of the monitored object 600 prepared in advance.

(Step S3-2)
As shown in FIG. 4, the teaching data group generation unit 302 of the server 33 preprocesses the sound source visualization image (teaching data group TD) generated by the teaching data creation device 20, and generates a plurality of images classified according to feature values. Teach data groups TD1, TD2, TD3, TD4, . . . are generated. Note that in the preprocessing, for example, a plurality of brightness thresholds TH1 to TH10 (see FIG. 6) are used.
In addition, each threshold value may be calculated|required by calculating|requiring the maximum value of the brightness of the defect D, and dividing this maximum value.
Thereafter, the server 33 uploads the preprocessed teaching data groups TD1, TD2, TD3, TD4, . . . to the machine learning model construction service 80 by the operation of the service provider or the user.
In this way, the service provider rather than the user performs the preprocessing of the teaching data that requires know-how, so the user can easily introduce a predictive maintenance system using a machine learning model.

(Step S3-3)
Based on the uploaded teaching data group, the machine learning model construction service 80 constructs machine learning models M1, M2, M3, M4, . . . , respectively.
The accuracy of each constructed trained machine learning model is verified by the optimal model selection unit 306 (see FIG. 14) of the server 33.
Note that if the accuracy is poor, the process returns to the previous step S3-2, and the teaching data group generation unit 302 preprocesses the sound source visualization image (teaching data group TD) using a different method, such as by applying a different filter process. .

(Step S3-4)
In response to a user's operation, the server 33 (see FIG. 14) downloads each trained machine learning model constructed by the machine learning model construction service 80. Each downloaded trained machine learning model is stored in a storage unit (not shown).

(Step S3-5)
The determining unit 334 of the server 33 inputs the teaching data group TD to each machine learning model stored in a storage unit (not shown), and virtually determines whether the operating state of the monitored object 600 is good or bad.

(Step S3-6)
The optimal model selection unit 306 evaluates the determination result of the operating state of the monitored object 600 by the determination unit 334 in the previous step S3-6, and selects the optimal model from among the plurality of machine learning models.

(Step S3-7)
The optimal model selected by the optimal model selection unit 306 is transmitted from the server 33 to the terminal 442.
The transmitted optimal model is received by the machine learning model receiving section 440a (see FIG. 15) and stored in the storage section 443d.

(Step S3-8)
This step S3-8 is a step of monitoring the monitored object 600.
The control unit 443b of the terminal 443 controls the PLC 450 to operate the monitored target 600. On the other hand, the sound source visualization device 500 measures the sound generated from the monitored object 600 and outputs a sound source visualization image at a predetermined period.
The terminal 442 determines whether the operating state of the monitored object 600 is good or bad based on the outputted sound source visualization image and the optimal model stored in the storage unit 443d .

(Step S3-9)
The operating state output unit 443c of the terminal 440 transmits operating information regarding the operating state of the sound source visualization device 500 to the server 33. The transmitted operating information is stored in a storage unit (not shown) of the server 33, and the operating state of the predictive maintenance system 10c is centrally managed by the server 33.

As described above, according to the predictive maintenance system 10c according to the present embodiment, the operational status of the monitored object 600 is evaluated using the optimal machine learning model selected from the plurality of constructed machine learning models. This makes it possible to perform predictive maintenance with higher accuracy.
Note that instead of the sound source visualization system, an arbitrary system that has a detector for measuring a physical quantity that changes due to a sign of a malfunction occurring in the monitored object 600 and can generate a plurality of visualized images that visualize the physical quantity is used. A visualization device may also be used.

[Fourth embodiment]
Next, a predictive maintenance system (an example of a quality determination system) 10d according to a fourth embodiment of the present invention will be described. Components having the same functions as those of the predictive maintenance system 10c (see FIG. 13) according to the third embodiment may be given the same reference numerals and detailed explanations may be omitted.

The predictive maintenance system according to the present embodiment can determine whether the operating state of the monitored object is good or bad by measuring vibrations emitted by the monitored object, and can be applied to predictive maintenance that predicts defects in the monitored object.
The monitoring target is, for example, a mechanical device, specifically a press machine or a conveyance device. However, the monitoring target may be any device or equipment that can predict failures due to vibrations.

As shown in FIG. 17, the predictive maintenance system 10d includes a vibration visualization device (an example of a visualization device) 700, a teaching data creation device 20, a server 33, and a terminal 443 (an example of a determination device).
The vibration visualization device 700 has a plurality of vibration sensors 702 for detecting vibrations generated from the monitored object 600, and can output a plurality of vibration visualization images in which vibrations detected by each vibration sensor 702 are visualized. This vibration visualization image is, for example, an image expressed as visualized information using different colors depending on at least one of the magnitude and frequency of vibration.

Here, in the predictive maintenance system 10d, the vibration visualization device 700 and the vibration visualization image correspond to the sound source visualization device 500 and the sound source visualization image in the third embodiment, respectively.

By using such a predictive maintenance system 10d and performing the above-mentioned operation steps S3-1 to S3-9 (see FIG. 16), the determination unit 443e (see FIG. 15) of the terminal 443 can determine the If a problem occurs in 600, it can be determined that there is an abnormality.

In this way, according to the predictive maintenance system 10d according to the present embodiment, the quality of the monitored object 600 is determined using the optimal machine learning model selected from the plurality of constructed machine learning models. , enabling predictive maintenance with higher accuracy.
Note that instead of the vibration visualization system, an arbitrary system that has a detector for measuring a physical quantity that changes due to a sign of a malfunction occurring in the monitored object 600 and can generate a plurality of visualized images that visualize the physical quantity is used. A visualization device may also be used.

Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and any changes in conditions that do not depart from the gist are within the scope of the present invention.

10a, 10b Quality determination system 10c, 10d Predictive maintenance system 14 Inspected object 20 Teaching data creation device 30 Server 40a, 40b Inspection system 450 PLC
80 Machine learning model construction service 202 Teaching data creation section 302 Teaching data group generation section 306 Optimal model selection section 308 Management section 334 Judgment section 440 Terminal 440a Machine learning model reception section 440b Control section 440c Operating state output section 442, 443 Terminal 443b Control Section 443c Operating state output section 443d Storage section 443e Judgment section 460 Imaging section 460a Camera section 460b Storage section 460c Judgment section 462 Imaging section 470 Inspection device 500 Sound source visualization device 502 Microphone 600 Monitored object 700 Vibration visualization device 702 Vibration sensor N Internet

Claims (11)

a visualization device that has a detector for measuring a physical quantity that changes due to a sign of a malfunction occurring in a monitored object, and that generates a plurality of visualized images visualizing the physical quantity;
a teaching data group generation unit that classifies the visualized image including a plurality of pre-obtained defects according to parameters characterizing the defects, and generates a plurality of classified teaching data groups;
a storage unit that stores the plurality of machine learning models each constructed based on the plurality of teaching data groups;
an optimal model selection unit that selects an optimal model that is an optimal machine learning model from among the plurality of machine learning models;
and a determination unit that determines whether the operating state of the monitored object is good or bad based on the plurality of visualized images generated by the visualization device and the optimal model selected by the optimal model selection unit, respectively. Judgment system. a sound source visualization device that has a microphone for identifying a sound source generated from a monitored target and that generates a plurality of sound source visualization images visualizing the sound source;
a teaching data group generation unit that classifies the sound source visualization image including a plurality of pre-obtained defects according to parameters characterizing the defects, and generates a plurality of classified teaching data groups;
a storage unit that stores the plurality of machine learning models each constructed based on the plurality of teaching data groups;
an optimal model selection unit that selects an optimal model that is an optimal machine learning model from among the plurality of machine learning models;
a determination unit that determines whether the operating state of the monitored object is good or bad based on the plurality of sound source visualization images generated by the sound source visualization device and the optimal model selected by the optimal model selection unit, respectively. pass/fail judgment system. a vibration visualization device that has a vibration sensor for detecting vibrations generated from a monitored object and generates a plurality of vibration visualization images visualizing the vibrations;
a teaching data group generation unit that classifies the vibration visualization image including a plurality of pre-obtained defects according to parameters characterizing the defects, and generates a plurality of classified teaching data groups;
a storage unit that stores the plurality of machine learning models each constructed based on the plurality of teaching data groups;
an optimal model selection unit that selects an optimal model that is an optimal machine learning model from among the plurality of machine learning models;
a determination unit that determines whether the operating state of the monitored object is good or bad based on the plurality of vibration visualization images generated by the vibration visualization device and the optimal model selected by the optimal model selection unit, respectively. pass/fail judgment system. A quality determination method using the quality determination system according to claim 9,
the visualization device generating the visualized image;
the teaching data group generation unit creating the plurality of teaching data groups;
the storage unit storing the plurality of machine learning models;
a step in which the optimal model selection unit selects an optimal machine learning model from among the plurality of machine learning models;
A quality determination method, comprising: the determination unit determining whether the operating state of the monitored object is good or bad using the machine learning model selected by the optimal model selection unit. A plurality of visualized images that visualize physical quantities that have changed due to signs of a malfunction occurring in the monitored target are respectively classified into the monitored target according to a plurality of teaching data groups classified according to parameters characterizing the malfunction. A network is connected to a cloud computing service that constructs a plurality of machine learning models for determining whether the operating state of the monitored object is good or bad, and a determination device that determines whether the monitored object is good or bad based on the plurality of machine learning models. A server connected via
a teaching data group generation unit that generates the plurality of teaching data groups;
An optimal model selection unit that selects an optimal machine learning model from the plurality of machine learning models. A plurality of visualized images that visualize physical quantities that have changed due to signs of a malfunction occurring in the monitored target are respectively classified into the monitored target according to a plurality of teaching data groups classified according to parameters characterizing the malfunction. A network is connected to a cloud computing service that constructs a plurality of machine learning models for determining whether the operating state of the monitored object is good or bad, and a determination device that determines whether the monitored object is good or bad based on the plurality of machine learning models. A computer connected via
teaching data group generation means for generating the plurality of teaching data groups;
A program that functions as an optimal model selection unit that evaluates the pass/fail results determined by the determination device and selects an optimal machine learning model from among the plurality of machine learning models. A plurality of visualized images that visualize physical quantities that have changed due to signs of a malfunction occurring in the monitored target are respectively classified into the monitored target according to a plurality of teaching data groups classified according to parameters characterizing the malfunction. A computer connected to a cloud computing service that constructs a plurality of machine learning models for determining the quality of the operating state of the object, and an imaging unit that captures an image of the object to be inspected.
storage means for storing the plurality of machine learning models constructed by the cloud computing service;
A program that functions as a determination unit that determines the quality of the inspection object imaged by the imaging unit, based on the plurality of machine learning models stored in the storage unit. a teaching data group generation unit that classifies images including defects of a plurality of objects to be inspected according to parameters characterizing the defects, and generates a plurality of classified teaching data groups;
a storage unit that stores the plurality of machine learning models each constructed based on the plurality of teaching data groups;
a camera unit that captures an image of the object to be inspected;
a determination unit that determines the quality of the inspected object imaged by the camera unit, based on the plurality of machine learning models stored in the storage unit;
An optimal model selection section that evaluates the result of the determination by the determination section and selects an optimal machine learning model from among the plurality of machine learning models. A plurality of machine learning models for determining the quality of each inspection object based on a plurality of teaching data groups in which images containing defects of the plurality of inspection objects are classified according to the brightness characterizing the defects. A server connected via a network to a cloud computing service that builds a cloud computing service, and a determination device that determines the quality of the inspection object based on the plurality of machine learning models, the server comprising:
a teaching data group generation unit that generates the plurality of teaching data groups;
An optimal model selection unit that evaluates the pass/fail result determined by the determination device and selects an optimal machine learning model from among the plurality of machine learning models. A plurality of machine learning models for determining the quality of each inspection object based on a plurality of teaching data groups in which images containing defects of the plurality of inspection objects are classified according to the brightness characterizing the defects. a computer connected via a network to a cloud computing service that constructs
teaching data group generation means for generating the plurality of teaching data groups;
A program that functions as an optimal model selection unit that evaluates the pass/fail results determined by the determination device and selects an optimal machine learning model from among the plurality of machine learning models. a server having a teaching data group generation unit that generates a plurality of teaching data groups in which images including defects of a plurality of objects to be inspected are classified according to the brightness characterizing the defects; A computer connected to a cloud computing service that constructs a plurality of machine learning models that each determine the quality of the inspected object, and an imaging unit that captures an image of the inspected object,
storage means for storing the plurality of machine learning models constructed by the cloud computing service;
A program that functions as a determination unit that determines the quality of the inspection object imaged by the imaging unit, based on the plurality of machine learning models stored in the storage unit.