JP2021189039A - Industrial inspection system - Google Patents

Abstract

To provide an industrial inspection system that can improve a determination performance by making the device resistant to an impact of noise in a quality determination of an inspection object.SOLUTION: An industrial inspection system 1 comprises: an output value acquisition unit 33 that acquires an expected value and dispersion as an output of a learned model using a variation auto encoder (VAE) when inputting inspection image data on an inspection object; a differential value computation unit 34 that computes a differential between the inspection image data on the inspection object and the expected value for each pixel; an individual pixel evaluation value computation unit 35 that multiplies a correlation value of the differential with a weighting coefficient represented by a correlation value of the dispersion for each pixel, and thereby computes an individual pixel evaluation value g1; an entire pixel evaluation value computation unit 36 that computes an entire pixel evaluation value on the basis of the individual pixel evaluation value g1 for each pixel; and a determination unit 37 that determines a quality of the inspection object on the basis of the entire pixel evaluation value.SELECTED DRAWING: Figure 3

Description

The present invention relates to an industrial inspection system.

Generally, equipment used in industry requires visual inspection, operation inspection, and the like in order to detect an abnormality. These inspections are performed not only when the factory-produced equipment is shipped, but also when the equipment is used. For example, devices used in industry include finished products such as industrial machines, automobiles, and aircraft, or parts constituting these finished products.

As a method for detecting anomalies, for example, as described in Patent Document 1, it is known to use an autoencoder as one of machine learning. The autoencoder is composed of an encoder and a decoder. In the learning phase, when the image data to be inspected is input to the encoder, machine learning is performed so that the output of the decoder matches the image data to be inspected. Then, in the inspection phase, when the image data to be inspected is input to the encoder, the output image data of the decoder is acquired, and the determination value is obtained from the difference between the input image data and the output image data. Judge an abnormality.

Japanese Unexamined Patent Publication No. 2019-503530

However, when the input image data contains noise, the autoencoder has low reproduction performance, so the judgment value obtained from the difference between the input image data and the output image data becomes large, and it can be judged as abnormal. There is sex. That is, although the inspection target of the input image data is normally normal, there is a possibility that it is erroneously determined that the inspection target of the input image data is the above due to noise.

An object of the present invention is to provide an industrial inspection system capable of improving the determination performance by making it less susceptible to noise in the quality determination of an inspection target.

The industrial inspection system includes an inspection image data acquisition unit that acquires inspection image data obtained by imaging an inspection target, which is a device used in industry, or inspection image data obtained by converting data caused by the operation of the inspection target, and a predetermined probability. A good product using the learning model that uses a variable auto encoder (VAE) with a distribution as a training model, the inspection image data as the input of the encoder, and the expected value and dispersion as the output of the decoder as the parameters of the distribution of the reconstructed data. When the trained model storage unit that stores the trained model that has undergone machine learning using the inspection image data as training data and the inspection image data to be inspected are input, the trained model is output as the output. An output value acquisition unit that acquires the expected value and the dispersion, and a difference value calculation unit that calculates the difference between the inspection image data to be inspected and the expected value acquired by the output value acquisition unit for each pixel. Individual pixel evaluation by multiplying the correlation value of the difference output by the difference value calculation unit for each pixel by the weighting coefficient represented by the correlation value of the dispersion acquired by the output value acquisition unit. An individual pixel evaluation value calculation unit that calculates a value, an all-pixel evaluation value calculation unit that calculates an all-pixel evaluation value based on the individual pixel evaluation value for each pixel, and an inspection target based on the all-pixel evaluation value. It is provided with a determination unit for determining pass / fail.

In the above industrial inspection system, a variational autoencoder is used as a machine learning method instead of an autoencoder. Variational autoencoders are constrained to have a predetermined probability distribution. Therefore, by using the variational autoencoder, the expected value and the variance as the parameters of the distribution of the reconstructed data can be obtained as the output of the decoder.

Then, in the industrial inspection system, the evaluation value of all pixels is used as the determination value in the determination unit. The all-pixel evaluation value is calculated based on the individual pixel evaluation value. The individual pixel evaluation value is calculated using the inspection image data to be inspected, the expected value, and the variance. Specifically, first, the difference value calculation unit calculates the difference between the inspection image data to be inspected and the expected value, and the individual pixel evaluation value calculation unit represents the difference correlation value by the variance correlation value. The individual pixel evaluation value is calculated by multiplying by the weighting coefficient.

Here, when the expected value is close to the inspection image data, it indicates a state in which the reproduction performance is high, that is, a state in which there is a high possibility that noise is not included. On the other hand, when the expected value is far from the inspection image data, it indicates a state in which the reproduction performance is low, that is, a state in which noise may be included. The weighting coefficient is represented by the correlation value of the variance. The variance has a value corresponding to the reproduction performance.

The individual pixel evaluation value is obtained by multiplying the correlation value of the difference by the weighting coefficient, as described above. Therefore, the influence of the correlation value of the difference becomes relatively large when the individual pixel evaluation value does not include noise, and the influence of the correlation value of the difference when there is a possibility that noise is included. Is relatively small.

Since the all-pixel evaluation value is obtained from the individual pixel evaluation value obtained as described above, the effect of noise is small as a result. That is, it is possible to reduce the influence of noise in the quality determination of the inspection target, and the determination performance can be improved.

It is a figure which shows the learning model of the first example. It is a figure which shows the learning model of the 2nd example. It is a figure which shows the structure of the industrial inspection system of 1st example. It is a figure which shows the structure of the industrial inspection system of the 2nd example. It is a figure which shows the structure of the steering apparatus as an example of an inspection target. It is a functional block diagram which shows the structural example of the inspection image data acquisition part. It is a figure which showed an example of the waveform data which is the primary data of a time series. It is a figure which shows the process which the secondary data is generated by the secondary data generation part. It is a figure which showed an example of spectrogram data which is a secondary data. It is a figure which showed an example of time-amplitude data. It is a figure which showed an example of the 2nd frequency component-amplitude data. It is a figure which showed an example of the image data which is a tertiary data.

(1. Outline of industrial inspection system)
The industrial inspection system determines the abnormality of the inspection target by setting the equipment used in the industry as the inspection target. For example, devices used in industry include industrial machines such as machine tools, finished products such as automobiles and aircraft, or parts constituting these finished products.

Then, the industrial inspection system can determine the quality of the inspection image data obtained by imaging the inspection target, or can determine the quality of the inspection image data obtained by converting the data caused by the operation of the inspection target.

That is, as an example of the former, it is possible to determine the appearance abnormality in the inspection target for which the appearance inspection is performed. For example, it is possible to determine the presence or absence of scratches on the surface of the product, determine the degree of shape error of the product, and determine the mounting position error of the product.

As an example of the latter, it is possible to determine whether or not abnormal sound or abnormal vibration has occurred by converting sound data or vibration data caused by the operation of the inspection target into inspection image data. For example, it is possible to determine the occurrence of a machining noise abnormality due to a machining abnormality of a machine tool, or determine the occurrence of vibration due to a machining abnormality. In addition, it is possible to determine the occurrence of an abnormal operation sound or the occurrence of vibration due to an abnormal operation in a component of an automobile.

In addition, the industrial inspection system uses a variational auto-encoder (VAE) as a learning model for machine learning. Here, the variational autoencoder is restricted to having a predetermined probability distribution. On the premise of this, the industrial inspection system adopts a configuration in which the output of the decoder of the variational autoencoder is the expected value and the variance as the parameters of the distribution of the reconstructed data. Then, the evaluation value of all pixels as a determination value is calculated using the inspection image data to be inspected and the expected value and the variance which are the outputs of the decoder.

(2. Learning model)
The learning model applied to this example will be described. As mentioned above, in this example, the learning model for machine learning is a variational autoencoder.

(2-1. Learning model M1 of the first example)
The learning model M1 of the first example will be described with reference to FIG. The learning model M1 is a variational autoencoder having a normal distribution as a predetermined probability distribution. The learning model M1 includes an encoder E1 and a decoder D1.

The inspection image data x is input to the encoder E1. Further, the distribution regarding the encoder E1 is expressed by the equation (1). That is, the encoder E1 is represented by a distribution with respect to z conditioned on x, and is represented assuming a normal distribution with _{an expected value (mean) μ φ} and a variance σ _φ ^2.

Further, the distribution regarding the latent space z is expressed by the equation (2). That is, the latent space z is represented on the assumption that the expected value (mean) is 0 and the variance is 1 as a standard normal distribution.

The distribution with respect to the decoder D1 is expressed by the equation (3). That is, the decoder D1 is represented by a distribution with respect to x conditioned on z, assuming a normal distribution of _{expected value (mean) μ θ} and variance σ _θ ^2. In this example, the decoder D1 outputs the _{expected value (mean) μ θ} and the variance σ _θ ² as parameters for the distribution of the reconstructed data.

Then, in the training model M1, machine learning is performed so that the inspection image data of a non-defective product is used as training data and the probability density function f1 of the normal distribution represented by the equation (4) is set as the maximum value.

As a result of machine learning, when the expected value (mean) μ _θ is close to x, it can be said that learning is performed so that the _{variance σ θ} ^{2 becomes small.} Further, when the expected value (average) μ _θ is far from x, it can be said that the learning is performed so that the _{variance σ θ} ^{2 becomes large.} Here, when the expected value (average) μ _θ is close to x, it means that the reproducibility of the reconstructed data is high. On the other hand, when the expected value (average) μ _θ is far from x, it means that the reproducibility of the reconstructed data is low. That is, in the result of machine learning, _{it can be said that the variance σ θ} ² is learning the reciprocal of the reproduction performance of the reconstructed data.

(2-2. Learning model M2 of the second example)
The learning model M2 of the second example will be described with reference to FIG. The learning model M2 is a variational autoencoder having a Laplace distribution as a predetermined probability distribution. The learning model M2 includes an encoder E2 and a decoder D2.

The inspection image data x is input to the encoder E2. The encoder E2 is represented by a distribution with respect to z conditioned on x, assuming a Laplace distribution with _{an expected value (mean) μ φ} and a variance 2b _φ ^2. Further, the latent space z is represented on the assumption that the expected value (mean) is 0 and the variance is 1.

The decoder D2 is represented by a distribution with respect to x conditioned on z, assuming a Laplace distribution with _{an expected value (mean) μ θ} and a variance 2b _θ ^2. In this example, the decoder D2 outputs the _{expected value (average) μ θ} and the variance 2b _θ ² as parameters for the distribution of the reconstructed data.

Then, in the learning model M1, machine learning is performed so that the inspection image data of a non-defective product is used as training data and the probability density function f2 of the Laplace distribution represented by the equation (5) is set as the maximum value. Here, b _θ is a positive value.

As a result of machine learning, when the expected value (mean) μ _θ is close to x, it can be said that learning is performed so that the _{variance 2b θ} ^{2 becomes small.} Further, when the expected value (average) μ _θ is far from x, it can be said that the learning is performed so that the _{variance 2b θ} ^{2 becomes large.} Here, when the expected value (average) μ _θ is close to x, it means that the reproducibility of the reconstructed data is high. On the other hand, when the expected value (average) μ _θ is far from x, it means that the reproducibility of the reconstructed data is low. That is, in the result of machine learning, _{it can be said that the variance 2b θ} ² is learning the reciprocal of the reproduction performance of the reconstructed data.

(3. Configuration of industrial inspection system)
(3-1. Hardware configuration)
The industrial inspection system includes, for example, an arithmetic processing unit including a processor, a storage device, an interface, and the like, an input device that can be connected to the interface of the arithmetic processing unit, and an output device that can be connected to the interface of the arithmetic processing unit. The output device may include, for example, a display device. Further, the arithmetic processing unit, the input device, and the output device may form one unit without going through the interface. Further, a physical server or a cloud server can be applied to a part of the arithmetic processing unit and a part of the storage device.

(3-2. Functional configuration)
(3-2-1. Outline of functional configuration)
The functional configuration of the industrial inspection system will be described. The industrial inspection system applies machine learning and may execute the processing of the learning phase and the processing of the inspection phase in machine learning, or may execute only the processing of the learning phase. , Only the processing of the inspection phase may be executed.

In this example, the industrial inspection system takes the case of executing the processing of the learning phase and the processing of the inspection phase as an example. In this example, the industrial inspection system executes the learning phase processing and the inspection phase processing in one arithmetic processing unit, but the learning phase processing and the inspection phase processing are separately executed. It may be executed by a processing device.

(3-2-2. Functional configuration of the industrial inspection system 1 of the first example)
The functional configuration of the industrial inspection system 1 of the first example will be described with reference to FIG. As shown in FIG. 3, the industrial inspection system 1 includes a detection device 10, a learning processing device 20 that executes processing in the learning phase, and an inspection processing device 30 that executes processing in the inspection phase. The industrial inspection system 1 of the first example applies the learning model M1 of the first example, that is, a variational autoencoder with a normal distribution.

The detection device 10 is a device for acquiring inspection data regarding an inspection target. The detection device 10 is, for example, an image pickup device that images an inspection target, a microphone that acquires sound data due to the operation of the inspection target, a vibration sensor that acquires vibration data due to the operation of the inspection target, and the like.

The learning processing device 20 includes an inspection image data acquisition unit 21, a training data storage unit 22, and a trained model generation unit 23. The inspection image data acquisition unit 21 acquires inspection image data of a non-defective product. Here, the inspection image data acquisition unit 21 acquires the detection data from the detection device 10. When the detection device 10 is an image pickup device, the inspection image data acquisition unit 21 acquires inspection image data of a non-defective product imaged by the image pickup device.

When the detection device 10 is a microphone or a vibration sensor, the inspection image data acquisition unit 21 acquires the non-defective product data acquired by the detection device 10 and converts the non-defective product data into inspection image data to obtain the non-defective product. Acquire (generate) inspection image data. The inspection image data acquisition unit 21 may not perform the data conversion, but another device may perform the data conversion. The training data storage unit 22 stores a large number of non-defective inspection image data acquired by the inspection image data acquisition unit 21 as training data.

The trained model generation unit 23 applies the learning model M1 of the first example shown in FIG. The trained model generation unit 23 performs machine learning on the training model M1 using the inspection image data of good products stored in the training data storage unit 22 as training data. Specifically, the trained model generation unit 23 performs machine learning in the learning model M1 so that the probability density function f1 of the normal distribution represented by the above-mentioned equation (4) is set as the maximum value. Then, the trained model is a model that outputs the expected value (average) μ _θ and the variance σ _θ ² as the parameters of the distribution of the reconstructed data when the inspection image data is input.

The inspection processing device 30 includes a trained model storage unit 31, an inspection image data acquisition unit 32, an output value acquisition unit 33, a difference value calculation unit 34, an individual pixel evaluation value calculation unit 35, an all-pixel evaluation value calculation unit 36, and a determination unit. 37 is provided.

The trained model storage unit 31 stores the trained model generated by the trained model generation unit 23 of the learning processing device 20. The inspection image data acquisition unit 32 acquires the inspection image data of the actual inspection target. The inspection image data acquisition unit 32 performs the same processing as the inspection image data acquisition unit 21 in the learning processing device 20 described above. As shown in FIG. 1, the inspection image data acquisition unit 32 has data x in a plurality of pixels.

The inspection image data acquisition unit 32 has been described as a separate configuration from the inspection image data acquisition unit 21 of the learning processing device 20 in order to explain the functions of the learning processing device 20 and the inspection processing device 30, but both are standardized. May be.

The output value acquisition unit 33 uses the trained model stored in the trained model storage unit 31 to input the inspection image data of the inspection target acquired by the inspection image data acquisition unit 32, and the trained model Get the output value. That is, the output value acquisition unit 33 acquires the expected value (average) μ _θ and the variance σ _θ ² as the parameters of the distribution of the reconstructed data in the trained model. Here, as shown in FIG. 1, the output value acquisition unit 33 has an expected value (average) μ _θ and a variance σ _θ ² for each pixel corresponding to each pixel of the inspection image data.

The difference value calculation unit 34 acquires the inspection image data x for all pixels from the inspection image data acquisition unit 32, and acquires the expected value (average) μ _θ for all pixels from the output value acquisition unit 33. Difference value calculation unit 34, for each pixel, calculating the inspection image data x to be inspected, the expected value obtained by the output value acquiring unit 33 (average) difference between _{μ θ (x-μ θ)} .

_{The individual pixel evaluation value calculation unit 35 multiplies the correlation value of the difference (x−μ θ} ) output by the difference value calculation unit 34 for each pixel by the weighting coefficient represented by the correlation value of the dispersion σ _θ ^2. Thereby, the individual pixel evaluation value g1 is calculated. In this example, the individual pixel evaluation value calculation unit 35 calculates the individual pixel evaluation value g1 according to the equation (6).

That is, the individual pixel evaluation value calculation unit 35 has the difference squared value (x−μ _θ ) ² as the correlation value of the difference and the weighting coefficient (1 / σ _θ ² ) which is the reciprocal of the variance σ _θ ^{2 for each pixel.} ) Is multiplied to calculate the individual pixel evaluation value g1. From the equation (6), the individual pixel evaluation value g1 is a value obtained by weighting the difference squared value according to the difference in the reproduction performance in each pixel.

Here, in the learning model M1, since the machine learning is performed so that the probability density function shown in the equation (4) becomes the maximum value, when the reproduction performance is high, that is, when μ _θ becomes a value close to x. Is learned so that the variance σ _θ ^{2 becomes small.} That is, when the reproduction performance is high, the weighting coefficient (1 / σ _θ ² ) becomes a large value. Therefore, when the reproduction performance is high, the individual pixel evaluation value g1 is a value that is greatly affected by the values of _{x and μ θ.} For example, when the inspection image data does not contain noise, the reproduction performance is high. That is, when noise is not included, the individual pixel evaluation value g1 in the pixel is a value greatly influenced by the values of _{x and μ θ.}

On the other hand, when the reproduction performance is low, that is, when μ _θ is a value far from x, the variance σ _θ ² is learned to be large. That is, when the reproduction performance is low, the weighting coefficient (1 / σ _θ ² ) becomes a small value. Therefore, when the reproduction performance is low, the individual pixel evaluation value g1 is a value in which the influence of the values of _{x and μ θ is reduced.} For example, if the inspection image data contains noise, the reproduction performance will be low. That is, when noise is included, the individual pixel evaluation value g1 in the pixel is a value in which the influence of the values of _{x and μ θ is reduced.}

The all-pixel evaluation value calculation unit 36 calculates the all-pixel evaluation value based on the individual pixel evaluation value g1 for each pixel. For example, the all-pixel evaluation value calculation unit 36 calculates the total value of the individual pixel evaluation values g1 in each pixel as the all-pixel evaluation value.

The determination unit 37 determines the quality of the inspection target based on the evaluation values of all pixels. For example, when the evaluation value of all pixels is larger than a preset threshold value, the determination unit 37 determines that the inspection target is abnormal, and when it is smaller than the threshold value, the determination unit 37 determines that the inspection target is normal. The determination unit 37 is not limited to the two-stage pass / fail determination, and may determine the degree of pass / fail in three or more stages.

(3-2-3. Functional configuration of the industrial inspection system 2 of the second example)
The functional configuration of the industrial inspection system 2 of the second example will be described with reference to FIG. As shown in FIG. 4, the industrial inspection system 2 includes a detection device 10, a learning processing device 40 that executes processing in the learning phase, and an inspection processing device 50 that executes processing in the inspection phase. The industrial inspection system 2 of the second example applies the learning model M2 of the second example, that is, a variational autoencoder of the Laplace distribution. Further, in the industrial inspection system 2 of the second example, the same reference numerals are given to the configurations substantially the same as those of the industrial inspection system 1 of the first example, and the description thereof will be omitted.

The learning processing device 40 includes an inspection image data acquisition unit 21, a training data storage unit 22, and a trained model generation unit 43. The trained model generation unit 43 applies the training model M2 of the second example shown in FIG. The trained model generation unit 43 performs machine learning on the training model M2 using the inspection image data of good products stored in the training data storage unit 22 as training data. Specifically, the trained model generation unit 43 performs machine learning in the learning model M2 so as to maximize the probability density function f2 of the Laplace distribution represented by the above-mentioned equation (5). Then, the trained model is a model that outputs the expected value (average) μ _θ and the variance 2b _θ ² as the parameters of the distribution of the reconstructed data when the inspection image data is input.

The inspection processing device 50 includes a trained model storage unit 51, an inspection image data acquisition unit 32, an output value acquisition unit 53, a difference value calculation unit 54, an individual pixel evaluation value calculation unit 55, an all-pixel evaluation value calculation unit 36, and a determination unit. 37 is provided.

The trained model storage unit 51 stores the trained model generated by the trained model generation unit 43 of the learning processing device 40.

The output value acquisition unit 53 uses the trained model stored in the trained model storage unit 51 to input the inspection image data of the inspection target acquired by the inspection image data acquisition unit 32, and the trained model Get the output value. That is, the output value acquisition unit 53 acquires the expected value (average) μ _θ and the variance 2b _θ ² as the parameters of the distribution of the reconstructed data in the trained model. Here, as shown in FIG. 2, the output value acquisition unit 53 has an expected value (average) μ _θ and a variance 2b _θ ² for each pixel corresponding to each pixel of the inspection image data.

The difference value calculation unit 54 acquires the inspection image data x for all pixels from the inspection image data acquisition unit 32, and acquires the expected value (average) μ _θ for all pixels from the output value acquisition unit 53. Difference value calculation unit 54 calculates for each pixel, and inspection image data x to be inspected, the expected value obtained by the output value acquiring unit 53 (average) difference between _{μ θ (x-μ θ)} .

Individual pixel evaluation value calculating unit 55, for each pixel, the correlation value of the output difference by the difference value calculation unit 54 (x-μ _θ), is multiplied by a weighting coefficient expressed by the correlation value of the dispersion 2b _theta ² As a result, the individual pixel evaluation value g2 is calculated. In this example, the individual pixel evaluation value calculation unit 55 calculates the individual pixel evaluation value g2 according to the equation (7).

That is, the individual pixel evaluation value calculation unit 55 has a weighting coefficient (1 / b _θ ) which is the reciprocal of the correlation value b _{θ of the dispersion in the absolute difference value | x−μ θ | as the correlation value of the difference for each pixel.} The individual pixel evaluation value g2 is calculated by multiplying by. From the equation (7), the individual pixel evaluation value g2 is a value obtained by weighting the difference absolute value according to the difference in the reproduction performance in each pixel.

Here, in the learning model M2, since the machine learning is performed so that the probability density function shown in the equation (5) becomes the maximum value, when the reproduction performance is high, that is, when μ _θ becomes a value close to x. Is learned so that the variance 2b ^{2 becomes smaller.} That is, when the reproduction performance is high, the weighting coefficient (1 / b _θ ) becomes a large value. Therefore, when the reproduction performance is high, the individual pixel evaluation value g2 is a value that is greatly affected by the values of _{x and μ θ.} For example, when the inspection image data does not contain noise, the reproduction performance is high. That is, when noise is not included, the individual pixel evaluation value g2 in the pixel is a value greatly affected by the values of _{x and μ θ.}

On the other hand, when the reproduction performance is low, that is, when μ _θ is a value far from x, the variance 2b _θ ² is learned to be large. That is, when the reproduction performance is low, the weighting coefficient (1 / b _θ ) becomes a small value. Therefore, when the reproduction performance is low, the individual pixel evaluation value g2 is a value in which the influence of the values of _{x and μ θ is reduced.} For example, if the inspection image data contains noise, the reproduction performance will be low. That is, when noise is included, the individual pixel evaluation value g2 in the pixel is a value in which the influence of the values of _{x and μ θ is reduced.}

The all-pixel evaluation value calculation unit 36 calculates the all-pixel evaluation value based on the individual pixel evaluation value g2 for each pixel. For example, the all-pixel evaluation value calculation unit 36 calculates the total value of the individual pixel evaluation values g2 in each pixel as the all-pixel evaluation value. The determination unit 37 determines the quality of the inspection target based on the evaluation values of all pixels. For example, when the evaluation value of all pixels is larger than a preset threshold value, the determination unit 37 determines that the inspection target is abnormal, and when it is smaller than the threshold value, the determination unit 37 determines that the inspection target is normal.

(3-2-4. Effect)
In the above-mentioned industrial inspection systems 1 and 2, a variational autoencoder is used as a machine learning method instead of an autoencoder. As described above, the variational autoencoder is restricted to having a predetermined probability distribution, for example, a normal distribution or a Laplace distribution. Therefore, by using the variational autoencoder, the expected value (average) μ _θ and the variances σ _θ ² and 2b _θ ² as the parameters of the distribution of the reconstructed data can be obtained as the output of the decoder.

Then, in the industrial inspection systems 1 and 2, all pixel evaluation values are used as the determination values in the determination unit 37. The all-pixel evaluation value is calculated based on the individual pixel evaluation values g1 and g2. The individual pixel evaluation values g1 and g2 are calculated using the inspection image data to be inspected, the expected value (average) μ _θ , and the variance σ _θ ² and 2b _θ ^2. Specifically, first, the difference value computing unit 34, 54 is the expected value and the test image data x to be inspected (average) calculates the difference (x-mu _theta) with mu _theta, individual pixel evaluation value calculating unit 35 , 55, the correlation value of the difference ^{_{(x-μ θ) 2,}} | x-μ θ | a, as a weighting coefficient represented by the correlation values of dispersion (1 / σ θ ^_2), a (1 / b _θ) By multiplying, the individual pixel evaluation values g1 and g2 are calculated.

Here, when the expected value (average) μ _θ is close to the inspection image data x, it indicates a state in which the reproduction performance is high, that is, a state in which there is a high possibility that noise is not included. On the other hand, when the expected value (average) μ _θ is far from the inspection image data x, it indicates a state in which the reproduction performance is low, that is, a state in which noise may be included. The weighting coefficients (1 / σ _θ ² ) and (1 / b _θ ) are represented by the correlation values of the variances σ _θ ² and 2b _θ ^2. Here, the variances σ _θ ² and 2b _θ ² have values corresponding to the reproduction performance.

Then, the individual pixel evaluation values g1, g2 is, as described above, the correlation value of the difference ^{_{(x-μ θ) 2,}} | x-μ θ | , the weighting factor ^{_{(1 / σ θ 2),}} (1 / b Obtained by multiplying _θ). Therefore, the individual pixel evaluation values g1, g2 is, the correlation value of the difference in the case of no noise ^{_{(x-μ θ) 2,}} | x-μ θ | of influence is relatively large, include noise in some cases may have been, the correlation value of the difference ^{_{(x-μ θ) 2,}} | x-μ θ | effect becomes relatively small.

Since the all-pixel evaluation value is obtained by the individual pixel evaluation values g1 and g2 obtained as described above, the influence of noise becomes small as a result. That is, it is possible to reduce the influence of noise in the quality determination of the inspection target, and the determination performance can be improved.

(4. Application example of industrial inspection system)
The case where the inspection target of the industrial inspection systems 1 and 2 is, for example, a component of a column type electric steering device will be described as an example. The industrial inspection systems 1 and 2 perform a predetermined operation on the electric steering device, and the electric steering device operates based on the degree of vibration generated from the components of the electric steering device that operates by the operation. Judge whether it is a good product or not. In the following, an example in which the components of the column type electric steering device are to be inspected will be described, but other types of electric steering devices may be to be inspected.

In particular, the description of the configuration of the electric steering device to be inspected, the description of the data detected by the detection device 10, and the processing by the inspection image data acquisition units 21 and 32 will be described below.

(4-1. Configuration of electric steering device 150)
As shown in FIG. 5, the electric steering device 150 to be inspected mainly includes a steering mechanism 110, an intermediate shaft 120, and a steering mechanism 130.

The steering mechanism 110 mainly includes a steering member 111, an input shaft 112, a torsion bar 113, an output shaft 114, and a steering assist mechanism 140. The steering member 111 is a steering wheel operated by the driver. The input shaft 112 is a shaft member that connects the steering member 111 and the torsion bar 113. The input shaft 112 transmits the rotation (steering) of the steering member 111 to the torsion bar 113. The torsion bar 113 connects the input shaft 112 and the output shaft 114 so as to be relatively rotatable. The torsion bar 113 elastically deforms in the twisting direction when the relative rotation between the input shaft 112 and the output shaft 114 occurs. The output shaft 114 is a shaft member that outputs the rotation input from the steering member 111 via the input shaft 112 and the torsion bar 113 to the steering mechanism 130. The steering assist mechanism 140 assists the rotation of the output shaft 114 (steering assist of the steering member 111). The details of the steering assist mechanism 140 will be described later.

The intermediate shaft 120 transmits rotation between the steering mechanism 110 and the steering mechanism 130. One end side of the intermediate shaft 120 in the axial direction is connected to the output shaft 114 of the steering mechanism 110 via a universal joint 121, and the other end side of the intermediate shaft 120 in the axial direction is a pinion of the steering mechanism 130 via the universal joint 122. It is connected to the shaft 131.

The steering mechanism 130 includes a pinion shaft 131, a steering shaft 132, and a steering wheel 133. One end side of the pinion shaft 131 in the axial direction is connected to the output shaft 114 via a universal joint 122, and the pinion 131a is formed on the other end side of the pinion shaft 131 in the axial direction. A rack 132a that meshes with the pinion 131a is formed on the steering shaft 132, and steering wheels 133 are connected to both ends of the steering shaft 132 in the axial direction via a pair of tie rods 134 and a pair of knuckle arms 135. The steering mechanism 130 changes the steering angle of the steering wheel 133 by moving the steering shaft 132 in the axial direction (vehicle width direction).

Here, the configuration of the steering assist mechanism 140 will be described. The steering assist mechanism 140 includes a speed reducer 141, an electric motor 142, a torque sensor 143, a vehicle speed sensor 144, a rotation angle sensor 145, and a control unit 146.

The reducer 141 is a worm reducer including a worm gear (not shown) and a worm wheel (not shown) that meshes with the worm gear. The worm gear is integrally rotatably connected to the motor shaft of the electric motor 142, and the worm wheel is rotatably connected to the output shaft 114. The electric motor 142 rotates and drives the worm gear of the speed reducer 141. The rotation of the motor shaft of the electric motor 142 is transmitted to the steering mechanism 130 via the reduction gear 141, the output shaft 114, and the intermediate shaft 120, and is converted into a force that moves the steering shaft 132 in the axial direction in the steering mechanism 130. Will be done.

The torque sensor 143 detects the steering torque applied to the steering member 111 based on the amount of twist of the torsion bar 113. The vehicle speed sensor 144 detects the vehicle speed of a vehicle (not shown) equipped with the electric steering device 150. The rotation angle sensor 145 detects the rotation angle of the motor shaft of the electric motor 142. The control unit 146 sets a target value of the steering auxiliary torque given to the speed reducer 141 by the electric motor 142 based on the output signals from the torque sensor 143, the vehicle speed sensor 144, and the rotation angle sensor 145, and sets the actual steering auxiliary torque. Is controlled so that the current supplied to the electric motor 142 becomes the target value.

(4-2. Example of detection device 10)
An example of the detection device 10 in FIGS. 3 and 4 will be described with reference to FIG. The inspection targets are the electric motor 142 and the speed reducer 141, which are components of the electric steering device 150. The detection device 10 includes two vibration sensors 151 and 152 and one microphone 153.

The vibration sensors 151 and 152 are 3-axis accelerometers capable of detecting the vibration of the electric motor 142, and the two vibration sensors 151 and 152 are attached to the electric motor 142 at positions separated from each other. The vibration sensor 151 detects acceleration (vibration) in three directions of the electric motor 142 that operates when the steering member 111 is rotated at a constant rotational speed. Then, the vibration sensors 151 and 152 each output time-series waveform data relating to three patterns of acceleration (vibration).

The microphone 153 detects the sound generated from the speed reducer 141. The microphone 153 is arranged in the vicinity of the speed reducer 141 at a position where the sound generated from the speed reducer 141 can be detected. The microphone 153 detects a sound caused by the vibration of the speed reducer 141 that operates when the steering member 111 is rotated at a constant rotation speed. Then, the microphone 153 outputs time-series waveform data relating to one pattern of sound.

That is, in this example, the three sensors 151, 152, and 153 as the detection device 10 output time-series waveform data as output signals of a total of 7 patterns.

(4-3. Configuration of inspection image data acquisition units 21 and 32)
The configurations of the inspection image data acquisition units 21 and 32 will be described with reference to FIGS. 6-12. As shown in FIG. 6, the inspection image data acquisition units 21 and 32 include a primary data storage unit 21a, a secondary data generation unit 21b, and a tertiary data generation unit 21c.

The primary data storage unit 21a acquires the output signals from the vibration sensors 151 and 152 and the microphone 153, and stores the waveform data of each output signal as the time-series primary data DT1.

The primary data DT1 is an output signal of the vibration sensors 151 and 152 and the microphone 153. FIG. 7 shows waveform data of one detection signal from the vibration sensor 151 as an example of the primary data DT1. In the primary data DT1 shown in FIG. 7, the horizontal axis represents time T1 and the vertical axis represents acceleration G, which is an amplitude component. The vibration sensor 151 can detect vibration in each direction of the three axes, and the primary data storage unit 21a has three patterns of primary data DT1 indicating the degree of vibration from the vibration sensor 151 in each direction. And memorize.

The secondary data generation unit 21b generates the secondary data DT2, which is spectrogram data including the first frequency component FL1, the time component T2, and the amplitude component G, by performing a short-time Fourier transform on the primary data DT1. do. That is, the secondary data DT2 is obtained by converting the primary data DT1 into three-dimensional data including the first frequency component FL1, the time component T2, and the amplitude component G.

First, the secondary data generation unit 21b extracts and extracts short-time waveform data (first waveform data) having a minute time ΔTa (for example, 0.1 seconds) from the waveform data of the primary data DT1 shown in FIG. Fourier transform the first waveform data. The short-time waveform data is time-series data including a time component T2 and an amplitude component (acceleration G), and by Fourier transforming the short-time waveform data, as shown in FIG. 8, the first frequency component FL1 and the amplitude. Waveform data DT2a including the component G is generated.

Subsequently, the secondary data generation unit 21b first takes out the waveform data of the primary data DT1 and takes out the short-time waveform data (second waveform data) of the minute time ΔTb different from the minute time ΔTa, and similarly , The extracted second waveform data is Fourier transformed. The second waveform data in the minute time ΔTb is short-time waveform data in which the time component is shifted by a predetermined time (for example, 0.05 seconds) from the first waveform data in the minute time ΔTa.

Then, the secondary data generation unit 21b extracts the short-time waveform data of the minute time ΔT while shifting the waveform data of the primary data DT1 by a predetermined time, and Fourier transforms each of the extracted short-time waveform data. In this way, as shown in FIG. 8, a plurality of waveform data DT2a are generated.

Then, as shown in FIG. 9, the secondary data generation unit 21b has a first frequency component FL1, a time component T2, and an amplitude component (acceleration) based on the result of performing a Fourier transform on a plurality of short-time waveform data. G) Generates spectrogram data including. In the spectrogram data shown in FIG. 9, the two-dimensional coordinate axes are the time t2 (horizontal axis) and the first frequency FL1 (vertical axis), and the color attribute indicates the acceleration G which is an amplitude component. The color attribute may include all attributes of hue, saturation, and lightness, or may include only some attributes.

The tertiary data generation unit 21c obtains the first frequency component FL1, the second frequency component FL2, and the amplitude component G by performing Fourier transform on the time-oscillation data for each first frequency component in the secondary data DT2. Generates tertiary data DT3, which is image data with specifications similar to spectrogram data including. That is, the tertiary data DT3 is obtained by converting the secondary data DT2 into three-dimensional data including the first frequency component FL1, the second frequency component FL2, and the amplitude component G.

First, the tertiary data generation unit 21c extracts the time-series data DT3a for each first frequency FL1 in FIG. 9. For example, in FIG. 9, the time-series data DT3a with respect to the first frequency FL1 of 4 kHz is the graph shown in FIG. That is, FIG. 10 shows a graph showing the behavior of the amplitude component (acceleration G) at which the first frequency FL1 is 4 kHz as time series data. The third data generation unit 21c similarly extracts the time series data DT3a having the time T2-amplitude G for the other first frequency FL1.

Subsequently, as shown in FIG. 11, the tertiary data generation unit 21c Fourier transforms the time series data of the time T2-amplitude G for each first frequency component FL1 and performs a Fourier transform on the second frequency for each first frequency component FL1. FL2-Amplitude G data DT3b is generated. Note that FIG. 11 shows a graph showing the amplitude component (acceleration G) at which the first frequency FL1 is 4 kHz for each second frequency component FL2. In this way, the tertiary data generation unit 21c Fourier transforms the time-series data of the time T2-amplitude G included in the secondary data DT2 for each first frequency component FL1 and does not include the time component DT3b. Convert to.

Subsequently, as shown in FIG. 12, the tertiary data generation unit 21c has the first frequency component FL1 and the second frequency based on the data DT3b for the second frequency FL2-amplitude G for each first frequency component FL1. The tertiary data DT3, which is image data representing the component FL2 and the amplitude component G, is generated. As shown in FIG. 12, the tertiary data DT3 is image data having specifications similar to spectrogram data, and the two-dimensional coordinate axes are the second frequency FL2 (horizontal axis) and the first frequency FL1 (vertical axis). , The color attribute indicates the acceleration G, which is an amplitude component. As a result, the tertiary data generation unit 21c can convert the secondary data DT2, which is image data including the time component, into the tertiary data DT3, which is the image data not including the time component.

Then, the inspection image data acquisition units 21 and 32 use the tertiary data DT3 shown in FIG. 12 as the inspection image data. That is, the above-mentioned processing is performed using the tertiary data DT3 shown in FIG. 12 as the input of the variational autoencoder.

Further, the inspection image data acquisition units 21 and 32 may use the secondary data DT2 shown in FIG. 9 as the inspection image data instead of the tertiary data DT3. In this case, the above-mentioned processing is performed using the secondary data DT2 shown in FIG. 9 as the input of the variational autoencoder. The inspection image data acquisition units 21 and 32 include a primary data storage unit 21a and a secondary data generation unit 21b, and do not include a tertiary data generation unit 21c.

1,2: Industrial inspection system, 10: Detection device, 20,40: Learning processing device, 21: Inspection image data acquisition unit, 21a: Primary data storage unit, 21b: Secondary data generation unit, 21c: Third data generation unit, 23, 43: trained model generation unit, 30, 50: inspection processing device, 31, 51: trained model storage unit, 32: inspection image data acquisition unit, 33, 53: output value acquisition Unit, 34, 54: Difference value calculation unit, 35, 55: Individual pixel evaluation value calculation unit, 36: All pixel evaluation value calculation unit, 37: Judgment unit, D1, D2: Decoder, DT1: Primary data, DT2 : Secondary data, DT3: Tertiary data, E1, E2: Encoder, M1, M2: Training model

Claims (11)

An inspection image data acquisition unit that acquires inspection image data obtained by capturing an image of an inspection target, which is a device used in industry, or converted data caused by the operation of the inspection target, and an inspection image data acquisition unit.
A learning model in which a variable auto encoder (VAE) having a predetermined probability distribution is used as a learning model, the inspection image data is used as an input of the encoder, and an expected value and a dispersion are output of a decoder as parameters of the distribution of the reconstructed data. A trained model storage unit that stores a trained model that has been machine-learned using the inspection image data of a non-defective product as training data.
When the inspection image data of the inspection target is input, the output value acquisition unit for acquiring the expected value and the variance as the output of the trained model,
A difference value calculation unit that calculates the difference between the inspection image data to be inspected and the expected value acquired by the output value acquisition unit for each pixel.
Individual pixel evaluation value by multiplying the correlation value of the difference output by the difference value calculation unit for each pixel by the weighting coefficient represented by the correlation value of the dispersion acquired by the output value acquisition unit. Individual pixel evaluation value calculation unit that calculates
An all-pixel evaluation value calculation unit that calculates an all-pixel evaluation value based on the individual pixel evaluation value for each pixel,
A determination unit that determines the quality of the inspection target based on the evaluation value of all pixels, and
Equipped with an industrial inspection system. The individual pixel evaluation value calculation unit has the weighting coefficient, which is the reciprocal of the correlation value of the dispersion acquired by the output value acquisition unit, to the correlation value of the difference output by the difference value calculation unit for each pixel. The industrial inspection system according to claim 1, wherein the individual pixel evaluation value is calculated by multiplying by. The trained model is a model in which machine learning is performed so that the inspection image data of a non-defective product is used as training data and the probability density function representing the predetermined probability distribution is set to the maximum value. The industrial inspection system described in. The industrial inspection system according to claim 3, wherein the probability density function is a probability density function having a normal distribution. The output value acquisition unit acquires the average μ in the probability density function of the normal distribution as the expected value, and acquires the variance σ ² in the probability density function of the normal distribution as the variance.
The difference value calculation unit calculates the difference (x−μ) between the inspection image data x to be inspected and the average μ acquired by the output value acquisition unit for each pixel.
The individual pixel evaluation value calculation unit multiplies the difference squared value (x−μ) ² as the correlation value of the difference by the weighting coefficient which is the reciprocal of the variance σ ^{2 for each pixel.} Calculate the individual pixel evaluation value and
The industrial inspection system according to claim 4, wherein the all-pixel evaluation value calculation unit calculates a value obtained by summing the individual pixel evaluation values as the all-pixel evaluation value. The industrial inspection system according to claim 3, wherein the probability density function is a probability density function of the Laplace distribution. The output value acquisition unit acquires the average μ in the probability density function of the Laplace distribution as the expected value, and acquires the variance 2b ² in the probability density function of the Laplace distribution as the variance.
The difference value calculation unit calculates the difference (x−μ) between the inspection image data x to be inspected and the average μ acquired by the output value acquisition unit for each pixel.
The individual pixel evaluation value calculation unit multiplies the absolute difference value | x-μ | as the correlation value of the difference by the weighting coefficient, which is the reciprocal of the correlation value b of the dispersion, for each pixel. Calculate the individual pixel evaluation value and
The industrial inspection system according to claim 5, wherein the all-pixel evaluation value calculation unit calculates a value obtained by summing the individual pixel evaluation values as the all-pixel evaluation value. The device used in the industry is a finished product including at least one of an industrial machine, an automobile, and an aircraft, or any one of claims 1-7 including at least one of the parts constituting the finished product. Described industrial inspection system. The inspection image data is inspection image data obtained by converting sound data or vibration data caused by the operation of the inspection target. The industrial inspection system according to any one of claims 1-8. The industrial inspection system is
A primary data storage unit that stores sound data or vibration data caused by the operation of the inspection target and stores primary data in a time series.
A secondary data generator that generates secondary data, which is a spectrogram including a primary frequency component, a time component, and an amplitude component, by performing a short-time Fourier transform on the primary data.
The third is image data representing the first frequency component, the second frequency component, and the amplitude component by Fourier transforming the time-oscillation data for each first frequency component in the secondary data. A tertiary data generation unit that generates data as the inspection image data,
9. The industrial inspection system according to claim 9. The industrial inspection system is
A primary data storage unit that stores sound data or vibration data caused by the operation of the inspection target and stores primary data in a time series.
A secondary data generation unit that generates secondary data, which is a spectrogram including a first frequency component, a time component, and an amplitude component, as the inspection image data by performing a short-time Fourier transform on the primary data.
9. The industrial inspection system according to claim 9.

Images