JP7417426B2 - Anomaly detection device and anomaly detection method - Google Patents

Description

The present invention relates to an abnormality detection device and an abnormality detection method, and particularly to a technique for detecting an abnormality in a photographing environment from a plurality of images.

In order to ensure judgment accuracy and reliability in quality inspections using images, it is important to collect images in which the state of items other than the product is constant. To this end, it is desirable to be able to quickly detect abnormalities in the shooting environment during daily operations. The photographing environment includes settings of the imaging device such as the position and fixation of the device, and imaging parameters such as aperture and shutter speed.

To detect changes in the imaging environment, there is a method of directly measuring the state of the imaging device using sensors such as a position sensor and a vibration sensor. However, since this increases the cost of the sensor and makes it difficult to add it to the existing environment, it would be more useful to be able to detect abnormalities from images that are originally captured.

As a conventional technique for detecting abnormalities in the photographing environment from photographed images, for example, Patent Document 1 discloses a method for detecting abnormalities by determining the physical quantity distribution of the inspection object from the image and comparing it with the past distribution, which is used to determine the quality of the object. Discloses technology to do so.

Further, for example, Patent Document 2 discloses a technique for detecting an abnormality by comparing a performance value obtained during template matching with a past value.

Japanese Patent Application Publication No. 10-153558 JP2013-037510A

However, the technology described in Patent Document 1 that detects the presence or absence of an abnormality by determining the physical quantity to be inspected from an image and comparing the current physical quantity distribution of a certain number of products with the past physical quantity distribution, It is necessary to set the physical quantities to be used. Therefore, in the case of quality inspection using images of products, there is a problem that it becomes difficult to obtain physical quantities in the inspection target.

Further, the technique using template matching described in Patent Document 2 has a problem in that a template image needs to be prepared at the photographing position of each product.

As described above, with the conventional technology, it is difficult to detect an abnormality occurring in the photographing environment based on the image of the product to be inspected.

The present invention has been made to solve the above-mentioned problems, and an object of the present invention is to easily detect abnormalities occurring in the photographing environment from images of products.

In order to solve the above-mentioned problems, an anomaly detection device according to the present invention includes a converting unit configured to convert each of a plurality of images acquired from an imaging device into a spectrum in a spatial frequency domain, and the converting unit A model application unit configured to apply a nonlinear model having an inflection point to the spectrum converted by the above method, and a spatial frequency corresponding to the inflection point from the nonlinear model applied to the spectrum. an extraction unit configured to extract, and an evaluation value configured to obtain and output an evaluation value indicating the quality of each of the plurality of images based on a spatial frequency corresponding to the extracted inflection point. a calculation unit; a storage unit configured to store time-series data of the evaluation values for the plurality of images output from the evaluation value calculation unit; and a storage unit configured to store time-series data of the evaluation values for the plurality of images output from the evaluation value calculation unit; A time series analysis unit configured to perform time series analysis and determine that an abnormality has occurred in a pre-built imaging environment including the imaging device if the result of the time series analysis satisfies a predetermined condition. Equipped with.

Further, in the anomaly detection device according to the present invention, the time series analysis unit detects that an evaluation value outside a preset allowable range of the evaluation value is detected within a preset period in the time series data. In this case, it may be determined that an abnormality has occurred in the photographing environment.

Further, in the anomaly detection device according to the present invention, the preset period includes the most recent certain period in the time series data, and the time series analysis unit is configured to analyze the time series data in the most recent certain period. It may be determined that an abnormality has occurred in the photographing environment when the number of included evaluation values outside the permissible range is equal to or greater than a set number.

Further, the anomaly detection device according to the present invention may further include a display section configured to display information regarding the results of the time series analysis by the time series analysis section on a display screen.

In order to solve the above-mentioned problems, an anomaly detection method according to the present invention includes a first step of converting each of a plurality of images acquired from an imaging device into a spectrum in a spatial frequency domain; a second step of applying a nonlinear model having an inflection point to the spectrum; a third step of extracting a spatial frequency corresponding to the inflection point from the nonlinear model applied to the spectrum; a fourth step of determining and outputting an evaluation value indicating the quality of each of the plurality of images based on the spatial frequency corresponding to the inflection point, and a fifth step of storing time-series data of evaluation values in a storage unit; performing a time-series analysis of the time-series data stored in the storage unit; and when the result of the time-series analysis satisfies a predetermined condition; and a sixth step of determining that an abnormality has occurred in a pre-constructed imaging environment including the imaging device.

Further, the anomaly detection method according to the present invention may further include a seventh step of displaying information regarding the result of the time series analysis executed in the sixth step on a display screen.

According to the present invention, time-series analysis of time-series data of evaluation values indicating image quality is performed, and if the result of the time-series analysis satisfies a predetermined condition, an abnormality occurs in a pre-built imaging environment including an imaging device. It is determined that this has occurred. Therefore, abnormalities occurring in the photographing environment can be easily detected from the product image.

FIG. 1 is a block diagram showing the configuration of an abnormality detection device according to an embodiment of the present invention. FIG. 2 is a block diagram showing the configuration of the time series analysis section according to this embodiment. FIG. 3 is a block diagram showing an example of the hardware configuration of the abnormality detection device according to the present embodiment. FIG. 4 is a flowchart showing an example of the operation of the abnormality detection device according to the present embodiment. FIG. 5 is a diagram for explaining evaluation values according to this embodiment. FIG. 6 is a diagram for explaining evaluation values according to this embodiment. FIG. 7 is a flowchart for explaining time series analysis processing according to this embodiment. FIG. 8 is a diagram for explaining the effects of this embodiment.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to FIGS. 1 to 8.

[Summary of the invention]
First, an overview of an abnormality detection device 1 according to an embodiment of the present invention will be explained.
For example, the abnormality detection device 1 detects abnormalities in the photographing environment of an image inspection system that performs external appearance inspections of products, such as non-defective products and defective products, based on images of products obtained in the manufacturing process. Detected by image.

In the image inspection system, the photographing environment such as the position of the camera 2, lighting, and installation position of the object 3 is constructed in advance, and each object 3 obtained in the manufacturing process is photographed under the same environmental settings. It is assumed that However, for some reason, vibration may occur in the equipment, the height of the product itself may change, or a slight shift in the position of the camera 2 may occur. In particular, when the object 3 is a minute product, minute environmental changes may cause a large change in the photographed image and may also affect the results of the visual inspection.

In the abnormality detection device 1 according to the embodiment of the present invention, an evaluation value indicating the quality of the image is obtained and accumulated for each image of the object 3 that is regularly photographed by the image inspection system. In addition, the anomaly detection device 1 analyzes evaluation values for a plurality of accumulated images over time, and detects abnormalities that occur in the time-series data of evaluation values, thereby detecting abnormalities that have occurred in the pre-constructed shooting environment. Detect things.

[Functional block of anomaly detection device]
As shown in FIG. 1, the abnormality detection device 1 acquires an image of a target object 3 such as a product taken by a camera 2 installed outside. The camera 2 can, for example, take one image or a plurality of images of each of a plurality of objects 3 manufactured by the same manufacturing process. The image taken by the camera 2 includes the appearance of the object 3 such as a product. In this embodiment, by repeatedly photographing the object 3 with the camera 2, a plurality of images can be obtained in chronological order.

A photographing environment such as the position of the camera 2, lighting, and the installation position of the object 3 is established in advance, and each object 3 obtained in the manufacturing process is photographed under the same environmental settings. However, in this embodiment, as mentioned above, vibration may occur in the equipment, the height of the product itself may change, or a slight shift in the position of the camera 2 may occur for some reason. Changes in environmental settings in a shooting environment are detected as abnormalities.

The anomaly detection device 1 includes an acquisition section 10, a correction section 11, a conversion section 12, a model application section 13, an extraction section 14, an evaluation value calculation section 15, a storage section 16, a time series analysis section 17, and a display section 18.

As shown in FIG. 1, the acquisition unit 10 acquires an image of the object 3 photographed by a camera 2 (imaging device) installed externally. Such an acquisition unit 10 can be realized by an interface circuit. The acquisition unit 10 acquires a plurality of images from the camera 2 through wired or wireless communication. The image is, for example, digital data of a still image.

The correction unit 11 performs preset correction processing such as cutting out a region of the object 3 included in the image of the object 3 acquired by the acquisition unit 10, correcting color tone, resizing, and removing noise.

The converting unit 12 converts each of the plurality of images subjected to the correction process by the correcting unit 11 into a spectrum in the spatial frequency domain. More specifically, the transform unit 12 performs two-dimensional discrete Fourier transform on the spatial domain image and performs polar coordinate transformation, thereby outputting a frequency spectrum, that is, a power spectrum, indicating the spatial frequency distribution for each image.

The model application unit 13 applies a nonlinear model having an inflection point to the spectrum obtained by the conversion unit 12. Here, the nonlinear model is a model of the spatial frequency spectrum of an image. A nonlinear function having only one parameter indicating an inflection point is used as the nonlinear model. As the nonlinear function having an inflection point, for example, a nonlinear monotonically decreasing function such as a sigmoid function can be used. In this embodiment, a case will be explained using a four-parameter sigmoid function as shown in the following equation (1) as an example.

ただし、ａ、ｂは上限および下限を決めるパラメータ、ｃは曲線形状を決めるパラメータ、ｄは変曲点を示すパラメータである。

However, a and b are parameters that determine the upper and lower limits, c is a parameter that determines the shape of the curve, and d is a parameter that indicates an inflection point.

For example, the model application unit 13 adjusts the values of parameters a, b, c, and d of the 4-parameter sigmoid function so that the error between the spectrum obtained by the conversion unit 12 and the 4-parameter sigmoid function model is minimized. demand.

The extraction unit 14 extracts the value of a parameter d indicating an inflection point from the nonlinear model fitted to the spectrum. More specifically, the extraction unit 14 extracts the value of the inflection point d of the four-parameter sigmoid function applied to the power spectrum of each image by the model application unit 13, that is, the value corresponding to the inflection point of the frequency spectrum of each image. Extract the spatial frequency (period).

It is known that when an image is out of focus and blurred, the high frequency components of the power spectrum decrease. In this embodiment, the value of the inflection point of the power spectrum to which the four-parameter sigmoid function shown in equation (1) above is applied is extracted as information indicating the degree of decline in high frequency components in the image.

The evaluation value calculation unit 15 calculates an evaluation value indicating the quality of each of the plurality of images related to the value of the inflection point extracted by the extraction unit 14, based on the criteria set for the value of the inflection point. In the present embodiment, the evaluation value calculation unit 15 obtains a normalized value by dividing the value of the inflection point extracted by the extraction unit 14 by the maximum frequency of the target image as an evaluation value.

The storage unit 16 is a storage circuit that stores time-series data of evaluation values calculated by the evaluation value calculation unit 15. Specifically, the storage unit 16 stores time-series data in which evaluation values correspond to image capturing times.

The time series analysis unit 17 performs time series analysis on the time series data of evaluation values stored in the storage unit 16, and when the result of the time series analysis satisfies a predetermined condition, the time series analysis unit 17 performs a time series analysis on the time series data of evaluation values stored in the storage unit 16, and when the result of the time series analysis satisfies a predetermined condition, It is determined that an abnormality has occurred. For example, when an evaluation value that is out of the allowable range of evaluation values set for image quality is detected within a preset period in time-series data of evaluation values, the time series analysis unit 17 It is determined that an abnormality has occurred. In the following, a case where a lower limit value is used as the permissible range of the preset evaluation value will be described as an example.

The time series analysis unit 17 can use various time series analysis algorithms such as a statistical distribution model as a method for analyzing time series data of evaluation values. For example, the time series analysis unit 17 calculates, from time series data of evaluation values indicating image quality, evaluations that are less than the lower limit of a preset evaluation value within a preset period, such as the most recent day. If the number of values is equal to or greater than the set number, it can be determined that an abnormality has occurred in the photographing environment. In other words, the time series analysis unit 17 determines that an abnormality has occurred in the pre-constructed photographing environment when there are a set number or more of images whose quality does not meet a certain standard.

As shown in FIG. 2, the time series analysis unit 17 includes, for example, a setting unit 170, a first determination unit 171, and a second determination unit 172.

The setting section 170 sets determination criteria used by the first determination section 171 and the second determination section 172. The setting unit 170 can set a dynamic value or a fixed value as a determination criterion. For example, the setting unit 170 may be configured to read a first criterion and a second criterion, which will be described later, as the determination criteria stored in the storage unit 16. Alternatively, the setting unit 170 can set the determination criteria according to an operation input received by the input device 108, which will be described later.

The first determination unit 171 determines, based on the evaluation value for each image, whether each image satisfies the first criterion regarding image quality set for the evaluation value. For example, assume that time-series data of evaluation values has a normal distribution determined by mean μ and standard deviation σ. In this case, for example, if the evaluation value of the image is less than the lower limit value obtained by subtracting a predetermined positive integer multiple of the standard deviation from the average value of evaluation values over a certain period of time in the past, the first determination unit 171 determines that It is determined that the image with that evaluation value does not satisfy the first criterion.

More specifically, the first determination unit 171 selects an image whose evaluation value is less than the lower limit [μ−3σ], which is obtained by subtracting three times the standard deviation σ from the average μ of the evaluation values for the past three days. It is determined that the object 3 including the object 3 does not satisfy the first criterion regarding image quality. For example, the first determination unit 171 determines that when a plurality of images are acquired for one object 3 and the number of images with evaluation values that are less than the lower limit value is more than a certain percentage, the first determination section 171 determines that the object 3 is It can be determined that the first criterion is not satisfied.

The second determining unit 172 determines whether the abnormality is determined when the number of images determined by the first determining unit 171 as not meeting the first criterion is greater than or equal to a preset number within a preset period. It is determined that the second criterion is satisfied. For example, the second determination unit 172 can determine that the second criterion is satisfied when the number of objects 3 that do not meet the first criterion exceeds 20 in one day. The time series analysis unit 17 determines that an abnormality has occurred in the photographing environment when the second determination unit 172 determines that the second criterion is satisfied.

The display unit 18 displays the results of the time-series analysis, including the time-series data of the evaluation values analyzed by the time-series analysis unit 17 and information indicating that an abnormality has occurred in the photographing environment, on the display device 109 described below. .

[Hardware configuration of anomaly detection device]
Next, an example of a hardware configuration for realizing the abnormality detection device 1 having the above-described functions will be described with reference to the block diagram of FIG. 3.

As shown in FIG. 3, the abnormality detection device 1 includes, for example, a processor 102 connected via a bus 101, a main storage device 103, a communication interface 104, an auxiliary storage device 105, an input/output I/O 106, a clock 107, an input This can be realized by a computer including the device 108 and the display device 109, and a program that controls these hardware resources. The processor 102 includes a CPU, a GPU, and the like.

The main storage device 103 stores in advance programs for the processor 102 to perform various controls and calculations. The processor 102 and the main storage device 103 operate the correction section 11, the conversion section 12, the model application section 13, the extraction section 14, the evaluation value calculation section 15, the time series analysis section 17, etc. shown in FIG. Each function is realized.

The communication interface 104 is an interface circuit for establishing a network connection between the abnormality detection device 1 and various external electronic devices. The acquisition unit 10 shown in FIG. 1 is realized by the communication interface 104.

The auxiliary storage device 105 includes a readable and writable storage medium and a drive device for reading and writing various information such as programs and data to and from the storage medium. For the auxiliary storage device 105, a semiconductor memory such as a hard disk or a flash memory can be used as a storage medium.

The auxiliary storage device 105 stores an anomaly detection program that the anomaly detection device 1 uses to include image correction processing, Fourier transformation processing, nonlinear model application processing, inflection point extraction processing, evaluation value calculation processing, and time series analysis processing. It has a program storage area. The auxiliary storage device 105 has an area for storing time-series data of evaluation values determined for each image. Further, the auxiliary storage device 105 has an area for storing a four-parameter sigmoid function used by the model application unit 13. The auxiliary storage device 105 has an area for storing images taken by the camera 2 and acquired by the acquisition unit 10. Further, the auxiliary storage device 105 has an area for storing a first criterion and a second criterion as predetermined conditions used by the time series analysis unit 17 in time series analysis.

The auxiliary storage device 105 implements the storage unit 16 described in FIG. Furthermore, for example, it may have a backup area for backing up the data, programs, etc. mentioned above.

The input/output I/O 106 is composed of I/O terminals that input signals from external devices and output signals to external devices.

The clock 107 is an internal clock of the abnormality detection device 1 and measures time. Alternatively, the clock 107 may obtain time information from an external time server.

The input device 108 includes physical keys, a touch panel, and the like, and outputs a signal in response to an external operation input. For example, the input device 108 can receive input of a lower limit value set for the evaluation value included in the first standard used by the time series analysis unit 17 and a second standard used for abnormality determination.

The display device 109 is configured with a liquid crystal display or the like. The display device 109 implements the display unit 18 described in FIG.

The camera 2 can convert optical signals into image signals and generate moving images and still images. More specifically, the camera 2 includes an image sensor such as a CCD (Charge-Coupled Device) image sensor or a CMOS image sensor, and images incident light from an imaging area on a light receiving surface. Convert to electrical signal. Note that the magnification, focus, and the like when the camera 2 photographs an image are automatically set in advance by a control unit (not shown) included in the camera 2. Communication between the camera 2 and the abnormality detection device 1 may be performed wirelessly.

[Anomaly detection method]
Next, the operation of the abnormality detection device 1 having the above-described configuration will be explained using the flowcharts of FIGS. 4 and 5. As a premise, it is assumed that settings have been made in advance such that all images are taken at a certain angle, distance, and lighting environment. However, assume that the images taken by the camera 2 vary due to variations in the surrounding environment and products, vibrations in equipment, and the like.

As shown in FIG. 4, first, the acquisition unit 10 acquires an image including the object 3 that is regularly photographed by the camera 2 (step S1). For example, the camera 2 can take multiple images of each of the multiple objects 3 related to the same product. The acquisition unit 10 acquires a plurality of captured images. Next, the correction unit 11 cuts out a region including the target object 3 from each acquired image (step S2). Next, the transform unit 12 performs two-dimensional discrete Fourier transform on each image corrected in step S2, and outputs a spectrum in the spatial frequency domain (step S3).

Next, the model application unit 13 applies a four-parameter sigmoid function shown in equation (1) to the power spectrum of each image obtained in step S3 (step S4). Next, the extraction unit 14 extracts the spatial frequency corresponding to the inflection point from the four-parameter sigmoid function applied to the power spectrum of each image (step S5).

Thereafter, the evaluation value calculation unit 15 calculates an evaluation value indicating the quality of each of the plurality of images based on the spatial frequency corresponding to the extracted inflection point (step S6). Specifically, the evaluation value calculation unit 15 obtains a value obtained by dividing the value of the spatial frequency corresponding to the inflection point extracted by the extraction unit 14 by the maximum frequency of the target image as the evaluation value.

FIG. 5 shows an example of an image in focus, that is, the best evaluation value (evaluation value: 1.0). Further, FIG. 5 shows the power spectrum (ps) of the image and the power spectrum (sigm) to which the 4-parameter sigmoid function is applied. In an in-focus natural image with the best evaluation value as shown in FIG. 5, the power spectrum is a straight line on the logarithmic axis. Further, as shown in FIG. 5, the inflection point d of the image with the best evaluation value of 1.0 appears at the right end on the horizontal axis (period) in the power spectrum, as shown by the broken line.

Figure 6 shows images (a), (b), (c), (d), and (e) of multiple objects 3, their respective power spectra (ps) in the spatial frequency domain, and the four-parameter sigmoid function. The fitted power spectrum (sigm), inflection point d, and evaluation value are shown.

In the example of FIG. 6, the image blur becomes stronger from image (a) to image (e). Focusing on the inflection point d (dashed line) of each image, as the blur of the image becomes stronger, the inflection point d moves from the right end to the left end on the horizontal axis of the spectrum, which represents the relationship between the intensity and period of the image. I understand that. In this way, by using the inflection point d as information indicating a decrease in the distribution of high frequency components of the image, a quantitative evaluation value for evaluating the quality of the image is calculated.

Returning to the flowchart of FIG. 4, the storage unit 16 accumulates the evaluation value for each image calculated in step S6 together with the shooting time (step S7). Next, the time series analysis unit 17 performs time series analysis on the time series data of evaluation values stored in the storage unit 16 (step S8).

Here, an example of the time series analysis process executed in step S8 will be described with reference to FIG. In the following, it is assumed that, for example, eight images are taken for each object 3, and an evaluation value for each image is calculated and stored in the storage unit 16 as time-series data. Further, the unit of the period for detecting an abnormality occurring in the photographing environment is, for example, one day.

First, the time series analysis unit 17 reads the evaluation value for each image stored in the storage unit 16 (step S80). Next, when the time series analysis unit 17 acquires evaluation value data for the past three days as a preset period (step S81: YES), the setting unit 170 stores the determination reference value ( Step S82). On the other hand, the time series analysis unit 17 reads the evaluation value for each image accumulated in the storage unit 16 (step S80) until evaluation value data for three days is acquired (step S81: NO).

For example, when the time series analysis unit 17 acquires the time series data of the evaluation values for the past three days in step S81, the time series analysis unit 17 acquires the time series data of the evaluation values for the past three days. Load the first and second criteria. For example, as the first criterion used by the first determination unit 171, the lower limit value [μ−3σ] set for the evaluation value is obtained. Further, as a second criterion used by the second determination unit 172, a reference value for abnormality determination [20 objects/day] (the number of objects 3 that do not satisfy the first criterion is 20 objects per day) is acquired.

Next, the first determination unit 171 determines whether or not the target object 3 does not satisfy the first criterion (step S83). For example, among the eight images taken per object 3, if the object 3 includes two or more images with an evaluation value lower than the lower limit of the evaluation value [μ-3σ] (step S83: YES) ), it is determined that the object 3 does not satisfy the first criterion, and the number of objects 3 that do not satisfy the first criterion per day is counted up (+1) (step S84). The storage unit 16 records the number of objects 3 that do not satisfy the first criterion per day, which is determined based on time-series data of evaluation values.

Next, the second determination unit 172 determines that the second criterion is satisfied when the number of objects 3 per day that do not meet the first criterion exceeds the abnormality determination criterion [20 objects/day]. (Step S85: YES), the time series analysis unit 17 determines that an abnormality has occurred in the photographing environment, and generates a notification (Step S86). Note that the generation of the notification in step S86 may be omitted.

Next, the display unit 18 updates the data by reflecting the results of the time series analysis, including the determination result in step S85, on the time series data (step S87). Next, when the analysis period set as the period for performing time series analysis has ended (step S88: YES), the process returns to step S9 in FIG. 4. On the other hand, if the end of the analysis period has not been reached (step S88: NO), the processes from step S80 to step S87 are repeated. For example, when the period set as the analysis period, for example, one day (24 hours) ends, the display section 18 moves the process to step S9 in FIG. 4.

Thereafter, returning to FIG. 4, the display unit 18 displays a display screen in which the results of the time-series analysis are reflected and the time-series data of the evaluation values is updated (step S9). The display unit 18 can also display the notification generated in step S86 on the display screen.

[Effects of the abnormality detection device according to this embodiment]
Next, a specific example of abnormality detection performed using the abnormality detection device 1 according to the present embodiment and determination results will be described with reference to FIG. 8.

In this example, a plurality of images taken to determine the quality of solder joints of a certain product were used to detect an abnormality that occurred in the photographing environment. Specifically, from November 19, 2018, we calculated evaluation values for each image and started accumulating evaluation values for images that were regularly taken during the manufacturing process and included solder joints of products.

In addition, in this example, a problem occurred in the shooting environment on November 29, 2018, and then on December 3, 2018, it was identified that the fixing screws of the imaging device were loosened. has been resolved.

The upper part of FIG. 8 shows time-series data of evaluation values for each image. Marker "e" indicates the evaluation value for each image. Moreover, in the upper part of FIG. 8, the value "m" indicates the time series of the average value (μ) of the evaluation values for the immediately preceding three days. Further, the value “th” in the upper row of FIG. 8 indicates a time series of the lower limit value [μ−3σ] of the evaluation value used by the first determination unit 171.

The lower part of FIG. 8 shows the number of objects 3 (products) that do not meet the first criterion for each day. The value "s" shown in the lower part of FIG. 8 is the second standard used when the second determining unit 172 makes an abnormality determination, and in this example, 20 or more products per day that do not meet the first standard are If this occurs, it is determined that an abnormality has occurred in the shooting environment.

The upper and lower parts of FIG. 8 are shown on the same time scale. Based on the evaluation values of multiple images taken on November 29, 2018, it was detected that there were more than 20 products per day that contained images with evaluation values less than the lower limit [μ-3σ]. Ta. In this way, the time series analysis unit 17 included in the anomaly detection device 1 according to the present embodiment detects the anomaly that occurs in the time series data of the evaluation value indicating the quality of the image. We were able to detect anomalies in the shooting environment that occurred during this period.

Note that the time series data of the evaluation values and the results of the abnormality determination shown in FIG. 8 are plotted in real time on the display screen of the display device 109 by the display unit 18, and the data are updated.

As explained above, according to the abnormality detection device 1 according to the present embodiment, since the evaluation value indicating the image quality obtained for each image is analyzed in time series, Abnormalities can be easily detected. As a result, it becomes possible to recognize whether the determination result of the product appearance inspection is invalid due to an abnormality in the imaging environment.

Further, according to the abnormality detection device 1, there is no need to separately prepare data such as a template image other than the photographed image of the product to be inspected, and it is possible to detect abnormalities that occur in the photographing environment with a simpler configuration. can.

Further, according to the abnormality detection device 1, it is possible to detect abnormalities that occur in the photographing environment in the photographed image of the product to be inspected, even in situations where quality is inspected rather than physical quantities, and the inspection contents can be adjusted accordingly. Regardless of the situation, abnormalities can be easily detected.

In addition, according to the abnormality detection device 1, it is possible to perform time-series analysis of evaluation values and detect abnormalities that occur in the shooting environment in real time, thereby improving judgment accuracy and reliability in quality inspection using images. can be ensured.

Furthermore, the abnormality detection device 1 analyzes time-series data of evaluation values indicating the quality of each image, in which information indicating the degree of decline in high frequency components in the spatial frequency domain of the image is extracted from each image. The evaluation value that is the subject of time-series analysis is an absolute value that represents the quality of the image based on a standard set for the normalized inflection point value. In this way, since the evaluation value that more easily and quantitatively evaluates the image quality is subject to time-series analysis, it is possible to easily and quantitatively evaluate the quality of the image, so it is possible to analyze the image quality regardless of the target or type of quality inspection using the image. Anomalies that occur can be detected.

In addition, in the embodiment described, a case has been described in which there is one camera 2 that takes an image of the target object 3, but a plurality of cameras 2 may be installed in the same photographing environment.

Furthermore, in the described embodiment, the time series analysis unit 17 detects an abnormality occurring in the shooting environment from the captured images, and the display unit 18 displays the results of the time series analysis, for example. I explained. However, the display unit 18 is configured to generate a notification that the judgment result of the external appearance inspection of the product is invalid due to the abnormality in the imaging environment and display it on the display screen when it is detected that an abnormality has occurred in the imaging environment. Good too. Alternatively, a notification may be sent to a preset external terminal device or the like that an abnormality is detected in the photographing environment and that the determination result of the visual inspection is invalid.

Furthermore, in the described embodiment, when there is an evaluation value that is less than the preset lower limit of the evaluation value within a preset period in the time series data, the time series analysis unit 17 The explanation was given using an example where it is determined that an abnormality has occurred. However, the present invention is not limited to the case where the lower limit value is used as the allowable range of the evaluation value; for example, it is also possible to use the upper limit value depending on the design.

Although the embodiments of the abnormality detection device and the abnormality detection method of the present invention have been described above, the present invention is not limited to the described embodiments, and those skilled in the art can understand the invention within the scope of the invention described in the claims. Various conceivable modifications can be made.

DESCRIPTION OF SYMBOLS 1... Anomaly detection device, 2... Camera, 3... Target object, 10... Acquisition part, 11... Correction part, 12... Conversion part, 13... Model application part, 14... Extraction part, 15... Evaluation value calculation part, 16... Storage section, 17... Time series analysis section, 18... Display section, 170... Setting section, 171... First judgment section, 172... Second judgment section, 101... Bus, 102... Processor, 103... Main storage device, 104... Communication interface, 105... Auxiliary storage device, 106... Input/output I/O, 107... Clock, 108... Input device, 109... Display device, NW... Communication network.

Claims (6)

a conversion unit configured to convert each of the plurality of images acquired from the imaging device into a spectrum in a spatial frequency domain;
a model application unit configured to apply a nonlinear model having an inflection point to the spectrum converted by the conversion unit;
an extraction unit configured to extract a spatial frequency corresponding to the inflection point from the nonlinear model applied to the spectrum;
an evaluation value calculation unit configured to calculate and output an evaluation value indicating the quality of each of the plurality of images based on the spatial frequency corresponding to the extracted inflection point;
a storage unit configured to store time-series data of the evaluation values for the plurality of images output from the evaluation value calculation unit;
Time-series analysis of the time-series data stored in the storage unit is performed, and if the result of the time-series analysis satisfies a predetermined condition, an abnormality occurs in a pre-built imaging environment including the imaging device. an anomaly detection device, comprising: a time series analysis section configured to determine that an abnormality has occurred; The abnormality detection device according to claim 1,
The time series analysis unit determines that an abnormality has occurred in the photographing environment when an evaluation value outside a preset allowable range of the evaluation value is detected within a preset period in the time series data. An anomaly detection device characterized by determining that. The abnormality detection device according to claim 2,
The preset period includes the most recent certain period in the time series data,
The time series analysis unit determines that an abnormality has occurred in the shooting environment when the number of evaluation values included in the time series data that are outside the allowable range is equal to or greater than a set number during the most recent certain period. An anomaly detection device characterized by determining that an abnormality has occurred. The abnormality detection device according to any one of claims 1 to 3,
An anomaly detection device further comprising a display section configured to display information regarding the results of the time series analysis by the time series analysis section on a display screen. a first step of converting each of the plurality of images acquired from the imaging device into a spectrum in the spatial frequency domain;
a second step of applying a nonlinear model having an inflection point to the spectrum transformed in the first step;
a third step of extracting a spatial frequency corresponding to the inflection point from the nonlinear model applied to the spectrum;
a fourth step of determining and outputting an evaluation value indicating the quality of each of the plurality of images based on the spatial frequency corresponding to the extracted inflection point;
a fifth step of storing time-series data of the evaluation values for the plurality of images output in the fourth step in a storage unit;
Time-series analysis of the time-series data stored in the storage unit is performed, and if the result of the time-series analysis satisfies a predetermined condition, an abnormality occurs in a pre-built imaging environment including the imaging device. An anomaly detection method comprising: a sixth step of determining that an abnormality has occurred; In the abnormality detection method according to claim 5,
An anomaly detection method, further comprising: a seventh step of displaying information regarding the result of the time series analysis executed in the sixth step on a display screen.

Images