JP7344501B1 - Program, method, information processing device, system - Google Patents

Abstract

There is a problem in that there is no abnormal image data for a product, or abnormal image data cannot be created for unknown abnormalities. A program for causing a computer including a processor and a storage unit to execute an image acquisition step in which the processor acquires a first image including a predetermined product image; an image generation step of generating a second image by applying an image processing model based on predetermined hyperparameters; and an instruction from the user to determine whether a predetermined product image included in the second image is an abnormal image. and a condition changing step of changing predetermined hyperparameters so that the second image becomes an abnormal image of a predetermined product in the image generation step based on the judgment instruction received in the judgment reception step. program to do. [Selection diagram] Figure 9

Description

The present disclosure relates to a program, a method, an information processing device, and a system.

2. Description of the Related Art Techniques for inspecting products, etc. based on image data obtained by capturing product images using deep learning models and the like are known.
Patent Document 1 discloses an automatic inspection device in which a deep learning technique is implemented in an image processing process, which allows creation of an inference program for determining defects/noise even with a small number of samples.

JP 2023-8379 Publication

There is a problem in that there is no abnormal image data for the product, or abnormal image data cannot be created for unknown abnormalities.
Therefore, the present disclosure has been made to solve the above-mentioned problems, and its purpose is to generate new abnormal image data for products for which there is no abnormal image data or for unknown abnormalities. , is to inflate the amount.

A program to be executed by a computer including a processor and a storage unit, the program comprising: an image acquisition step in which the processor acquires a first image including a predetermined product image; and a predetermined hyperparameter for the first image. an image generation step of generating a second image by applying an image processing model based on , and a determination reception for receiving a determination instruction from a user regarding whether or not a predetermined product image included in the second image is an abnormal image. and a condition changing step of changing predetermined hyperparameters so that the second image becomes an abnormal image of a predetermined product in the image generation step based on the determination instruction received in the determination receiving step.

According to the present disclosure, it is possible to newly generate or inflate abnormal image data even when there is no abnormal image data for a product or for an unknown abnormality.

1 is a block diagram showing the functional configuration of system 1. FIG. 1 is a block diagram showing the functional configuration of a server 10. FIG. 3 is a block diagram showing the functional configuration of an administrator terminal 30. FIG. 3 is a diagram showing a data structure of a user table 1012. FIG. 3 is a diagram showing the data structure of an image table 1013. FIG. 3 is a diagram showing a data structure of a generation table 1014. FIG. 3 is a flowchart showing the operation of abnormal image generation model creation processing. It is an example of a screen showing the operation of abnormal image generation model creation processing. It is an example of a screen showing the determination process in the abnormal image generation model creation process. 2 is a block diagram showing the basic hardware configuration of a computer 90. FIG.

Embodiments of the present disclosure will be described below with reference to the drawings. In all the figures explaining the embodiments, common components are given the same reference numerals and repeated explanations will be omitted. Note that the following embodiments do not unduly limit the content of the present disclosure described in the claims. Furthermore, not all components shown in the embodiments are essential components of the present disclosure. Furthermore, each figure is a schematic diagram and is not necessarily strictly illustrated.

<System 1 configuration>
System 1 in the present disclosure uses image data (normal images) of the external appearance of a predetermined product in a state where there are no defects or defects related to the product, to determine whether the product has defects or defects in the production process, etc. This is an information processing system for generating images (abnormal images) of external appearance, etc. Note that the present disclosure is not limited to image data such as the external appearance of a product, but can be applied to image data related to the internal state of a product obtained by non-destructive testing, etc., and any other image data related to the product. .
The system 1 includes information processing devices such as a server 10 and a user terminal 20, which are connected via a network N.
FIG. 1 is a block diagram showing the functional configuration of the system 1. As shown in FIG.
FIG. 2 is a block diagram showing the functional configuration of the server 10.
FIG. 3 is a block diagram showing the functional configuration of the administrator terminal 30.

Each information processing device is composed of a computer including an arithmetic unit and a storage device. The basic hardware configuration of the computer and the basic functional configuration of the computer realized by the hardware configuration will be described later. Regarding each of the server 10 and the user terminal 20, explanations that overlap with the basic hardware configuration of the computer and the basic functional configuration of the computer, which will be described later, will be omitted.

<Configuration of server 10>
The server 10 is an information processing device that provides an information processing service that generates abnormal image data by applying an image processing model 1021 to normal image data regarding a predetermined product. With the information processing service according to the present disclosure, it is possible to newly generate or inflate abnormal image data even when there is no abnormal image data for a predetermined product or for an unknown abnormality. The generated abnormal image data can be used as training data when creating a deep learning model or the like for abnormality detection such as visual inspection on the production line of the product. In other words, by inflating the training data, the performance of the deep learning model for anomaly detection can be improved.
The server 10 includes a storage section 101, a control section 104, an input device 106, and an output device 108.

<Configuration of storage unit 101 of server 10>
The storage unit 101 of the server 10 includes an application program 1011, a user ID 2011, an application program 2012, a user table 1012, an image table 1013, a generation table 1014, an image processing model 1021, and a classification model 1022.

The application program 1011 is a program for causing the control unit 104 of the server 10 to function as each functional unit.
Application programs 1011 include applications such as web browser applications.

User ID 2011 is the user's account ID. The user transmits the user ID 2011 from the user terminal 20 to the server 10. The server 10 identifies the user based on the user ID 2011 and provides the service according to the present disclosure to the user. Note that the user ID 2011 includes information such as a session ID temporarily given by the server 10 to identify the user using the user terminal 20.

The application program 2012 may be stored in advance in the storage unit 201, or may be downloaded from a web server operated by a service provider via a communication IF.
Application programs 2012 include applications such as web browser applications.
Application program 2012 includes an interpreted programming language such as JavaScript® that is executed on a web browser application stored on user terminal 20.

The user table 1012 is a table that stores and manages information about member users (hereinafter referred to as users) who use the service. When a user registers to use a service, the user's information is stored in a new record in the user table 1012. This allows the user to use the service according to the present disclosure.
The user table 1012 is a table that uses the user ID as a primary key and has columns of user ID and user name.
FIG. 4 is a diagram showing the data structure of the user table 1012.

The user ID is an item that stores user identification information for identifying a user. The user identification information is an item in which a unique value is set for each user.
The user name is an item that stores the user's name. The user name may be any character string such as a nickname instead of a full name.

The image table 1013 is a table for storing and managing image data (normal image data) obtained by photographing the external appearance of a predetermined product in a state where there are no defects or defects related to the product.
The image table 1013 is a table having columns of image ID, user ID, image data, and image label.
FIG. 5 is a diagram showing the data structure of the image table 1013.

The image ID is an item that stores image identification information for identifying normal image data.
The user ID is an item that stores user identification information of a user who has stored normal image data.
Image data is an item that stores normal image data taken by a camera or the like. Reference information (paths) for image data files located elsewhere may also be stored. The format of the image data may be any data format such as jpeg, png, bmp, tiff, gif, eps, and svg.
The image label is an item that stores label information regarding normal image data. Specifically, the image label can include information indicating the condition and grade (good, defective) of the product included in the image data, product category information indicating the type and classification of the product, and other arbitrary label information.

The generation table 1014 is a table for storing and managing generated image data (generated data) generated by applying the image processing model 1021 to normal image data.
The generation table 1014 is a table having columns of generation ID, generation data, generation condition, and determination instruction.
FIG. 6 is a diagram showing the data structure of the generation table 1014.

The generation ID is an item that stores image identification information for identifying generated data.
Generated data is an item that stores generated data. Reference information (paths) for image data files located elsewhere may also be stored. The format of the image data may be any data format such as jpeg, png, bmp, tiff, gif, eps, and svg.
The generation condition is an item that stores information regarding generation conditions when generating generated data. The generation conditions include all information necessary for generating generation data from normal image data.
The generation conditions include image identification information of normal image data that is input to the image processing model 1021 when generating generated data.
The generation conditions include hyperparameter values that are input to the image processing model 1021 when generating generation data.
In addition, if there are multiple image processing models 1021, the generation conditions include information that identifies the image processing model 1021 used to generate the generated data among the multiple image processing models 1021, and information regarding hyperparameters. include.
The determination instruction includes information indicating a determination result regarding whether or not the generated data is abnormal image data.
The determination instruction may include either information indicating that the abnormal image data is "abnormal" or information indicating that the abnormal image data is "not abnormal."
The determination instruction may include information regarding the type of abnormality in the abnormal image data, such as "paint peeling", "corrosion", and "color abnormality".

The image processing model 1021 is a model to which image processing can be applied that adds an abnormality related to at least one of a chip, a distortion, a scratch, etc. to input image data. An image processing model can include one or more hyperparameters as input variables.
The hyperparameters include parameters regarding the size of the abnormal location, rotation angle, inversion angle, and color change. The hyperparameters include hyperparameters that correspond to the image processing model 1021, such as the filter size and the number of layers in the image processing model 1021. Depending on the value of one or the hyperparameters input, the image processing model can generate different generated data. In addition, by applying an image processing model based on hyperparameters optimized by abnormal image generation model creation processing to input image data containing normal images of the product, output image data containing abnormal images of the product can be generated. It can be output.
The hyperparameters applied to the image processing model are multiple hyperparameter groups that can cover the variable space corresponding to each image processing, depending on the content of the image processing such as "chip", "distortion", "scratch", etc. may also be included.
For example, a group of hyperparameters related to "chip" that corresponds to an image processing model related to "chip" is set as a first hyperparameter group. A hyperparameter group related to "distortion" corresponding to an image processing model related to "distortion" is defined as a second hyperparameter group. A hyperparameter group related to "abrasions" corresponding to an image processing model related to "abrasions" is defined as a third hyperparameter group. Each of the first hyperparameter group, second hyperparameter group, and third hyperparameter group can cover different variable spaces.
Specifically, when the hyperparameter group is a vector consisting of 10 components (hyperparameters), the first hyperparameter group is a vector consisting of the 0th to 3rd components, and the second hyperparameter group is a vector consisting of the 4th to 6th components. The third hyperparameter group can be a vector consisting of the 7th to 10th components.
Note that multiple hyperparameter groups do not necessarily have to be independent; some or all of the hyperparameters included in each hyperparameter group may be included in or overlap with other hyperparameter groups. . For example, the first hyperparameter group is a vector consisting of 0 to 3 and 6th components, the second hyperparameter group is a vector consisting of 4th to 6th, 8th, and 9th components, and the third hyperparameter group is a vector consisting of 7th to 10th components. It may also be a vector consisting of components.
Note that the image processing model according to the present disclosure does not need to be a rule-based image processing model, and may include any generative model such as a GAN (Generative Adversarial Network) in a deep learning model. Specifically, it includes any image processing model that can stochastically output image data based on normal image data and arbitrary hyperparameters.

The classification model 1022 is a learning model that receives image data as input information and outputs information (specifically, a classification label such as 0 or 1) indicating whether the input image data is an abnormal image.

<Configuration of control unit 104 of server 10>
The control unit 104 of the server 10 includes a user registration control unit 1041, a learning unit 1051, an input control unit 2041, and an output control unit 2042. The control unit 104 realizes each functional unit by executing the application program 1011 stored in the storage unit 101.

The user registration control unit 1041 performs a process of storing information of users who wish to use the service according to the present disclosure in the user table 1012.
Information stored in the user table 1012 is transmitted to the server 10 by a user opening a web page operated by a service provider from any information processing terminal, inputting information into a predetermined input form, and transmitting the information to the server 10 . The user registration control unit 1041 stores the received information in a new record of the user table 1012, and the user registration is completed. This allows the users stored in the user table 1012 to use the service.
Prior to the registration of user information in the user table 1012 by the user registration control unit 1041, the service provider may perform a predetermined examination to restrict whether or not the user can use the service.
The user ID may be any character string or number that can identify the user, and may be any character string or number desired by the user, or may be automatically set by the user registration control unit 1041. .

The learning unit 1051 executes learning processing for learning hyperparameters to be applied to the image processing model 1021. The learning process will be described later. Further, the learning unit 1051 executes a learning process for learning a classification model.

<Configuration of input device 106 of server 10>
The input device 106 of the server 10 includes a camera 2061, a microphone 2062, a position information sensor 2063, a motion sensor 2064, and a touch device 2065.

<Configuration of output device 108 of server 10>
The output device 108 of the server 10 includes a display 2081 and a speaker 2082.

<Configuration of user terminal 20>
The user terminal 20 is an information processing device operated by a user who uses the service. The user terminal 20 may be, for example, a mobile terminal such as a smartphone or a tablet, a stationary PC (Personal Computer), or a laptop PC. Further, it may be a wearable terminal such as an HMD (Head Mount Display) or a wristwatch type terminal.
The user terminal 20 includes a storage section 201, a control section 204, an input device 206, and an output device 208.

<Configuration of storage unit 201 of user terminal 20>
The storage unit 201 of the user terminal 20 includes a user ID 2011 and an application program 2012.

User ID 2011 is the user's account ID. The user transmits the user ID 2011 from the user terminal 20 to the server 10. The server 10 identifies the user based on the user ID 2011 and provides the service according to the present disclosure to the user. Note that the user ID 2011 includes information such as a session ID temporarily given by the server 10 to identify the user using the user terminal 20.

The application program 2012 may be stored in advance in the storage unit 201, or may be downloaded from a web server operated by a service provider via a communication IF.
Application programs 2012 include applications such as web browser applications.
Application program 2012 includes an interpreted programming language such as JavaScript® that is executed on a web browser application stored on user terminal 20.

<Configuration of control unit 204 of user terminal 20>
The control unit 204 of the user terminal 20 includes an input control unit 2041 and an output control unit 2042. The control unit 204 realizes each functional unit by executing the application program 2012 stored in the storage unit 201.

<Configuration of input device 206 of user terminal 20>
The input device 206 of the user terminal 20 includes a camera 2061, a microphone 2062, a position information sensor 2063, a motion sensor 2064, and a touch device 2065.

<Configuration of output device 208 of user terminal 20>
The output device 208 of the user terminal 20 includes a display 2081 and a speaker 2082.

<Operation of system 1>
Each process of the system 1 will be explained below.
FIG. 7 is a flowchart showing the operation of the abnormal image generation model creation process.
FIG. 8 is a screen example showing the operation of the abnormal image generation model creation process.
FIG. 9 is an example of a screen showing the determination process in the abnormal image generation model creation process.

<Abnormal image generation model creation process>
The abnormal image generation model creation process is a process of creating an abnormal image generation model for generating an abnormal image based on a user's determination instruction for generated data generated based on a normal image. Below, abnormal image generation model creation processing (first embodiment) and abnormal image generation model creation processing (second embodiment) will be described.

<Overview of abnormal image generation model creation process (first embodiment)>
The abnormal image generation model creation process (first embodiment) accepts registration of normal image data related to a product, identifies hyperparameters to be optimized, and creates an image processing model using the hyperparameters for the normal image data. This is a series of processes in which generated data is created by applying , the generated data is presented to the user, a judgment instruction from the user regarding the generated data is received, and hyperparameters are optimized based on the judgment instruction.

<Details of abnormal image generation model creation process (first embodiment)>
Details of the abnormal image generation model creation process (first embodiment) will be described below.

In step S101, the control unit 104 of the server 10 executes an image acquisition step of acquiring a first image including a predetermined product image.
Specifically, by operating the input device 206 of the user terminal 20, the user inputs the URL of the page (image registration processing page) for executing the image registration processing into a web browser, etc., and executes the image registration processing page. open. The control unit 204 of the user terminal 20 transmits a request to the server 10 to open an image registration processing page. The control unit 104 of the server 10 generates an image registration processing page based on the received request and transmits it to the user terminal 20. The control unit 204 of the user terminal 20 displays the received image registration processing page on the display 2081 of the user terminal 20.

By operating the input device 206 of the user terminal 20, the user performs an input operation on a predetermined input form, etc. on the image registration processing page, thereby inputting image data stored in advance in the storage unit 201 of the user terminal 20. Select. Specifically, the image data is image data (normal image data) obtained by photographing the external appearance of a predetermined product in a state where there are no defects or defects related to the product.
The user performs an operation to transmit the selected image data to the server 10 by operating the input device 206 of the user terminal 20 . The control unit 204 of the user terminal 20 transmits the user ID 2011 and the selected image data to the server 10.
The control unit 104 of the server 10 receives and accepts the user ID 2011 and image data from the user terminal 20. The control unit 104 of the server 10 stores and registers the user ID 2011 and image data in the user ID and image data fields of the image table 1013, respectively. Note that by operating the input device 206 of the user terminal 20 and editing a predetermined input field provided on the image registration processing page, the user can store the image data in association with the transmitted image data. You may also enter an image label. The image label input by the user is stored in the image label field of the image table 1013 in association with the image data.
In step S101, the user can register a plurality of normal image data in the server 10. Note that in order to improve the performance of the image processing model, it is desirable to register in the server 10 image data of a predetermined product that has been photographed under different photographing conditions, such as from various angles and under different lighting conditions. .

In step S102, the control unit 104 of the server 10 specifies a plurality of hyperparameters to be applied to the image processing model 1021. The plurality of hyperparameters applicable to the image processing model 1021 are as described in the description of the image processing model 1021.
The control unit 104 of the server 10 sets initial values for the plurality of identified hyperparameters. The control unit 104 of the server 10 sets a random value to each hyperparameter as an initial value. Note that the value set for each hyperparameter is a random value within a predetermined hyperparameter space determined according to the image processing model.

In step S102, the control unit 104 of the server 10 may perform an initial value selection step of selecting an initial value of a predetermined hyperparameter depending on a predetermined product.
The initial value of the predetermined hyperparameter may be set in response to a selection operation by the user. For example, the user may set a predetermined initial value according to the type of product, product name, etc. included in the normal image data registered in step S101.
The initial values of the predetermined hyperparameters are determined by the control of the server 10 depending on the type of product to which the image processing model 1021 is applied (auto parts such as screws, plates, bumpers, etc.) and the product name (model number, product name). It may also be automatically set by the unit 104.
The predetermined hyperparameter initial values may be automatically set by the control unit 104 of the server 10 according to the image data of the product to which the image processing model 1021 is applied.
For example, the control unit 104 of the server 10 may set the initial values of a plurality of hyperparameters by referring to a table (not shown) in which initial values of a plurality of hyperparameters are stored according to the type of product. . The control unit 104 of the server 10 may set only the initial values of some hyperparameters among the plurality of hyperparameters, and may set the initial values of other hyperparameters at random.
The initial value of the hyperparameter may be a hyperparameter value that has been optimized in the past by the abnormal image generation model creation process according to the present disclosure for a predetermined product type or the like. For example, if another user has executed abnormal image generation model creation processing for a predetermined product in the past, the optimized hyperparameter value may be selected as the initial value.
Furthermore, the initial value of the predetermined hyperparameter does not need to be a predetermined value, and may also include a random value within a hyperparameter space defined for a predetermined product type or the like. The control unit 104 of the server 10 may also select arbitrary initial values included in a narrowed range of hyperparameter space based on the type of product or the like.
Thereby, by setting initial values of hyperparameters according to the type of a predetermined product, etc., it is possible to efficiently search for suitable hyperparameters. For example, if hyperparameters that have been optimized in advance already exist for each predetermined product type, the hyperparameters can be efficiently searched for by setting the hyperparameters as initial values.

Note that the initial value of the hyperparameter does not need to be one, and it is possible to set the initial value of a plurality of hyperparameters distributed around a predetermined initial value. The control unit of the server 10 not only sets initial values for one set of multiple hyperparameters in one batch, but also sets initial values in different hyperparameter spaces for multiple sets of multiple hyperparameters. Also good. For example, different initial values may be set for 100 sets of multiple hyperparameters in one batch.

In step S103, the control unit 104 of the server 10 executes an image generation step of generating a second image by applying an image processing model based on predetermined hyperparameters to the first image.
Specifically, the control unit 104 of the server 10 searches the image table 1013 and obtains one or more pieces of normal image data registered in step S101. The control unit 104 of the server 10 applies the image processing model 1021 based on the plurality of hyperparameters whose initial values were set in step S102 to the acquired one or plurality of normal image data. Generate image data (generated data).

In step S103, the image generation step includes extracting a region of a predetermined product image included in the first image, and applying an image processing model based on predetermined hyperparameters to the extracted region of the predetermined product image. generating a second image by doing so.
Specifically, the control unit 104 of the server 10 may execute a process of extracting a region of the product image included in the normal image data before applying the image processing model 1021 to the normal image data. The area of the product image included in the normal image data is determined by calculating the difference between the normal image data to be input and the image data obtained by smoothing (averaging) the plurality of normal image data registered in step S101. It can be extracted by calculation. Further, the region of the product image may be extracted by applying normal image data to a deep learning model or the like that can extract any object.
If the control unit 104 of the server 10 is able to extract the product image region from the normal image data, the control unit 104 performs image processing on the product image region based on a plurality of hyperparameters whose initial values were set in step S102. By applying the model 1021, generated data is generated.
Thereby, when a predetermined product image is included in a part of the first image, it is possible to apply an image processing model for generating an abnormal image only to the predetermined product image.

In step S103, in the image generation step, if a region of a predetermined product image included in the first image cannot be extracted, an image processing model based on predetermined hyperparameters is applied to the entire first image. generating a second image by applying a second image;
If the area of the product image cannot be extracted from the normal image data, the control unit 104 of the server 10 extracts an image based on a plurality of hyperparameters whose initial values were set in step S102 for the entire normal image data. By applying the processing model 1021, generated data is generated.
The control unit 104 of the server 10 determines that the product image region could not be extracted if the extracted product image region spans the entire image or if there is no product image region. Specifically, the control unit 104 of the server 10 can determine whether or not the region of the product image has been extracted based on the pixel values of the image data of the extracted product image.

In step S103, the control unit 104 of the server 10 may collectively apply the image processing model 1021 to the plurality of normal image data registered in step S101 to create generated data.
In step S103, the control unit 104 of the server 10 applies the image processing model 1021 based on the plurality of sets of hyperparameters set in one batch in step S102 to the plurality of normal image data. The product of the number of hyperparameter sets and the number of normal image data) may be created.

The control unit 104 of the server 10 stores the generated data, image identification information of the normal image data when generating the generated data, hyperparameter values, information identifying the applied image processing model 1021, etc. and the generation conditions included in the generation data are stored in the generation data and generation condition fields of the generation table 1014, respectively.

In step S104, the control unit 104 of the server 10 generates a determination processing page including one or more generated data and transmits it to the user terminal 20. The control unit 204 of the user terminal 20 displays the received determination processing page (determination processing screen) on the display 2081 of the user terminal 20.
FIG. 9 is an example of a determination processing screen showing determination processing in the abnormal image generation model creation process. The determination processing screen D20 includes received generated data D201, D202, and D203, determination buttons D211, D212, and D213 that can accept determination instructions for each of the generated data, and input fields D221, D222, and D223.

In step S105, the control unit 104 of the server 10 executes a determination reception step in which a determination instruction regarding whether or not a predetermined product image included in the second image is an abnormal image is received from the user.
Specifically, the user visually checks the generated data D201, D202, and D203 displayed on the display 2081 of the user terminal 20, and determines whether or not the product corresponds to an abnormal image that has a defect or defect. conduct. By operating the input device 206 of the user terminal 20, the user presses the determination button D213 corresponding to the generated data D203 determined to be an abnormal image, thereby displaying the generated ID and determination result corresponding to the generated data D203. information indicating the information is sent to the server 10. The control unit 104 of the server 10 receives and accepts the generation ID and information indicating the determination result.

In step S105, the determination reception step is a determination reception step in which an input operation of an abnormal image type regarding the type of abnormal image of a predetermined product image included in the second image is received from the user.
Specifically, by operating the input device 206 of the user terminal 20, the user obtains information regarding the type and content of the product abnormality (paint peeling, corrosion, color abnormality, etc.) indicated by the abnormality image data included in the generated data D203. ) may be input into input fields D221, D222, and D223. Specifically, instead of or in addition to the determination buttons D211, D212, and D213, input fields D221, D222, and D223 are provided, such as a text input field and a selection field such as a dropdown that allows you to select the type of abnormality. Also good. In this case, the information indicating the determination result includes information regarding the type and content of the product abnormality.

The control unit 104 of the server 10 searches the generation ID item of the generation table 1014 based on the received generation ID, and stores information indicating the received determination result in the determination instruction item of the identified record.

The judgment instruction in step S105 does not need to be given by a specific user, and a configuration may be adopted in which a plurality of different users each give the judgment instruction for each generated data. Any method may be used as long as information indicating the determination result can be stored in association with each generated data. Alternatively, a configuration may be adopted in which a plurality of different users each issue a determination instruction for each of a plurality of sets of hyperparameters.

In step S106, the control unit 104 of the server 10 changes conditions to change predetermined hyperparameters so that the second image becomes an abnormal image of a predetermined product in the image generation step, based on the determination instruction received in the determination reception step. Execute the steps.
Specifically, the control unit 104 of the server 10 changes the value of the hyperparameter set in step S102 based on the determination instruction received in step S105. Specifically, the control unit 104 of the server 10 specifies the hyperparameter search range according to the generation conditions and determination instruction values stored in the generation table 1014.
In the present disclosure, any search method such as grid search or Bayesian optimization can be applied to the hyperparameter search method. In particular, when searching a wide hyperparameter space as in the present disclosure, a hyperparameter search method using Bayesian optimization is suitable.

In step S106, the condition changing step executes a step of changing the predetermined hyperparameter by applying Bayesian optimization based on the predetermined hyperparameter and the determination instruction.

In step S106, the condition changing step executes the step of applying Bayesian optimization based on the accuracy of the classification model that determines whether or not the image is an abnormal image, which is learned based on the second image generated by the image processing model.
The accuracy (including accuracy rate, precision rate, recall rate, f value, F1 score, etc.) of the classification model 1022 may be fed back to the hyperparameter search of the image processing model 1021. Specifically, the accuracy of the classification model 1022 is calculated by dividing the generated data into learning data and verification data, and calculating the prediction accuracy for the verification data.
Specifically, in step S103, the control unit 104 of the server 10 divides the plurality of generated data into datasets of learning data and verification data. The control unit 104 of the server 10 applies the classification model 1022 to the learning data, and outputs a classification label indicating whether or not each piece of generated data is an abnormal image. In step S105, the control unit 104 of the server 10 obtains a determination result regarding whether each of the generated data (learning data and verification data) is an abnormal image. data, correct answer data). The learning unit 1051 of the server 10 executes a learning process for learning the classification model 1022 based on the learning data. The control unit 104 of the server 10 evaluates the prediction accuracy of the classification model 1022 based on the verification data. Specifically, the control unit 104 of the server 10 compares the predicted value output by the classification model 1022 with the determination result (data, correct answer data) for the verification data, and calculates the accuracy of the classification model 1022.
The control unit 104 of the server 10 specifies the hyperparameter search range in the Bayesian optimization in step S106 based on the accuracy calculated for the classification model 1022. For example, the higher the accuracy, the narrower the search range for the hyperparameter (the search range is narrower from the vicinity of the hyperparameter), and the lower the accuracy, the wider the search range for the hyperparameter (the search range is wider from the vicinity of the hyperparameter). range). Thereby, even in a vast hyperparameter space, it is possible to efficiently search the hyperparameter space while minimizing the troublesomeness of the user's determination instructions.

In step S107, the control unit 104 of the server 10 determines whether hyperparameter optimization is completed. Specifically, it may be determined whether the optimization is completed by accepting an input operation from the user. Alternatively, the control unit 104 of the server 10 may verify the quality of the hyperparameters and automatically determine whether hyperparameter optimization has been completed. Alternatively, it may be determined that hyperparameter optimization is completed when certain conditions such as hyperparameter convergence conditions and number of trials are satisfied.
If the control unit 104 of the server 10 determines that the optimization has been completed, the process proceeds to step S108.
If the control unit 104 of the server 10 does not determine that the optimization has been completed, it re-executes the processes from step S103 to step S106 based on the changed hyperparameters. Specifically, in step S103, the hyperparameters changed in step S106 are applied instead of the initial values of the hyperparameters specified in step S102, and the processing contents are the same, so a detailed explanation will be omitted.

The control unit 104 of the server 10 executes hyperparameter search processing based on an algorithm such as Bayesian optimization until it is determined in step S107 that the optimization has been completed. As a result, the values of the hyperparameters are changed and updated so that the generated data generated by the image processing model 1021 becomes abnormal image data of a predetermined product.
In the present disclosure, the control unit 104 of the server 10 uses Bayesian optimization to control areas with high evaluation values (areas of hyperparameters that are determined to generate more abnormal images based on determination instructions) in a vast hyperparameter space. ), it is possible to efficiently search the hyperparameter space.

In step S108, the control unit 104 of the server 10 stores the changed hyperparameters in the storage unit 101 of the server 10. Specifically, it may be stored in association with the type, name, etc. of the product included in the normal image data. Thereby, the user can create abnormal image data by applying the image processing model 1021 based on the hyperparameters to normal image data. This completes the creation of the abnormal image generation model.

<Overview of abnormal image generation model creation process (second embodiment)>
The abnormal image generation model creation process (second embodiment) accepts registration of normal image data regarding a product, and creates an abnormal image generation model for each of a plurality of hyperparameter groups that constitute hyperparameters applicable to the image processing model. Optimization of hyperparameter groups by the creation process (first embodiment) is performed, and after the optimization of each hyperparameter group is completed, the abnormal image generation model creation process (first embodiment) is performed for the combination of multiple hyperparameter groups. This is a series of processes for performing hyperparameter optimization according to one embodiment).

<Details of abnormal image generation model creation process (second embodiment)>
Details of the abnormal image generation model creation process (second embodiment) will be described below.

Step S301 is the same as the abnormal image generation model creation process (first embodiment), so a description thereof will be omitted.

In step S302, the control unit 104 of the server 10 independently performs an image generation step, a determination reception step, and a condition change for each of a plurality of different hyperparameter groups included in a plurality of hyperparameters applicable to the image processing model. Execute an independent processing step that executes the step.
Specifically, the control unit 104 of the server 10 selects and specifies a predetermined hyperparameter group from a plurality of hyperparameter groups applicable to the image processing model 1021.

The control unit 104 of the server 10 executes a selection acceptance step of accepting selection of some hyperparameter groups from among a plurality of different hyperparameter groups.
Specifically, by operating the input device 206 of the user terminal 20, the user may specify a hyperparameter group by selecting or excluding a predetermined hyperparameter group from among the plurality of hyperparameter groups.
As a result, the spatial dimension of hyperparameters can be reduced by excluding one or more hyperparameter groups related to image processing that are not suitable for abnormal image generation of a given product, thereby reducing the search time for suitable hyperparameters. be able to. By using prior knowledge about a given product (product type, product name) (for example, there can be no abnormal images related to "scratches"), suitable hyperparameters can be found by narrowing the hyperparameter search space. You can search efficiently.

The selection receiving step is a step of receiving a selection of some hyperparameter groups from among a plurality of different hyperparameter groups depending on a predetermined product.
The control unit 104 of the server 10 selects or excludes a predetermined hyperparameter group from a plurality of hyperparameter groups according to the product type and product name included in the normal image data registered in step S301. A group of parameters may also be specified.
This makes it possible to narrow the search space for hyperparameters by using prior knowledge (for example, there can be no abnormal images related to "scratches") regarding a given product (product type, product name). Hyperparameters can be searched efficiently.

The control unit 104 of the server 10 executes generation determination independent processing for each of the identified hyperparameter groups. The generation determination independent process is the same as the one or more hyperparameters in steps S102 to S106 of the abnormal image generation model creation process (first embodiment), except that one or more hyperparameters are replaced with the identified hyperparameter group. Therefore, a description of the redundant processing will be omitted.
The generation determination independent process is performed independently for each of the plurality of hyperparameter groups. For example, the generation determination independent process may be executed in parallel for each hyperparameter group, or may be executed sequentially for each hyperparameter group.
Further, similar to the abnormal image generation model creation process (first embodiment), the process may be executed on multiple sets of hyperparameters for each batch at once.

The predetermined hyperparameter group may be selected in response to a selection operation by the user. For example, the user may appropriately select a predetermined hyperparameter group by operating the input device 206 of the user terminal 20, depending on the type of product included in the normal image data registered in step S101.
The predetermined hyperparameter group is configured to be automatically selected by the control unit 104 of the server 10 depending on the type of product to which the image processing model 1021 is applied (auto parts such as screws, plates, bumpers, etc.). Also good. Specifically, the predetermined hyperparameter group may be automatically selected by the control unit 104 of the server 10 based on an analysis result of image data of a product to which the image processing model 1021 is applied. .
For example, the control unit 104 of the server 10 may specify a predetermined hyperparameter group by referring to a table (not shown) or by using an arbitrary deep learning model, machine learning model, artificial intelligence model, etc. can.

In step S302, the image generation step generates a third image by applying an image processing model based on one or more hyperparameters included in the third hyperparameter group among a plurality of hyperparameters applicable to the image processing model. a step of generating
The method includes the step of generating a fourth image by applying an image processing model based on one or more hyperparameters included in a fourth hyperparameter group among a plurality of hyperparameters applicable to the image processing model.
Specifically, the identified plurality of hyperparameter groups are defined as a third hyperparameter group (hyperparameters related to "missing") and a fourth hyperparameter group (hyperparameters related to "distortion"). In this case, in the generation determination independent process (generated data creation process in step S103), by applying the image processing model 1021 based on each of the third hyperparameter group and the fourth hyperparameter group to the normal image data. , third generated data and fourth generated data are generated, respectively. Note that the third generated data and the fourth generated data each include a plurality of generated data.

In step S302, the determination receiving step includes receiving a third determination instruction from the user regarding whether or not a predetermined product image included in the third image is an abnormal image; The method includes the step of receiving a fourth determination instruction regarding whether or not a predetermined product image is an abnormal image.
Specifically, in the generation determination independent process (determination instruction acceptance process in step S105), the user visually checks the third generation data and fourth generation data displayed on the display 2081 of the user terminal 20, and determines the product. A determination is made as to whether or not the image corresponds to an abnormal image having defects, defects, etc. By operating the input device 206 of the user terminal 20, the user presses the determination button corresponding to the third generated data and the fourth generated data determined to be abnormal images. 4. The generation ID corresponding to the generated data and information indicating the determination result are transmitted to the server 10. The control unit 104 of the server 10 receives and accepts the generation ID and information indicating the determination result.
The control unit 104 of the server 10 searches the generation ID item of the generation table 1014 based on the received generation ID, and stores information indicating the received determination result in the determination instruction item of the specified record.

In step S302, the condition changing step includes changing the third hyperparameter group based on the third determination instruction, and changing the fourth hyperparameter group based on the fourth determination instruction.
Specifically, the control unit 104 of the server 10 changes the values of the third hyperparameter group and the fourth hyperparameter group based on the determination instructions received for the third generated data and the fourth generated data, respectively. Specifically, the control unit 104 of the server 10 specifies the search range of the third hyperparameter group and the fourth hyperparameter group according to the generation conditions and the values of the determination instructions stored in the generation table 1014.

In step S303, the control unit 104 of the server 10 executes a completion reception step of receiving an instruction indicating that the independent processing step has been completed.
Specifically, it may be determined whether the optimization of each hyperparameter group is completed by receiving an input operation from the user.
Note that it is determined whether the optimization in step S107 of the abnormal image generation model creation process (first embodiment) is completed for each of the plurality of hyperparameter groups, and the optimization for all hyperparameter groups is performed. If it is determined that the generation determination independent process has been completed, it may be determined that the generation determination independent process has been completed.

In step S303, the control unit 104 of the server 10 executes a selection reception step of accepting selection of some hyperparameter groups from among a plurality of different hyperparameter groups. The selection receiving step is a step of receiving a selection of some hyperparameter groups from among a plurality of different hyperparameter groups depending on a predetermined product.
Specifically, in step S303, the user may select or exclude a predetermined hyperparameter group from among the plurality of hyperparameter groups by operating the input device 206 of the user terminal 20. Further, the control unit 104 of the server 10 may select or exclude a predetermined hyperparameter group from among the plurality of hyperparameter groups according to the content of the determination instruction.
For example, the optimization process may be completed by excluding hyperparameter groups for which abnormal image data could not be sufficiently generated in the generation determination independent process. The hyperparameters included in the excluded hyperparameter group are excluded from the processing targets from step S304 onwards. By selecting or excluding a specific hyperparameter group from a plurality of hyperparameter groups, the search space for hyperparameters can be narrowed, so that suitable hyperparameters can be efficiently searched for.

In step S304, after executing the independent processing step, the control unit 104 of the server 10 performs an image generation step, a determination reception step, and a condition change step for each hyperparameter group that is a combination of a plurality of different hyperparameter groups. Execute a combination processing step that executes
Specifically, after the optimization of a plurality of hyperparameter groups is completed in the generation determination independent process of step S302, the control unit 104 of the server 10 performs a generation determination combination process on hyperparameters that are a combination of multiple hyperparameter groups. Execute. The generation/judgment combination process is the same as in steps S102 to S106 of the abnormal image generation model creation process (first embodiment), except that the hyperparameters are replaced with hyperparameters that are a combination of a plurality of hyperparameter groups. Therefore, a description of the redundant processing will be omitted.

In step S304, the combination processing step executes the steps executed in response to the instruction received in the completion reception step.
Specifically, the generation/judgment combination process may be started upon receiving an input operation from the user indicating that the optimization of each hyperparameter group has been completed in step S303.

In step S304, the combination processing step performs an image generation step, a determination reception step, and a condition change step for each hyperparameter group related to a combination excluding some hyperparameter groups among a plurality of different hyperparameter groups. Execute the steps you want to perform.
Specifically, in step S303, when a predetermined hyperparameter group is selected or excluded from a plurality of hyperparameter groups, the hyperparameters included in the hyperparameter group selected in the generation determination combination process, or Hyperparameter search and optimization are performed on the hyperparameters excluding the excluded hyperparameter group.
This allows the search time for suitable hyperparameters to be reduced.

In step S304, the control unit 104 of the server 10 performs image processing on the first image based on one or more hyperparameters included in a fifth hyperparameter group consisting of a third hyperparameter group and a fourth hyperparameter group. A second image generation step is performed to generate a fifth image by applying the model.
Specifically, in the generation determination combination process (generated data creation process in step S103), a fifth hyperparameter group ("missing" ” and a group of hyperparameters related to “distortion”), fifth generation data is generated. Note that each of the fifth generated data includes a plurality of generated data.

In step S304, the control unit 104 of the server 10 executes a second determination reception step of receiving a fifth determination instruction regarding whether or not a predetermined product image included in the fifth image is an abnormal image from the user.
Specifically, in the generation determination combination process (judgment instruction reception process in step S105), the user visually confirms the fifth generation data displayed on the display 2081 of the user terminal 20, and detects any defects or defects in the product. It is determined whether the image corresponds to an abnormal image having By operating the input device 206 of the user terminal 20, the user presses the determination button corresponding to the fifth generated data determined to be an abnormal image, thereby displaying the generated ID and determination corresponding to the fifth generated data. Information indicating the result is transmitted to the server 10. The control unit 104 of the server 10 receives and accepts the generation ID and information indicating the determination result.

In step S304, the control unit 104 of the server 10, based on the fifth judgment instruction received in the second judgment reception step, sets the fifth hyperlink so that the fifth image becomes an abnormal image of a predetermined product in the second image generation step. A second condition changing step for changing the parameter group is executed.
Specifically, the control unit 104 of the server 10 changes the values of the fifth hyperparameter group based on the determination instruction received for the fifth data. Specifically, the control unit 104 of the server 10 specifies the search range of the fifth hyperparameter group according to the generation conditions and the values of the determination instructions stored in the generation table 1014.

Note that in the present disclosure, as an example, generation determination independent processing is executed for each of the hyperparameter groups that constitute a plurality of hyperparameters, and then generation determination processing is performed for the hyperparameters formed by combining each of the hyperparameter groups. The configuration for executing the judgment combination process has been disclosed as an example. Note that the generation/judgment combination process may be executed in multiple stages for combinations of hyperparameter groups.
For example, a hyperparameter group consisting of 20 components (hyperparameters) is group A (vector consisting of the 0th to 3rd column components), group B (vector consisting of the 4th to 8th components), group C (vector consisting of the 9th column components), and group C (vector consisting of the 4th to 8th components). In a case where there are five types of vectors, group A to E (vectors consisting of the 11th to 11th components), group D (vectors consisting of the 12th to 15th components), and group E (vectors consisting of the 16th to 20th components), Generation determination independent processing is executed for each hyperparameter group, and hyperparameter search and optimization are executed for each of the hyperparameter groups A to E. After that, hyperparameter groups formed by combining Group A and Group B, Group C and Group D (group AB (vector consisting of 0th to 8th components), group CD (vector consisting of 9th to 15th components)) The generation/decision combination processing is executed for each of the groups AB and CD, and the hyperparameter group search and optimization are executed for each of the groups AB and CD. After that, generation/judgment combination processing is performed on the hyperparameter group (group ABCDE (vector consisting of the 0th to 20th components)) formed by combining the groups AB, CD, and E, and the hyperparameter space Search and optimization may be performed for the entire .
Thereby, suitable hyperparameters can be efficiently searched for by narrowing the hyperparameter search space.

<Basic hardware configuration of the computer>
FIG. 10 is a block diagram showing the basic hardware configuration of the computer 90. The computer 90 includes at least a processor 901, a main storage device 902, an auxiliary storage device 903, and a communication IF 991 (interface). These are electrically connected to each other by a communication bus 921.

The processor 901 is hardware for executing an instruction set written in a program. The processor 901 includes an arithmetic unit, registers, peripheral circuits, and the like.

The main storage device 902 is for temporarily storing programs, data processed by the programs, and the like. For example, it is a volatile memory such as DRAM (Dynamic Random Access Memory).

The auxiliary storage device 903 is a storage device for storing data and programs. Examples include flash memory, HDD (Hard Disc Drive), magneto-optical disk, CD-ROM, DVD-ROM, semiconductor memory, and the like.

The communication IF 991 is an interface for inputting and outputting signals for communicating with other computers via a network using a wired or wireless communication standard.
The network is composed of various mobile communication systems constructed using the Internet, LAN, wireless base stations, and the like. For example, the network includes 3G, 4G, 5G mobile communication systems, LTE (Long Term Evolution), a wireless network (for example, Wi-Fi (registered trademark)) that can be connected to the Internet through a predetermined access point, and the like. When connecting wirelessly, communication protocols include, for example, Z-Wave (registered trademark), ZigBee (registered trademark), Bluetooth (registered trademark), and the like. In the case of a wired connection, the network includes a network that is directly connected using a USB (Universal Serial Bus) cable or the like.

Note that the computers 90 can be virtually realized by distributing all or part of each hardware configuration to a plurality of computers 90 and interconnecting them via a network. In this way, the concept of the computer 90 includes not only the computer 90 housed in a single housing or case, but also a virtualized computer system.

<Basic functional configuration of computer 90>
The functional configuration of the computer realized by the basic hardware configuration of the computer 90 (FIG. 10) will be described. The computer includes at least functional units of a control section, a storage section, and a communication section.

Note that the functional units included in the computer 90 can also be implemented by distributing all or part of each functional unit to a plurality of computers 90 interconnected via a network. The computer 90 is a concept that includes not only a single computer 90 but also a virtualized computer system.

The control unit is realized by the processor 901 reading various programs stored in the auxiliary storage device 903, loading them into the main storage device 902, and executing processing according to the programs. The control unit can implement a functional unit that performs various information processing depending on the type of program. Thereby, the computer is realized as an information processing device that performs information processing.

The storage unit is realized by a main storage device 902 and an auxiliary storage device 903. The storage unit stores data, various programs, and various databases. Further, the processor 901 can secure a storage area corresponding to the storage unit in the main storage device 902 or the auxiliary storage device 903 according to the program. Further, the control unit can cause the processor 901 to execute processing for adding, updating, and deleting data stored in the storage unit according to various programs.

A database refers to a relational database, which is used to manage a data set called a master, which is a tabular table whose structure is defined by rows and columns, in relation to each other. In a database, a table is called a table or a master, a table column is called a column, and a table row is called a record. In a relational database, you can set and associate relationships between tables and masters.
Usually, each table and each master is set with a column that serves as a primary key to uniquely identify a record, but it is not essential to set a primary key on a column. The control unit can cause the processor 901 to add, delete, or update records to a specific table or master stored in the storage unit according to various programs.
Further, by storing data, various programs, and various databases in the storage unit, the information processing device and the information processing system according to the present disclosure can be considered as manufactured.

Note that the database and master in the present disclosure may include any data structure (list, dictionary, associative array, object, etc.) in which information is structurally defined. Data structures include data that can be considered as a data structure by combining data with functions, classes, methods, etc. written in any programming language.

The communication unit is realized by a communication IF 991. The communication unit realizes a function of communicating with other computers 90 via a network. The communication unit can receive information transmitted from other computers 90 and input it to the control unit. The control unit can cause the processor 901 to execute information processing on the received information according to various programs. Further, the communication unit can transmit information output from the control unit to another computer 90.

<Additional notes>
The matters explained in each of the above embodiments are additionally described below.

(Additional note 1)
A program to be executed by a computer including a processor and a storage unit, the program includes an image acquisition step (S101) in which the processor acquires a first image including a predetermined product image; an image generation step (S103) of generating a second image by applying an image processing model based on hyperparameters; a determination reception step (S105) that receives a determination instruction; and a condition change that changes predetermined hyperparameters so that the second image becomes an abnormal image of a predetermined product in the image generation step based on the determination instruction received in the determination reception step. A program that executes step (S106).
As a result, even if there is no abnormal image data for a given product or there is an unknown abnormality, it is possible to newly generate or inflate abnormal image data by using the operator's judgment instructions for the abnormal image. can. In addition, there is no need to prepare prior knowledge regarding abnormalities for each product (the product to be visually inspected and the type of abnormality (chips, distortions, scratches, etc.) do not need to be known in advance. ), it is possible to newly generate abnormal image data for various products using the operator's judgment instructions. In other words, various abnormal image data can be generated not only for a specific product but also for various products.
Note that the generated abnormal image data can be used as training data for an abnormality detection model used during product manufacturing.

(Additional note 2)
an independent processing step in which the processor independently executes an image generation step, a determination reception step, and a condition change step for each of a plurality of different hyperparameter groups included in a plurality of hyperparameters applicable to the image processing model; The program according to supplementary note 1 that executes S302).
In the present disclosure, the hyperparameters that can be applied to the image processing model applied to the product image are a plurality of different hyperparameter groups depending on the content of the image processing, such as "chip", "distortion", "scratch", etc. By optimizing each hyperparameter group independently, suitable hyperparameters can be efficiently searched for by narrowing the hyperparameter search space.

(Additional note 3)
After the processor executes the independent processing step, a combination processing step (S304 ) and the program described in Appendix 2 that executes.
As a result, after optimizing each of multiple independent hyperparameter groups, hyperparameters can be searched according to combinations of image processing such as "chips,""distortions," and "scratches." While efficiently searching for parameters, it is possible to appropriately generate abnormal images that combine multiple image processes such as "chips" and "scratches." In other words, it is possible to efficiently search for hyperparameters suitable for generating various abnormal image data not only for a specific product but also for various products.

(Additional note 4)
The processor executes a completion reception step (S303) in which the processor receives an instruction indicating that the independent processing step is completed, and the combination processing step (S304) is a step executed in response to the instruction received in the completion reception step. There is a program described in Appendix 3.
As a result, after optimizing each of multiple hyperparameter groups, hyperparameters can be searched according to combinations of image processing such as "chips,""distortions," and "scratches." By narrowing the search space, suitable hyperparameters can be efficiently searched.

(Appendix 5)
The combination processing step (S304) executes an image generation step, a determination reception step, and a condition change step for each hyperparameter group related to a combination excluding some hyperparameter groups among a plurality of different hyperparameter groups. The program described in Appendix 3, which is the step of
As a result, by excluding one or more hyperparameter groups related to image processing that are not suitable for abnormal image generation of a predetermined product from among a plurality of hyperparameter groups, the search space for hyperparameters is narrowed, thereby selecting suitable hyperparameters. can be searched efficiently.

(Appendix 6)
The processor executes a selection reception step (S303) in which the processor accepts the selection of some hyperparameter groups from a plurality of different hyperparameter groups, and the combination processing step (S304) includes selecting the hyperparameters selected in the selection reception step. A step of executing an image generation step, a determination reception step, and a condition change step for each hyperparameter group related to a combination excluding some hyperparameter groups among a plurality of different hyperparameter groups based on the group. , the program described in Appendix 5.
Thereby, in response to a selection instruction received from a user or automatically performed by a computer, one or more hyperparameter groups related to image processing that are not suitable for generating an abnormal image of a predetermined product are selected from a plurality of hyperparameter groups. By excluding , the search time for suitable hyperparameters can be reduced.
By using prior knowledge about a given product (for example, there can be no abnormal images related to "scratches"), it is possible to efficiently search for suitable hyperparameters by narrowing the hyperparameter search space. .

(Appendix 7)
The program according to appendix 6, wherein the selection acceptance step (S303) is a step of accepting selection of some hyperparameter groups from among a plurality of different hyperparameter groups depending on a predetermined product.
By using prior knowledge about a given product (for example, there can be no abnormal images related to "scratches"), it is possible to efficiently search for suitable hyperparameters by narrowing the hyperparameter search space. .

(Appendix 8)
The image generation step (S302) generates the third image by applying an image processing model based on one or more hyperparameters included in the third hyperparameter group among a plurality of hyperparameters applicable to the image processing model. and a step of generating a fourth image by applying an image processing model based on one or more hyperparameters included in the fourth hyperparameter group among the plurality of hyperparameters applicable to the image processing model. The determination receiving step (S302) includes a step of receiving a third determination instruction from the user regarding whether or not a predetermined product image included in the third image is an abnormal image; The step of accepting a fourth determination instruction regarding whether a predetermined product image included in the image is an abnormal image, and the condition changing step (S302) includes a step of accepting a fourth determination instruction regarding whether or not a predetermined product image included in the image is an abnormal image, and the condition changing step (S302) includes a step of accepting a fourth determination instruction regarding whether or not a predetermined product image included in the image is an abnormal image, and the condition changing step (S302) includes a step of accepting a fourth determination instruction regarding whether or not a predetermined product image included in the image is an abnormal image. and changing the fourth hyperparameter group based on the fourth determination instruction.
In the present disclosure, the hyperparameters that can be applied to the image processing model applied to the product image are a plurality of different hyperparameter groups depending on the content of the image processing, such as "chip", "distortion", "scratch", etc. By optimizing each hyperparameter group independently, suitable hyperparameters can be efficiently searched for by narrowing the hyperparameter search space.

(Appendix 9)
The processor applies, to the first image, an image processing model based on one or more hyperparameters included in a fifth hyperparameter group consisting of a third hyperparameter group and a fourth hyperparameter group, thereby generating a fifth image. a second image generation step (S304) that generates a second image, and a second determination reception step (S304) that receives a fifth determination instruction from the user regarding whether or not a predetermined product image included in the fifth image is an abnormal image. and a second condition changing step of changing the fifth hyperparameter group so that the fifth image becomes an abnormal image of a predetermined product in the second image generating step based on the fifth judgment instruction received in the second judgment receiving step. (S304) and the program according to supplementary note 8.
As a result, after optimizing each of multiple hyperparameter groups, hyperparameters can be searched according to combinations of image processing such as "chips,""distortions," and "scratches." It is possible to appropriately generate an abnormal image by combining multiple image processes such as "chips" and "scratches" while reducing the search time. In other words, it is possible to efficiently search for hyperparameters suitable for generating various abnormal image data not only for a specific product but also for various products.

(Appendix 10)
The image generation step (S103) includes a step of extracting a region of a predetermined product image included in the first image, and applying an image processing model based on predetermined hyperparameters to the extracted region of the predetermined product image. The program according to supplementary note 1, comprising the step of generating a second image by:
Thereby, when a predetermined product image is included in a part of the first image, it is possible to apply an image processing model for generating an abnormal image only to the predetermined product image.

(Appendix 11)
In the image generation step (S103), if a region of a predetermined product image included in the first image cannot be extracted, an image processing model based on predetermined hyperparameters is applied to the entire first image. The program according to appendix 10, which is a step of generating a second image by applying the program.
Thereby, when the predetermined product image is included in the entire first image, the image processing model for generating the abnormal image can be applied to the entire predetermined product image.

(Appendix 12)
The program according to supplementary note 1, wherein the condition changing step (S106) is a step of changing the predetermined hyperparameter by applying Bayesian optimization based on the predetermined hyperparameter and the determination instruction.
As a result, even when the hyperparameter search space is wide, such as generating various abnormal image data not only for a specific product but also for various products as in the present disclosure, suitable hyperparameters can be efficiently determined. can be explored.

(Appendix 13)
Supplementary note 12, the condition changing step (S106) is a step of applying Bayesian optimization based on the accuracy of the classification model that determines whether or not an abnormal image is an abnormal image that is learned based on the second image generated by the image processing model. Programs listed.
This makes it possible to efficiently search for suitable hyperparameters.

(Appendix 14)
The program according to supplementary note 1, wherein the determination reception step (S105) receives an input operation of an abnormal image type regarding the type of abnormal image of a predetermined product image included in the second image from the user.
As a result, not only can the generated abnormal image be judged as ``abnormal'' or ``not abnormal'', but also the types of abnormalities such as ``paint peeling'', ``corrosion'', and ``color abnormality'' can be determined. An abnormal image can be generated accordingly.

(Appendix 15)
The processor executes an initial value selection step (S102) of selecting an initial value of a predetermined hyperparameter according to a predetermined product, and an image generation step (S103) executes an initial value selection step (S102) of selecting an initial value of a predetermined hyperparameter according to a predetermined product. The program according to appendix 1, the step of which is to generate the second image by applying a value-based image processing model.
Thereby, by setting initial values of hyperparameters according to the type of a predetermined product, etc., it is possible to efficiently search for suitable hyperparameters. For example, if hyperparameters that have been optimized in advance already exist for each predetermined product type, the hyperparameters can be efficiently searched for by setting the hyperparameters as initial values.

(Appendix 16)
The image processing model is a model that can apply image processing to an input image to add an abnormal part related to at least one of chipping, distortion, and scratches, and predetermined hyperparameters include the size and rotation of the abnormal part. The program according to appendix 1, including parameters for angle, inversion angle, and color change.
Thereby, it is possible to efficiently generate various abnormal image data not only for a specific product but also for various products.

(Appendix 17)
A method executed by a computer comprising a processor and a memory, wherein the processor executes all steps performed in the invention according to any one of appendices 1 to 16.
As a result, even if there is no abnormal image data for a given product or there is an unknown abnormality, it is possible to newly generate or inflate abnormal image data by using the operator's judgment instructions for the abnormal image. can. In addition, there is no need to prepare prior knowledge regarding abnormalities for each product (the product to be visually inspected and the type of abnormality (chips, distortions, scratches, etc.) do not need to be known in advance. ), it is possible to newly generate abnormal image data for various products using the operator's judgment instructions. In other words, various abnormal image data can be generated not only for a specific product but also for various products.
Note that the generated abnormal image data can be used as training data for an abnormality detection model used during product manufacturing.

(Appendix 18)
An information processing device comprising a control unit and a storage unit, wherein the control unit executes all steps performed in the invention according to any one of Supplementary Notes 1 to 16.
As a result, even if there is no abnormal image data for a given product or there is an unknown abnormality, it is possible to newly generate or inflate abnormal image data by using the operator's judgment instructions for the abnormal image. can. In addition, there is no need to prepare prior knowledge regarding abnormalities for each product (the product to be visually inspected and the type of abnormality (chips, distortions, scratches, etc.) do not need to be known in advance. ), it is possible to newly generate abnormal image data for various products using the operator's judgment instructions. In other words, various abnormal image data can be generated not only for a specific product but also for various products.
Note that the generated abnormal image data can be used as training data for an abnormality detection model used during product manufacturing.

(Appendix 19)
A system comprising means for executing all steps performed in the invention according to any one of appendices 1 to 16.
As a result, even if there is no abnormal image data for a given product or there is an unknown abnormality, it is possible to newly generate or inflate abnormal image data by using the operator's judgment instructions for the abnormal image. can. In addition, there is no need to prepare prior knowledge regarding abnormalities for each product (the product to be visually inspected and the type of abnormality (chips, distortions, scratches, etc.) do not need to be known in advance. ), it is possible to newly generate abnormal image data for various products using the operator's judgment instructions. In other words, various abnormal image data can be generated not only for a specific product but also for various products.
Note that the generated abnormal image data can be used as training data for an abnormality detection model used during product manufacturing.

Reference Signs List 1 system, 10 server, 101 storage unit, 104 control unit, 106 input device, 108 output device, 20 user terminal, 201 storage unit, 204 control unit, 206 input device, 208 output device

Claims (17)

A program to be executed by a computer including a processor and a storage unit, the processor comprising:
an image acquisition step of acquiring a first image including a predetermined product image;
an image generation step of generating a second image by applying an image processing model based on predetermined hyperparameters to the first image;
a determination reception step of receiving a determination instruction from a user regarding whether or not the predetermined product image included in the second image is an abnormal image;
a condition changing step of changing the predetermined hyperparameters so that the second image becomes an abnormal image of the predetermined product in the image generation step, based on the determination instruction received in the determination receiving step;
an independent processing step of independently performing the image generation step, the determination reception step, and the condition change step for each of a plurality of different hyperparameter groups included in a plurality of hyperparameters applicable to the image processing model; and,
A program to run. The processor,
After performing the independent processing step, a combination processing step of performing the image generation step, the determination reception step, and the condition change step for each hyperparameter group that is a combination of each of the plurality of different hyperparameter groups. The program according to claim 1 , which executes the following. The processor,
a completion reception step of receiving an instruction indicating that the independent processing step is completed;
3. The program according to claim 2 , wherein the combination processing step is a step executed in response to the instruction received in the completion acceptance step. The combination processing step includes the image generation step, the determination reception step, and the condition change step for each hyperparameter group related to a combination excluding some hyperparameter groups among the plurality of different hyperparameter groups. The program according to claim 2 , which is a step of executing. The processor,
a selection reception step of accepting selection of some hyperparameter groups from the plurality of different hyperparameter groups;
The combination processing step includes processing the image for each hyperparameter group related to a combination excluding some hyperparameter groups among the plurality of different hyperparameter groups, based on the hyperparameter group selected in the selection reception step. 5. The program according to claim 4 , which is a step of executing a generating step, said determination accepting step, and said condition changing step. 6. The program according to claim 5 , wherein the selection accepting step is a step of accepting selection of some of the plurality of different hyperparameter groups depending on the predetermined product. A program to be executed by a computer including a processor and a storage unit, the processor comprising:
an image acquisition step of acquiring a first image including a predetermined product image;
an image generation step of generating a second image by applying an image processing model based on predetermined hyperparameters to the first image;
a determination reception step of receiving a determination instruction from a user regarding whether or not the predetermined product image included in the second image is an abnormal image;
a condition changing step of changing the predetermined hyperparameters so that the second image becomes an abnormal image of the predetermined product in the image generation step, based on the determination instruction received in the determination receiving step;
Run
The image generation step includes:
generating a third image by applying the image processing model based on one or more hyperparameters included in a third hyperparameter group among a plurality of hyperparameters applicable to the image processing model;
generating a fourth image by applying the image processing model based on one or more hyperparameters included in a fourth hyperparameter group among the plurality of hyperparameters applicable to the image processing model;
including;
The determination reception step includes:
receiving from a user a third determination instruction regarding whether or not the predetermined product image included in the third image is an abnormal image;
receiving from a user a fourth determination instruction regarding whether or not the predetermined product image included in the fourth image is an abnormal image;
including;
The condition changing step includes:
changing the third hyperparameter group based on the third determination instruction;
changing the fourth hyperparameter group based on the fourth determination instruction;
programs containing. The processor,
a fifth hyperparameter group by applying the image processing model based on one or more hyperparameters included in a fifth hyperparameter group consisting of the third hyperparameter group and the fourth hyperparameter group to the first image; a second image generation step of generating an image;
a second determination reception step of receiving a fifth determination instruction from a user regarding whether or not the predetermined product image included in the fifth image is an abnormal image;
changing the fifth hyperparameter group so that the fifth image becomes an abnormal image of the predetermined product in the second image generation step based on the fifth determination instruction received in the second determination receiving step; 8. The program according to claim 7 , which executes the step of changing two conditions. A program to be executed by a computer including a processor and a storage unit, the processor comprising:
an image acquisition step of acquiring a first image including a predetermined product image;
an image generation step of generating a second image by applying an image processing model based on predetermined hyperparameters to the first image;
a determination reception step of receiving a determination instruction from a user regarding whether or not the predetermined product image included in the second image is an abnormal image;
a condition changing step of changing the predetermined hyperparameters so that the second image becomes an abnormal image of the predetermined product in the image generation step, based on the determination instruction received in the determination receiving step;
Run
In the image generation step, if a region of the predetermined product image included in the first image cannot be extracted, the image generation step includes generating the image based on the predetermined hyperparameters for the entire first image. generating the second image by applying a processing model;
program. The image generation step includes:
extracting a region of the predetermined product image included in the first image;
By applying the image processing model based on the predetermined hyperparameter to the extracted region of the predetermined product image when the region of the predetermined product image included in the first image can be extracted. generating the second image;
The program according to claim 9 , comprising: A program to be executed by a computer including a processor and a storage unit, the processor comprising:
an image acquisition step of acquiring a first image including a predetermined product image;
an image generation step of generating a second image by applying an image processing model based on predetermined hyperparameters to the first image;
a determination reception step of receiving a determination instruction from a user regarding whether or not the predetermined product image included in the second image is an abnormal image;
a condition changing step of changing the predetermined hyperparameters so that the second image becomes an abnormal image of the predetermined product in the image generation step, based on the determination instruction received in the determination receiving step;
Run
The condition changing step is based on the predetermined hyperparameter, the determination instruction, and the accuracy of a classification model that determines whether the image is an abnormal image and is learned based on the second image generated by the image processing model. , changing the predetermined hyperparameters by applying Bayesian optimization;
program. A program to be executed by a computer including a processor and a storage unit, the processor comprising:
an image acquisition step of acquiring a first image including a predetermined product image;
an image generation step of generating a second image by applying an image processing model based on predetermined hyperparameters to the first image;
a determination reception step of receiving a determination instruction from a user regarding whether or not the predetermined product image included in the second image is an abnormal image;
a condition changing step of changing the predetermined hyperparameters so that the second image becomes an abnormal image of the predetermined product in the image generation step, based on the determination instruction received in the determination receiving step;
Run
The determination receiving step is a step of receiving an input operation for an abnormal image type regarding the type of abnormal image of the predetermined product image included in the second image from the user.
program. The processor,
performing an initial value selection step of selecting an initial value of the predetermined hyperparameter according to the predetermined product;
The image generation step is a step of generating the second image by applying the image processing model based on the initial value selected in the initial value selection step.
The program according to any one of claims 1 to 12 . The image processing model is a model that can apply image processing to an input image to add an abnormal location related to at least one of a chip, a distortion, and a scratch, and the predetermined hyperparameter is a including parameters for rotation angle, flip angle and color change.
The program according to any one of claims 1 to 12 . A method implemented in a computer comprising a processor and a memory, wherein the processor performs all the steps performed in the invention according to any of claims 1 to 12 . An information processing device comprising a control section and a storage section, wherein the control section executes all steps executed in the invention according to any one of claims 1 to 12. A system comprising means for performing all steps performed in the invention according to any of claims 1 to 12 .