WO2024075251A1

WO2024075251A1 - Data generation system, industrial machine, data generation method, and data generation program

Info

Publication number: WO2024075251A1
Application number: PCT/JP2022/037462
Authority: WO
Inventors: 浩貴太刀掛; 智大元田
Original assignee: 株式会社安川電機
Priority date: 2022-10-06
Filing date: 2022-10-06
Publication date: 2024-04-11

Abstract

This data generation system comprises an acquisition unit that acquires an evaluation value corresponding to an evaluation of a target object, and a data generation unit that generates pseudo data representing a pseudo object corresponding to the acquired evaluation value on the basis of the acquired evaluation value and a trained model trained to receive input of an evaluation value to output pseudo data representing a target object.

Description

DATA GENERATION SYSTEM, INDUSTRIAL MACHINERY, DATA GENERATION METHOD, AND DATA GENERATION PROGRAM

One aspect of the present disclosure relates to a data generation system, an industrial machine, a data generation method, and a data generation program.

Patent Document 1 describes an information processing system that trains a pseudo image generation model, which is a machine learning model that converts a first domain image, which is an image in a first domain of an object to be recognized, into a pseudo second domain image, which is an image similar to a second domain image, which is an image in a second domain.

JP 2020-95364 A

There is a need for a mechanism to easily generate pseudo-data of objects.

A data generation system according to one aspect of the present disclosure includes an acquisition unit that acquires an evaluation value corresponding to an evaluation of an object, and a data generation unit that generates pseudo data representing a pseudo object corresponding to the acquired evaluation value based on the acquired evaluation value and a trained model that has been trained to output pseudo data representing an object when an evaluation value is input.

A data generation method according to one aspect of the present disclosure is a data generation method executed by a data generation system having at least one processor, and includes the steps of acquiring an evaluation value corresponding to an evaluation of an object, and generating pseudo data representing a pseudo object corresponding to the acquired evaluation value based on the acquired evaluation value and a trained model trained to output pseudo data representing an object when an evaluation value is input.

A data generation program according to one aspect of the present disclosure causes a computer to execute the steps of acquiring an evaluation value corresponding to an evaluation of an object, and generating pseudo data representing a pseudo object corresponding to the acquired evaluation value based on the acquired evaluation value and a trained model that has been trained to output pseudo data representing an object when an evaluation value is input.

According to one aspect of the present disclosure, pseudo data of an object can be easily generated.

FIG. 1 is a diagram illustrating an example of an application and a functional configuration of a data generation system. FIG. 2 is a diagram illustrating an example of a hardware configuration of a computer used in the data generation system. FIG. 1 is a diagram illustrating an example of a machine learning model (trained model). FIG. 1 is a diagram showing an overview of a learning phase. 13 is a flowchart illustrating an example of a learning phase. FIG. 1 is a diagram showing an overall view of the operation phase. 13 is a flowchart illustrating an example of an operation phase.

Below, an embodiment of the present disclosure will be described in detail with reference to the attached drawings. In the description of the drawings, identical or equivalent elements are given the same reference numerals, and duplicate descriptions will be omitted.

[System Overview]
A data generation system according to the present disclosure is a computer system that generates pseudo data representing a pseudo object. The data generation system acquires an evaluation value corresponding to an evaluation of the object, and generates the pseudo data based on the evaluation value and a trained model that is at least partially generated by machine learning.

An object refers to a tangible object of some kind. In one example, an object is a tangible object of variable appearance. "Various appearance" means that the individual appearances are not exactly the same. An example of a tangible object of variable appearance is fresh produce, so the object may be a type of vegetable, fruit, fish, or meat. A pseudo object refers to a virtual object that is generated on a computer system to resemble a real object. Pseudo data is electronic data that represents a pseudo object. The pseudo data may be an image that depicts the object, and in this disclosure, the image is referred to as a pseudo image. The pseudo data may be model data that shows the three-dimensional shape of the object, and in this disclosure, the model data is referred to as a pseudo three-dimensional model.

The evaluation of an object refers to the value of the object judged qualitatively. In one example, the evaluation of an object is a classification of the appearance. For example, the classification is determined comprehensively based on various elements of the object's appearance, such as color, size, shape, gloss, etc. The evaluation of an object may be related to quality. The evaluation value refers to a value that quantitatively indicates the evaluation of the object. In one example, the evaluation value indicates the quality of the object. For example, the evaluation value is represented by a continuous value or a discrete value. Regardless of whether the evaluation value is a continuous value or a discrete value, the evaluation value may indicate a classification of the appearance or quality of the object. For example, the classification may be a class (grade) such as rank A, B, C, etc. When the evaluation value is a continuous value, for example, the class "Rank C" may correspond to an evaluation value of 0 or more and less than 1, the class "Rank B" may correspond to an evaluation value of 1 or more and less than 2, and the class "Rank A" may correspond to an evaluation value of 2 or more and less than 3. When the evaluation value is a discrete value, the correspondence between the discrete value and the class may be 1:1 or N:1.

Machine learning is a method of autonomously finding laws or rules by repeatedly learning based on given information. A trained model is constructed using algorithms and data structures. At least a portion of the trained model is generated by machine learning. Generative Adversarial Networks (GAN) may be used as a machine learning architecture for generating pseudo data. By using a trained model, the task of generating pseudo data can be quantified from evaluation values, which are qualitative indicators.

In one example, the data generation system generates pseudo data based on an evaluation value input by a user and a trained model. When a user specifies an evaluation value based on an ambiguous judgment based on their own intuition or tacit knowledge, the data generation system generates pseudo data corresponding to that evaluation value. The data generation system can automate the generation of pseudo data involving such ambiguous judgments. In addition, the user can compare the generated pseudo data with the ambiguous judgment based on their own intuition or tacit knowledge, and specify or change the evaluation value or its range so that pseudo data closer to their own judgment is generated.

[System Configuration]
FIG. 1 is a diagram showing an example of application and functional configuration of a data generation system 10 according to an example. In this example, the data generation system 10 includes a model generation unit 11, an acquisition unit 12, and a data generation unit 13 as functional modules. The model generation unit 11 is a functional module that generates at least a part of a trained model 20. The trained model 20 is a computational model trained to output pseudo data in response to an input of an evaluation value. The acquisition unit 12 is a functional module that acquires an evaluation value corresponding to an evaluation of an object. The data generation unit 13 is a functional module that generates pseudo data representing a pseudo object corresponding to the evaluation value based on the acquired evaluation value and the generated trained model 20. The "pseudo object corresponding to the evaluation value" refers to a pseudo object generated so as to be determined to have a value indicated by the evaluation value. The generation of the trained model 20 by the model generation unit 11 corresponds to a learning phase. The use of the trained model 20 by the acquisition unit 12 and the data generation unit 13 corresponds to an operation phase or an inference phase.

The data generation system 10 can be realized by any type of computer. The computer may be a general-purpose computer such as a personal computer or a business server, or may be incorporated into a dedicated device that executes specific processing. The data generation system 10 may be realized by a single computer, or may be realized by a distributed system having multiple computers.

In one example, the data generation system 10 connects to the first database 30 and the second database 40 via a communication network.

The first database 30 is a device that stores a training dataset used to generate the trained model 20. In one example, the training dataset includes a plurality of records corresponding to a plurality of real objects. Each record includes data items necessary for generating the trained model 20. For example, each record includes an RGB image, a mask image, a three-dimensional model, an evaluation value, and a class for a given real object. The RGB image represents the projected shape and texture of the object. The mask image represents the two-dimensional shape of the object. In one example, the mask image is a binary image in which pixel values representing the object are a first color (e.g., white) and other pixel values are a second color (e.g., black). The three-dimensional model represents the three-dimensional shape and texture of the object. Texture refers to elements other than shape that characterize an object. For example, texture is represented by color, gloss, etc.

The second database 40 is a device that stores the pseudo data generated by the data generation system 10. In one example, the second database 40 stores at least one of a pseudo image and a pseudo three-dimensional model for each pseudo object.

In the example of FIG. 1, the pseudo data is used to operate the industrial machine 60. The second database 40 is connected to the control model generating device 50 via a communication network. The control model generating device 50 is a computer that generates a control model 70 by executing machine learning based on the pseudo data stored in the second database 40. The control model 70 learned based on the pseudo data has a function of, for example, evaluating a real object corresponding to the pseudo object and estimating a control value related to the evaluation. The control model 70 is provided to the industrial machine 60 via a communication network. The industrial machine 60 is a device that executes a predetermined process based on the control model 70. The industrial machine 60 can be any device such as a judger, a robot controller, or a robot. In one example, the industrial machine 60 includes an output unit 61 that outputs a control value by the control model 70. For example, the output unit 61 may output the control value to another device such as a monitor or a robot.

The communication network connecting the devices may be a wired network or a wireless network. The communication network may be configured to include at least one of the Internet and an intranet. Alternatively, the communication network may be realized simply by a single communication cable.

FIG. 2 is a diagram showing an example of the hardware configuration of a computer 100 used in the data generation system 10. In this example, the computer 100 includes a main unit 110, a monitor 120, and an input device 130.

The main body 110 is a device that executes the main functions of the computer. The main body 110 has a circuit 160. The circuit 160 has at least one processor 161, a memory 162, a storage 163, an input/output port 164, and a communication port 165. The storage 163 records programs for configuring each functional module of the main body 110. The storage 163 is a computer-readable recording medium such as a hard disk, a non-volatile semiconductor memory, a magnetic disk, or an optical disk. The memory 162 temporarily stores programs loaded from the storage 163, the results of calculations by the processor 161, and the like. The processor 161 configures each functional module by executing programs in cooperation with the memory 162. The input/output port 164 inputs/outputs electrical signals between the monitor 120 or the input device 130 in response to instructions from the processor 161. The input/output port 164 may input/output electrical signals between other devices. The communication port 165 performs data communication with other devices via the communication network N in response to instructions from the processor 161.

Monitor 120 is a device for displaying information output from main body 110. Monitor 120 may be any device capable of displaying graphics, a specific example of which is a liquid crystal panel.

The input device 130 is a device for inputting information to the main body 110. The input device 130 may be any device capable of inputting desired information, and specific examples include operation interfaces such as a keypad, a mouse, and an operation controller.

The monitor 120 and the input device 130 may be integrated as a touch panel. For example, the main body 110, the monitor 120, and the input device 130 may be integrated as in a tablet computer.

Each functional module of data generation system 10 is realized by loading a data generation program onto processor 161 or memory 162 and having processor 161 execute the program. The data generation program includes code for realizing each functional module of data generation system 10. Processor 161 operates input/output port 164 or communication port 165 in accordance with the data generation program, and executes reading and writing of data in memory 162 or storage 163.

The data generation program may be provided in a state where it is permanently recorded on a non-transitory recording medium such as a CD-ROM, DVD-ROM, or semiconductor memory. Alternatively, the data generation program may be provided via a communications network as a data signal superimposed on a carrier wave.

[Trained model]
3 is a diagram showing an example of a machine learning model 200 used to generate the trained model 20. The machine learning model 200 can be said to be a pre-complete trained model 20. In one example, the machine learning model 200 includes a conversion model 210, a data generation model 220, and an evaluation model 230.

The conversion model 210 is a computational model that converts the evaluation value into a latent variable. The conversion model 210 can be said to be a mapping from the evaluation value to the latent variable. The latent variable is data indicating n-dimensional features for generating pseudo data. In one example, the conversion model 210 includes a converter 211 and an adder 212. The converter 211 is a component that converts the evaluation value into a latent variable. The adder 212 is a component that applies another latent variable Z _s set by a probability distribution such as a uniform distribution to the converted latent variable. The conversion model 210 outputs a latent variable to which the other latent variable Z _s has been applied.

The data generation model 220 is a computational model that generates pseudo data based on the latent variables generated by the transformation model 210. In one example, the data generation model 220 includes a shape generation model 221, a texture generation model 223, and a three-dimensional shape generation model 224. The shape generation model 221 is a computational model that outputs a mask image 301 that indicates the shape of an object based on the latent variables generated by the transformation model 210. The texture generation model 223 is a computational model that outputs a pseudo image 302 that represents the projected shape and texture of an object based on the mask image 301 and a latent variable Z _t that is set by a probability distribution such as a uniform distribution. The pseudo image 302 may be an RGB image. The three-dimensional shape generation model 224 is a computational model that outputs a pseudo three-dimensional model 303 that represents the three-dimensional shape and texture of a pseudo object based on the pseudo image 302.

The evaluation model 230 is a computational model that calculates an evaluation value based on pseudo data. In this disclosure, in relation to the learning phase, the evaluation value input to the conversion model 210 is also referred to as the "first evaluation value," and the evaluation value output from the evaluation model 230 is referred to as the "second evaluation value," to distinguish between the two types of evaluation values as necessary.

In Figure 3, a strawberry is shown as an example of an object. Mask image 301 represents the two-dimensional shape of the strawberry, pseudo image 302 represents the projected shape and texture of the strawberry, and pseudo 3D model 303 represents the three-dimensional shape and texture of the strawberry.

[System Operation]
(Generating a trained model)
Machine learning is performed on at least a part of the machine learning model 200 to generate a trained model 20. Fig. 4 is a diagram showing an overview of the training phase.

FIG. 4 shows the feature space 410 and latent space 420 in addition to the transformation model 210, the data generation model 220, and the evaluation model 230. The feature space 410 is a coordinate space that defines the evaluation value. In the example of FIG. 4, each point in the feature space 410 indicates an evaluation value that changes continuously. That is, the evaluation value in this example is expressed as a continuous value. In the example of FIG. 4, these evaluation values are classified into three classes Qa, Qb, and Qc. The latent space 420 is a coordinate space that defines the latent variables. Each point in the latent space 420 indicates a latent variable.

The data generation model 220 is generated by machine learning using individual latent variables and a training dataset. The evaluation model 230 is generated by machine learning using a training dataset. The conversion model 210 is generated by machine learning using evaluation values. In training the conversion model 210, the conversion model 210 converts the input first evaluation value into a latent variable, the data generation model 220 generates pseudo data based on the latent variable, and the evaluation model 230 outputs a second evaluation value based on the pseudo data. The evaluation model 230 is updated based on the first evaluation value and the second evaluation value in the feature space 410.

FIG. 5 is a flowchart showing an example of the learning phase as processing flow S1. That is, the data generation system 10 executes processing flow S1.

In step S11, the model generation unit 11 generates the shape generation model 221. In one example, the model generation unit 11 generates the shape generation model 221 by machine learning (e.g., machine learning using GAN) based on multiple latent variables and multiple mask images of a training dataset.

In step S12, the model generation unit 11 generates the texture generation model 223. In one example, the model generation unit 11 generates the texture generation model 223 by machine learning (e.g., machine learning using a GAN) based on multiple RGB images and mask images of the training dataset.

In step S13, the model generation unit 11 generates the three-dimensional shape generation model 224. In one example, the model generation unit 11 generates the three-dimensional shape generation model 224 by machine learning (e.g., machine learning using a GAN) based on multiple RGB images and multiple three-dimensional models of the training dataset.

As shown in steps S11 to S13, the model generation unit 11 generates a data generation model 220 to output pseudo data based on latent variables. In this process, by introducing machine learning using GAN, it becomes possible to generate a data generation model 220 that generates pseudo data of pseudo objects that are likely to exist in reality.

In step S14, the model generation unit 11 generates the evaluation model 230. In one example, the model generation unit 11 generates the evaluation model 230 by machine learning based on multiple RGB images and multiple evaluation values of the training dataset.

In step S15, the model generation unit 11 generates the conversion model 210. In one example, the model generation unit 11 completes the conversion model by executing the following process.

The model generation unit 11 inputs the first evaluation value into the uncompleted conversion model 210 to generate a latent variable. The model generation unit 11 generates a pseudo image (e.g., a pseudo RGB image) corresponding to the first evaluation value based on the latent variable and the data generation model 220. The model generation unit 11 inputs the latent variable into the data generation model 220. In the data generation model 220, processing is executed using the shape generation model 221 and the texture generation model 223. The model generation unit 11 obtains a pseudo image obtained by the processing. The model generation unit 11 inputs the pseudo image into the evaluation model 230 to calculate a second evaluation value. The model generation unit 11 updates the uncompleted conversion model 210 based on the first evaluation value and the second evaluation value.

The model generation unit 11 executes this series of processes for each of the multiple first evaluation values, and finally generates the conversion model 210. For example, the model generation unit 11 generates the conversion model 210 so that the first evaluation value and the second evaluation value match. Alternatively, the model generation unit 11 may generate the conversion model 210 so that the error between the first evaluation value and the second evaluation value is less than a predetermined threshold. In this way, by repeatedly updating the uncompleted conversion model 210 while comparing the second evaluation value with the first evaluation value, and finally generating the conversion model 210, the correspondence between the first evaluation value and the second evaluation value becomes one-to-one. In other words, the conversion model 210 becomes able to output latent variables for generating pseudo data corresponding to the specified evaluation value.

In process flow S1, the model generation unit 11 may generate the conversion model 210 after generating the data generation model 220. That is, the model generation unit 11 may execute steps S11 to S13 and then execute step S15. The model generation unit 11 may generate the evaluation model 230 after generating the data generation model 220, and then generate the conversion model 210 using the data generation model 220 and the evaluation model 230. That is, the model generation unit 11 may execute steps S11 to S13 and then execute step S14, and execute step S15 after executing step S14.

The trained model 20 is obtained by the process flow S1. Please note that the trained model 20 is a computational model that is estimated to be optimal, and is not necessarily a "computational model that is actually optimal."

(Generating pseudo data)
In one example, a combination of the conversion model 210 and the data generation model 220 obtained by the learning phase is provided as a trained model 20. The trained model 20 may or may not include an evaluation model 230. In the operation phase (inference phase), pseudo data is generated using the trained model 20. Fig. 6 is a diagram showing an overall view of the operation phase.

In the operation phase, an evaluation value 411 for generating pseudo data is input to the trained model 20. In this disclosure, the evaluation value 411 is also referred to as a specified evaluation value. Naturally, the evaluation value 411 is a value specified in the feature space 410. In the trained model 20, the conversion model 210 converts the evaluation value 411 into a latent variable 421 in the latent space 420. Next, the data generation model 220 generates at least one of the pseudo image 302 and the pseudo 3D model 303 as pseudo data based on the latent variable 421.

FIG. 7 is a flowchart showing an example of the operation phase as processing flow S2. That is, the data generation system 10 executes processing flow S2.

In step S21, the acquisition unit 12 acquires an evaluation value (specified evaluation value). In one example, the acquisition unit 12 acquires an evaluation value input by a user. Alternatively, the acquisition unit 12 may receive an evaluation value transmitted from another computer, or may read an evaluation value stored in a specified storage device.

In step S22, the data generation unit 13 converts the evaluation value into a latent variable. The data generation unit 13 generates a latent variable based on the evaluation value and the conversion model 210. The data generation unit 13 inputs the evaluation value into the conversion model 210 and obtains the latent variable output from the conversion model 210.

In step S23, the data generation unit 13 generates a mask image corresponding to the evaluation value based on the latent variables and the shape generation model 221. The data generation unit 13 inputs the latent variables to the shape generation model 221 and obtains the mask image output from the shape generation model 221.

In step S24, the data generation unit 13 generates a pseudo image corresponding to the evaluation value based on the mask image and the texture generation model 223. The data generation unit 13 inputs the mask image 301 and the latent variable _Zt to the texture generation model 223 and obtains a pseudo image output from the texture generation model 223.

In step S25, the data generation unit 13 generates a pseudo three-dimensional model corresponding to the evaluation value based on the pseudo image and the three-dimensional shape generation model 224. The data generation unit 13 inputs the pseudo image to the three-dimensional shape generation model 224 and obtains the pseudo three-dimensional model output from the three-dimensional shape generation model 224.

In step S26, the data generation unit 13 stores the pseudo 3D model in the second database 40. As described above, the pseudo 3D model is used to generate the control model 70.

In step S27, the user adjusts the evaluation value or its range as necessary. The user compares the generated pseudo data (pseudo image or pseudo 3D model) with his/her own judgment based on intuition or tacit knowledge. The user then adjusts the evaluation value or its range so that pseudo data that is closer to his/her own judgment is generated. For example, the user specifies or changes the evaluation value or its range. Since the user makes the adjustment as necessary, step S27 may be omitted.

Process flow S2 can be executed repeatedly. Through this repeated processing, multiple pseudo 3D models of a certain type of object (e.g., strawberries) are accumulated in the second database 40.

The control model generating device 50 performs machine learning based on the pseudo three-dimensional model stored in the second database 40 to generate a control model 70. The industrial machine 60 outputs a control value based on the control model 70.

The pseudo three-dimensional model in the second database 40 may be used for other purposes. For example, the data generation system 10 may display the pseudo three-dimensional model on the monitor 120 in the form of a computer graphic (CG) video or still image.

As shown in steps S22 to S25, the data generation unit 13 generates a pseudo three-dimensional model based on the latent variables and the data generation model 220. The data generation unit 13 inputs the latent variables to the data generation model 220 and obtains a pseudo three-dimensional model output from the data generation model 220. Therefore, in this example, the data generation unit 13 generates a pseudo three-dimensional model as pseudo data.

In step S26, the data generation unit 13 may store a pseudo image in the second database 40 in addition to or instead of the pseudo three-dimensional model. That is, the data generation unit 13 may generate at least one of the pseudo image and the pseudo three-dimensional model as pseudo data.

[Modification]
Various examples of the present disclosure have been described in detail above. However, the present disclosure is not limited to these examples. Various modifications of the present disclosure are possible without departing from the spirit and scope of the present disclosure.

The texture generation model may output a pseudo image based on the latent variable obtained by the transformation model in addition to the mask image and the latent variable _Zt . In this case, the data generation unit inputs the mask image, the latent variable _Zt , and the latent variable obtained by the transformation model to the texture generation model, and obtains a pseudo image output from the texture generation model. Alternatively, the texture generation model may output a pseudo image based on the latent variable obtained by the transformation model without using the mask image. In this example, the shape generation model is omitted. The data generation unit inputs the latent variable obtained by the transformation model to the texture generation model, and obtains a pseudo image output from the texture generation model.

The conversion model may be generated without using machine learning. For example, a function that converts the evaluation value into a latent variable so that the first evaluation value and the second evaluation value match, or so that the error between the two evaluation values is less than a predetermined threshold, may be prepared as the conversion model.

The data generation model does not have to include a three-dimensional shape generation model. In this case, the data generation unit generates a pseudo image as pseudo data.

The data generation system does not need to include a model generation unit. Trained models are portable between computer systems. Therefore, the data generation unit may use a trained model generated in another computer system.

The hardware configuration of the system is not limited to a configuration in which each functional module is realized by executing a program. For example, at least a portion of the functional modules described above may be configured with a logic circuit specialized for that function, or may be configured with an ASIC (Application Specific Integrated Circuit) that integrates the logic circuit.

The processing steps of the method executed by at least one processor are not limited to the above examples. For example, some of the steps or processes described above may be omitted, or the steps may be executed in a different order. Also, two or more of the steps described above may be combined, or some of the steps may be modified or deleted. Alternatively, other steps may be executed in addition to the steps described above.

When comparing the magnitude of two numbers within a computer system or computer, you can use either of the two criteria of "greater than or equal to" and "greater than", or you can use either of the two criteria of "less than or equal to" and "less than".

[Additional Notes]
As can be seen from the various examples above, the present disclosure includes the following aspects.
(Appendix 1)
an acquisition unit that acquires an evaluation value corresponding to an evaluation of an object;
a data generation unit that generates pseudo data representing a pseudo object corresponding to the acquired evaluation value based on the acquired evaluation value and a trained model that has been trained to output pseudo data representing the object when the evaluation value is input; and
A data generation system comprising:
(Appendix 2)
the object is an object with an indefinite appearance,
The evaluation is a classification of the appearance.
2. The data generation system of claim 1.
(Appendix 3)
The acquisition unit acquires the evaluation value input by a user.
3. The data generation system according to claim 1 or 2.
(Appendix 4)
The trained model includes a data generation model that outputs the pseudo data based on a latent variable,
The data generation unit
converting the evaluation values into latent variables;
generating the pseudo data based on the latent variables and the data generation model;
4. A data generation system according to any one of claims 1 to 3.
(Appendix 5)
Further comprising a model generation unit that generates at least a part of the trained model,
The model generation unit
generating the data generation model to output the pseudo data based on the latent variables;
After generating the data generation model, a conversion model is generated to convert the evaluation value into the latent variable.
5. The data generation system of claim 4.
(Appendix 6)
The model generation unit
After generating the data generation model, an evaluation model is generated to calculate the evaluation value based on the pseudo data;
after generating the evaluation model, generating the transformation model using the data generation model and the evaluation model;
6. The data generation system of claim 5.
(Appendix 7)
The model generation unit, in generating the conversion model,
inputting the first evaluation value into the uncompleted transformation model to generate the latent variables;
generating the pseudo data corresponding to the first evaluation value based on the latent variables and the data generation model;
inputting the pseudo data into the evaluation model to calculate a second evaluation value;
updating the uncompleted transformation model based on the first evaluation value and the second evaluation value to generate the transformation model;
7. The data generation system of claim 6.
(Appendix 8)
The data generation unit generates a pseudo image as the pseudo data.
8. A data generation system according to any one of claims 1 to 7.
(Appendix 9)
The data generation unit generates a pseudo three-dimensional model as the pseudo data.
9. A data generation system according to any one of claims 1 to 8.
(Appendix 10)
The data generation model is
a shape generation model that outputs a mask image indicating the shape of the object based on the latent variables;
a texture generation model that outputs the pseudo data representing the shape and texture of the object based on the mask image;
Equipped with
The data generation unit
generating the mask image corresponding to the obtained evaluation value based on the latent variables and the shape generation model;
generating the pseudo data based on the generated mask image and the texture generation model;
10. A data generation system according to any one of claims 4 to 9.
(Appendix 11)
An industrial machine comprising an output unit that outputs a control value according to a control model trained on the basis of pseudo data generated by the data generation system according to any one of appendixes 1 to 10.
(Appendix 12)
1. A data generation method performed by a data generation system comprising at least one processor, comprising:
obtaining an evaluation value corresponding to an evaluation of the object;
generating the pseudo data representing a pseudo object corresponding to the acquired evaluation value based on the acquired evaluation value and a trained model trained to output pseudo data representing the object when the evaluation value is input;
A data generation method comprising:
(Appendix 13)
obtaining an evaluation value corresponding to an evaluation of the object;
generating the pseudo data representing a pseudo object corresponding to the acquired evaluation value based on the acquired evaluation value and a trained model trained to output pseudo data representing the object when the evaluation value is input;
A data generation program that causes a computer to execute the above.

According to

Supplementary Notes

1, 12, and 13, pseudo data representing a pseudo object corresponding to an evaluation value is automatically obtained based on a trained model, so that pseudo data of the object can be easily generated.

According to Appendix 2, pseudo data is generated for objects with variations in appearance, where classification of appearance is impossible or very difficult to quantify. The configuration of Appendix 2 makes it possible to generate pseudo data that corresponds to assessments that are impossible or difficult to quantify.

According to Appendix 3, pseudo data can be generated that corresponds to an evaluation based on a qualitative judgment by the user (e.g., an ambiguous judgment based on the user's intuition).

According to Appendix 4, by using latent variables, it is possible to generate diverse pseudo-data from evaluation values with limited dimensions or ranges.

According to Appendix 5, by generating a conversion model after generating a data generation model, a highly accurate trained model can be achieved.

According to Appendix 6, by adopting an evaluation model, in addition to the evaluation values used to obtain the latent variables, evaluation values calculated by the evaluation model are obtained. By using these two types of evaluation values, a highly accurate conversion model can be realized.

According to Supplementary Note 7, the conversion model is updated based on the first evaluation value input to generate the pseudo data and the second evaluation value calculated based on the generated pseudo data. This update can improve the accuracy of the conversion model for obtaining latent variables.

According to Appendix 8, it is easy to generate a pseudo-image of the object.

According to Appendix 9, it is easy to generate a pseudo 3D model of the object.

According to Appendix 10, since the shape and texture are generated using separate computational models, it is possible to more reliably generate pseudo data that represents pseudo objects that are likely to exist in reality.

According to Appendix 11, since pseudo data can be easily obtained, learning of a control model for operating industrial machinery can also be easily performed. As a result, industrial machinery that outputs desired control values can be easily realized.

10...data generation system, 11...model generation unit, 12...acquisition unit, 13...data generation unit, 20...trained model, 30...first database, 40...second database, 50...control model generation device, 60...industrial machine, 61...output unit, 70...control model, 200...machine learning model, 210...conversion model, 220...data generation model, 221...shape generation model, 223...texture generation model, 224...3D shape generation model, 230...evaluation model, 301...mask image, 302...pseudo image, 303...pseudo 3D model.

Claims

an acquisition unit that acquires an evaluation value corresponding to an evaluation of an object;
a data generation unit that generates pseudo data representing a pseudo object corresponding to the acquired evaluation value based on the acquired evaluation value and a trained model that has been trained to output pseudo data representing the object when the evaluation value is input; and
A data generation system comprising:
the object is an object with an indefinite appearance,
The evaluation is a classification of the appearance.
The data generation system of claim 1 .
The acquisition unit acquires the evaluation value input by a user.
The data generation system of claim 1 .
The trained model includes a data generation model that outputs the pseudo data based on a latent variable,
The data generation unit
converting the evaluation values into latent variables;
generating the pseudo data based on the latent variables and the data generation model;
The data generation system according to any one of claims 1 to 3.
Further comprising a model generation unit that generates at least a part of the trained model,
The model generation unit
generating the data generation model to output the pseudo data based on the latent variables;
After generating the data generation model, a conversion model is generated to convert the evaluation value into the latent variable.
The data generation system of claim 4.
The model generation unit
After generating the data generation model, an evaluation model is generated to calculate the evaluation value based on the pseudo data;
after generating the evaluation model, generating the transformation model using the data generation model and the evaluation model;
The data generation system of claim 5 .
The model generation unit, in generating the conversion model,
inputting the first evaluation value into the uncompleted transformation model to generate the latent variables;
generating the pseudo data corresponding to the first evaluation value based on the latent variables and the data generation model;
inputting the pseudo data into the evaluation model to calculate a second evaluation value;
updating the uncompleted transformation model based on the first evaluation value and the second evaluation value to generate the transformation model;
The data generation system of claim 6.
The data generation unit generates a pseudo image as the pseudo data.
The data generation system according to any one of claims 1 to 3.
The data generation unit generates a pseudo three-dimensional model as the pseudo data.
The data generation system according to any one of claims 1 to 3.
The data generation model is
a shape generation model that outputs a mask image indicating the shape of the object based on the latent variables;
a texture generation model that outputs the pseudo data representing the shape and texture of the object based on the mask image;
Equipped with
The data generation unit
generating the mask image corresponding to the obtained evaluation value based on the latent variables and the shape generation model;
generating the pseudo data based on the generated mask image and the texture generation model;
The data generation system of claim 4.
An industrial machine having an output unit that outputs a control value based on a control model trained on the basis of pseudo data generated by the data generation system according to any one of claims 1 to 3.
1. A data generation method performed by a data generation system comprising at least one processor, comprising:
obtaining an evaluation value corresponding to an evaluation of the object;
generating the pseudo data representing a pseudo object corresponding to the acquired evaluation value based on the acquired evaluation value and a trained model trained to output pseudo data representing the object when the evaluation value is input;
A data generation method comprising:
obtaining an evaluation value corresponding to an evaluation of the object;
generating the pseudo data representing a pseudo object corresponding to the acquired evaluation value based on the acquired evaluation value and a trained model trained to output pseudo data representing the object when the evaluation value is input;
A data generation program that causes a computer to execute the above.