JP7457879B1 - Data generation method, program, data generation device, and trained model - Google Patents

Abstract

A data generation method for improving accuracy of a trained model suitable for transfer learning for image classification, the method comprising: generating, by a computer, an image in which a three-dimensional fractal is rendered based on a shape of the three-dimensional fractal and rendering parameters that set virtual light reflection in the three-dimensional fractal arranged in a three-dimensional space; generating label information that identifies the shape and the rendering parameters of the three-dimensional fractal; and recording a set of the label information and the image as data.

Description

The present invention relates to a data generation method, a program, a data generation device, and a learned model.

Image classification using machine learning is used for various purposes. A trained model for image classification is constructed, for example, based on learning data that is a set of an image and a label given to the image. A trained model for image classification according to the purpose is generated by, for example, performing transfer learning or fine tuning using a base model that is an existing trained model.

The base model is, for example, a trained model that is trained using a large-scale image dataset (ImageNet, etc.) that has many pairs of images and labels given to the images. ImageNet is an open dataset that has been released to the public, but images are randomly collected from the Internet, and there are concerns over, for example, copyright and portrait rights regarding these images.

Non-Patent Document 1 discloses a method of generating images based on mathematical formulas to train a trained model for image classification, instead of images collected from the Internet.

R. Yamada et al., MV-FractalDB: Formula-driven Supervised Learning for Multi-view Image Recognition, “2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)”, 2021, pp. 2076-2083

In the method described in Non-Patent Document 1, images are generated when a three-dimensional fractal generated based on a mathematical formula is viewed from various viewpoints. The resulting images do not include, for example, the unevenness or three-dimensionality of a three-dimensional fractal, and therefore differ greatly from images captured in the real world that serve as input for transfer learning. Therefore, when using the generated images as a dataset for generating a base model for transfer learning, it is difficult to extract the features required to classify images captured in the real world, and there is room for improvement in the accuracy of the base model.

Therefore, an object of the present invention is to improve the accuracy of a trained model suitable for transfer learning for image classification.

In a data generation method according to one aspect of the present invention, a computer generates three-dimensional data based on the shape of a three-dimensional fractal and rendering parameters that set virtual light reflection in a three-dimensional fractal arranged in a three-dimensional space. The method includes generating an image in which a dimensional fractal is rendered, generating label information that identifies the shape and rendering parameters of the three-dimensional fractal, and recording a combination of the label information and the image as data.

According to this aspect, the virtual light reflection in a three-dimensional fractal arranged in a three-dimensional space is set as a rendering parameter, and an image in which the three-dimensional fractal is rendered is generated. By setting the light reflection during rendering, an image including a three-dimensional fractal can be made to show a state closer to an image in the real world. This makes it possible to extract more features necessary for image classification, and generates data that can improve the accuracy of the base model.

In the above aspect, the rendering parameters may include object texture information for setting the texture on the surface of the three-dimensional fractal. By including texture information in the rendering, it is possible to generate a three-dimensional fractal image that can be used to classify images that have information on the texture of the surface of an object.

In the above aspect, the object texture information may include information for setting the color of the three-dimensional fractal. By including the color information of the model in rendering, it is possible to generate a three-dimensional fractal image that can be used to classify images that have color information.

In the above aspect, the rendering parameters may include viewpoint information for setting a viewpoint for the three-dimensional fractal in three-dimensional space. By making it possible to set the viewpoint, the image including the three-dimensional fractal can be made to show a state closer to the image in the real world.

In the above aspect, the rendering parameters may include light source information for setting a light source to be placed in the three-dimensional space. By making it possible to set the light source, the image including the three-dimensional fractal can be made to show a state closer to the image in the real world.

In the above aspect, the data generation method may further include the computer generating a three-dimensional fractal shape based on object parameters that set the three-dimensional fractal shape. According to this aspect, by setting the object parameters, the computer can generate various shapes. This makes it possible to increase the variety of images that include three-dimensional fractals, and makes it easier to extract features required for image classification.

A program according to another aspect of the present invention causes a computer to generate a three-dimensional fractal based on the shape of a three-dimensional fractal and rendering parameters for setting virtual light reflection in a three-dimensional fractal arranged in a three-dimensional space. Generating an image in which a fractal is rendered and generating label information that identifies the shape and rendering parameters of the three-dimensional fractal are executed.

A data generation device according to another aspect of the present invention generates a three-dimensional fractal based on the shape of the three-dimensional fractal and rendering parameters that set virtual light reflection in the three-dimensional fractal arranged in a three-dimensional space. an image generation unit that generates a rendered image, a label information generation unit that generates label information that identifies the shape of the three-dimensional fractal and rendering parameters, and a data recording unit that records a combination of label information and image as data. and.

A trained model according to another aspect of the present invention is trained using training data in which a first image in which a first three-dimensional fractal is rendered is input, and first label information identifying the shape of the first three-dimensional fractal and rendering parameters of the first three-dimensional fractal is output, and causes a computer to function in response to an input of a second image in which a second three-dimensional fractal is rendered, and to output second label information identifying the shape of the second three-dimensional fractal and rendering parameters of the second three-dimensional fractal.

According to the present invention, it is possible to improve the accuracy of a trained model suitable for transfer learning for image classification.

FIG. 1 is a block diagram showing the configuration of a data generation device according to the present embodiment. FIG. 2 is a block diagram showing a hardware configuration of the data generating device according to the present embodiment. 7 is a flowchart of data generation processing according to the present embodiment. It is an example of the parameter list based on this embodiment. 4 is an example of a plurality of images according to the present embodiment. This is an example of a plurality of images according to the present embodiment. This is an example of a plurality of images according to the present embodiment. This is another example of a plurality of images according to the present embodiment. This is another example of a plurality of images according to the present embodiment. This is another example of a plurality of images according to the present embodiment. 13 is another example of a plurality of images according to the embodiment. This is another example of a plurality of images according to the present embodiment. 13 is another example of a plurality of images according to the embodiment. This is another example of a plurality of images according to the present embodiment. This is another example of a plurality of images according to the present embodiment. This is another example of a plurality of images according to the present embodiment. 13 is an example of generated data according to the embodiment. 7 is a flowchart of base model generation processing according to the present embodiment.

Preferred embodiments of the present invention will be described with reference to the accompanying drawings. In addition, in each figure, those with the same reference numerals have the same or similar configurations.

FIG. 1 shows a block diagram of a data generation device 101 according to this embodiment. The data generation device 101 includes a display section 1011 , a storage section 1012 , a setting acquisition section 1013 , a shape generation section 1014 , an image generation section 1015 , a label information generation section 1016 , and a data recording section 1017 . Each part of the data generation device 101 can be realized by a processor executing a program stored in a storage device in an information processing device such as a computer having a hardware configuration described below.

The display unit 1011 is a display of the data generation device 101. The storage unit 1012 stores various types of information used in processing by the data generation device 101. Furthermore, data generated by the data generation device 101 is stored in the storage unit 1012.

The setting acquisition unit 1013 acquires object parameters for setting the shape of a three-dimensional fractal and rendering parameters for setting conditions for rendering the three-dimensional fractal. The setting acquisition unit 1013 acquires, for example, a large number of object parameters and rendering parameters generated by the data generation device 101 and stored in the storage unit 1012.

The object parameters include the position and orientation of a three-dimensional fractal placed in a three-dimensional space, the type of three-dimensional fractal, the number of iterations in fractal calculation, and fractal parameters. The rendering parameters are parameters for setting virtual light reflection in a three-dimensional fractal arranged in a three-dimensional space.

The shape generation unit 1014 generates a three-dimensional fractal shape based on the object parameters. The shape generation unit 1014 generates, for example, a Mandelbulb, which is a three-dimensional fractal generated based on the Mandelbrot set.

The shape generation unit 1014 obtains, for example, parameters used to generate a Mandel bulb as object parameters. Further, the shape generation unit 1014 generates a point in a three-dimensional space generated based on the object parameters as a shape.

In this embodiment, the data generating device 101 is described as generating a three-dimensional fractal shape, but the shape does not necessarily have to be generated by the data generating device 101. For example, an information processing device different from the data generating device 101 may generate a three-dimensional fractal shape based on object parameters, and the data generating device 101 may acquire the shape generated in advance.

The image generation unit 1015 renders a three-dimensional fractal in a three-dimensional space based on the shape and rendering parameters, and generates an image of the rendered three-dimensional fractal.

Rendering parameters include, for example, model color information for setting the color of the 3D fractal, object texture information for setting the texture on the surface of the 3D fractal, and viewpoint (camera position) for the 3D fractal in the 3D space. This information includes at least one of viewpoint information for setting, and light source information for setting a light source arranged in a three-dimensional space.

The label information generation unit 1016 generates label information that identifies the shape and rendering parameters of the three-dimensional fractal. Label information is, for example, a set of object parameters and rendering parameters, and is information that allows individual images to be identified. Alternatively, the label information may be a set of a shape and a rendering parameter, for example. Note that the label information does not have to be provided in a one-to-one correspondence with respect to a set of object parameters and rendering parameters. For example, the same label information may be given to common three-dimensional fractals rendered from different viewpoints. Alternatively, for example, the same label information may be given to common three-dimensional fractals rendered based on different textures.

The data recording unit 1017 records the label information generated by the label information generation unit 1016 in the storage unit 1012 in association with the image generated by the image generation unit 1015.

An example of the hardware configuration of the data generation device 101 will be described with reference to FIG. 2. The data generation device 101 includes a processor 201 such as a CPU (Central Processing Unit) and a GPU (Graphical Processing Unit), a memory, a storage device 202 such as an HDD (Hard Disk Drive) and/or an SSD (Solid State Drive), and a wired or wireless It has a communication IF (Interface) 203 for communicating, an input device 204 for accepting input operations, and an output device 205 for outputting information. The processor 201, the storage device 202, the communication IF 203, the input device 204, and the output device 205 are connected in the data generation device 101 so that data can be exchanged. The input device 204 is, for example, a keyboard, a touch panel, a mouse, or the like. The output device 205 is, for example, a display.

The display unit 1011 can be realized using the output device 205 included in the data generation device 101. The storage unit 1012 can be realized using the storage device 202 included in the data generation device 101. The setting acquisition unit 1013, shape generation unit 1014, image generation unit 1015, label information generation unit 1016, and data recording unit 1017 are realized by the processor 201 of the data generation device 101 executing a program stored in the storage device 202. can do. Further, the program can be stored in a storage medium. The storage medium storing the program may be a non-transitory computer readable medium. The non-temporary storage medium is not particularly limited, and may be, for example, a storage medium such as an HDD, SSD, or USB memory.

Processing in the data generation device 101 will be described with reference to FIGS. 3 to 9. FIG. 3 shows a flowchart of processing in the data generation device 101.

In step S301, the setting acquisition unit 1013 acquires object parameters from the user operating the data generation device 101. A plurality of object parameters may be acquired at once.

In step S302, the setting acquisition unit 1013 acquires rendering parameters from the user. A plurality of rendering parameters may be acquired at once.

In step S303, the setting acquisition unit 1013 generates a parameter list and stores it in the storage unit 1012. FIG. 4 shows an example of a parameter list.

The parameter list has items of "object parameters" and "rendering parameters." The "rendering parameters" include, for example, "object texture," "background texture," "viewpoint," "light source," and "rendering settings."

In the "Object Texture" item, for example, information is set that indicates the texture (feel) of the three-dimensional fractal according to the material of the three-dimensional fractal. The information stored in the "Object Texture" item may also include information that indicates the color of the three-dimensional fractal (e.g., the color of the surface). The state of light reflection in the three-dimensional fractal, etc. changes according to the information set in the "Object Texture".

In the "background texture" item, for example, information indicating the texture of the background portion, which corresponds to the material of the background portion that is a portion other than the three-dimensional fractal in the three-dimensional space, is set. Further, the information stored in the "background texture" item may include information indicating the color of the background portion. Ambient light, background color, etc. change depending on the information set in "background texture".

The "Viewpoint" item contains viewpoint information to set the viewpoint (camera position) for the 3D fractal placed in the 3D space when imaging the 3D fractal placed in the 3D space. Set. The viewpoint information includes, for example, the angle of view of a camera arranged in a three-dimensional space, the position of the camera, the attitude of the camera, and the like. By making it possible to set the viewpoint, it is possible to make the feature amounts of data different even if the images are based on the same three-dimensional fractal.

In the "light source" item, light source information is set in order to set a light source arranged in a three-dimensional space. The light source information is, for example, information indicating the position of the light source in a three-dimensional space, the type of the light source, the intensity of the light source, and the like. By making it possible to set the light source, it is possible to make the data feature values different even for images based on the same three-dimensional fractal.

The "rendering settings" item includes information indicating the size of the generated image and the accuracy of the image. Here, the image accuracy is information for setting how finely a three-dimensional fractal is to be drawn in a three-dimensional fractal that can be drawn in infinite detail.

In the parameter list, each parameter is set as a fixed value, a discrete value within a certain range, or a continuous value within a certain range. For example, the size of the image is set to a fixed value. Further, for example, the type of fractal and the parameter of the fractal in "object parameter" are set as one of discrete values (or continuous values) within a certain range. Further, for example, the position of the camera and the position of the light source are set as one of continuous values within a certain range. When generating the parameter list, the setting acquisition unit 1013 may set random values based on a seed for randomly setting discrete values or continuous values within a predetermined range.

In the example of FIG. 4, a plurality of rendering parameters are set for the object parameter "XXX1". Further, a plurality of rendering parameters are also set for the object parameter “XXX2”. In this way, the data generation device 101 can generate large-scale data sets based on mathematical formulas by rendering three-dimensional fractals that have different shapes and shapes depending on different object parameters based on different rendering parameters. Become.

In addition, when the data generating device 101 does not generate a shape but acquires a shape from another information processing device, the acquired shape may be set in the parameter list instead of the object parameters.

In step S304, the shape generation unit 1014 refers to the parameter list and generates a three-dimensional fractal shape based on a certain object parameter. Note that if the data generation device 101 does not generate a shape but acquires a shape from another information processing device, the process of step S304 is skipped.

In step S305, the image generation unit 1015 renders the three-dimensional fractal based on the rendering parameters, and generates an image including the three-dimensional fractal.

5A, 5B, and 5C respectively show images I11, I12, and I13 that are generated based on a shape based on a certain object parameter and different rendering parameters. Images I11, I12, and I13 are images generated by rendering a situation in which a common three-dimensional fractal F1 is illuminated by light sources located at different light source positions. For example, it can be seen that the locations indicated by arrows A1, A2, and A3 reflect light differently. In this way, the data generation device 101 can generate images having different characteristics from a common three-dimensional fractal.

In each of FIGS. 6A, 6B, and 6C, images 21, 22, and 23 are generated based on shapes based on object parameters different from those in FIGS. 5A, 5B, and 5C, and based on different rendering parameters, respectively. are shown respectively. Images I21, I22, and I23 are images generated by rendering a situation in which a common three-dimensional fractal F2 is illuminated by light sources located at different light source positions. 7A, 7B, and 7C also show images I31, I32, and I33 that are generated based on a shape based on a certain object parameter and different rendering parameters, respectively. Images I31, I32, and I33 are images generated by rendering a situation in which a common three-dimensional fractal F3 is illuminated by light sources located at different light source positions.

8A, 8B, and 8C respectively show images I41, I42, and I43 that are generated based on shapes based on object parameters common to those in FIG. 5 and rendering parameters that are different from each other. Images I41, I42, and I43 are images generated by rendering a situation in which the three-dimensional fractal F1 shown in FIG. 5 is illuminated by light sources at different light source positions, and by zooming a part of the three-dimensional fractal F1. be. It can also be seen in FIGS. 8A, 8B, and 8C that the locations indicated by arrows A4, A5, and A6 in each figure are illuminated in different ways.

By using a characteristic 3D fractal with a periodic structure, even when a part of the 3D fractal is enlarged, it is possible to identify differences in the state of light reflection that occur based on differences in the detailed structure. A possible image is generated. Thereby, the data generation device 101 can generate images having different characteristics from a common three-dimensional fractal. Therefore, the data generation device 101 can generate images that can extract smaller differences as feature quantities, and the accuracy of the learning model trained using the images generated by the data generation device 101 is further improved.

In step S306, the label information generating unit 1016 generates label information that identifies the shape and rendering parameters of the three-dimensional fractal. The label information generating unit 1016 generates the label information based on, for example, the object parameters and rendering parameters used to generate the image.

In step S307, the data recording unit 1017 records the label information in association with the image in the storage unit 1012. FIG. 9 shows an example of a data set recorded by the data recording unit 1017.

The data set has items of "data ID", "image", and "label information". Information identifying each data is stored in the "data ID" item. Each image generated by the image generation unit 1015 is stored in the “Image” item. The label information generated by the label information generation unit 1016 is stored in the “label information” item. For example, data whose "data ID" is "ID101" is data in which image 11 in FIG. 5 is associated with label information based on the object parameter "XXX" and rendering parameter "YYY1" used to generate image 11. be.

In step S308, the data recording unit 1017 determines whether or not the pairs of label information and images have been recorded based on all parameters in the parameter list. If a negative determination is made in step S308, the processes from step S304 to S307 are repeated based on different rendering parameters or different object parameters, and the images are recorded. If a positive determination is made in step S308, the processing by the data generating device 101 ends.

With reference to FIG. 10, generation of a trained model using a dataset generated by the data generation device 101 will be described. The trained model can be generated using any information processing device.

In step S1001, the information processing apparatus refers to the data set generated by the data generation apparatus 101. In step S1002, the information processing apparatus performs machine learning to output correct label information for each input image in the data set. In step S1003, the information processing device stores the trained base model in, for example, a storage unit of the information processing device. Thereby, by using the data set generated by the data generation device 101, learning is performed based on more feature amounts, and a base model suitable for transfer learning is generated.

The embodiments described above are intended to facilitate understanding of the present invention, and are not intended to be interpreted as limiting the present invention. Each element included in the embodiment, as well as its arrangement, conditions, shape, size, etc., are not limited to those illustrated, and can be changed as appropriate. Further, it is possible to partially replace or combine the structures shown in different embodiments.

101...Data generation device, 1011...Display section, 1012...Storage section, 1013...Setting acquisition section, 1014...Shape generation section, 1015...Image generation section, 1016...Label information generation section, 1017...Data recording section

Claims (9)

The computer is
Generating an image in which the three-dimensional fractal is rendered based on the shape of the three-dimensional fractal and rendering parameters that set virtual light reflection in the three-dimensional fractal arranged in a three-dimensional space;
generating label information identifying the shape of the three-dimensional fractal and the rendering parameters;
A data generation method comprising: recording a plurality of sets of the label information and the image as a data set used for machine learning of a learning model that outputs the label information in response to input of the image . 2. The data generation method according to claim 1,
A data generation method, wherein the rendering parameters include object texture information for setting a texture on a surface of the three-dimensional fractal. The data generation method according to claim 2,
The object texture information includes information for setting a color of the three-dimensional fractal. The data generation method according to claim 1,
The rendering parameter includes viewpoint information for setting a viewpoint for the three-dimensional fractal in the three-dimensional space. The data generation method according to claim 1,
The data generation method, wherein the rendering parameters include light source information for setting a light source arranged in the three-dimensional space. The data generation method according to claim 1,
The data generation method further comprises: the computer generating the shape of the three-dimensional fractal based on object parameters that set the shape of the three-dimensional fractal. to the computer,
Generating an image in which the three-dimensional fractal is rendered based on the shape of the three-dimensional fractal and rendering parameters that set virtual light reflection in the three-dimensional fractal arranged in a three-dimensional space;
generating label information that identifies the shape of the three-dimensional fractal and the rendering parameters;
A program that causes a program to record a plurality of pairs of the label information and the image as a data set used for machine learning of a learning model that outputs the label information in response to input of the image. an image generation unit that generates an image in which the three-dimensional fractal is rendered based on the shape of the three-dimensional fractal and rendering parameters that set virtual light reflection in the three-dimensional fractal arranged in a three-dimensional space; and,
a label information generation unit that generates identifying label information that identifies the three-dimensional fractal shape and the rendering parameters;
a data recording unit that records a plurality of sets of the label information and the image as a data set used for machine learning of a learning model that outputs the label information in response to input of the image; Device. A first image in which a first three-dimensional fractal is rendered is input, and the shape of the first three-dimensional fractal and the first three-dimensional fractal arranged in the three-dimensional space of the first three-dimensional fractal are trained using training data whose output is first label information identifying rendering parameters for setting virtual light reflection ;
For inputting a second image in which a second three-dimensional fractal is rendered, the shape of the second three-dimensional fractal and the second three-dimensional fractal arranged in the three-dimensional space of the second three-dimensional fractal. A trained model that causes a computer to function to output second label information that identifies rendering parameters that configure virtual light reflection in the fractal .

Images