JP2021154570A - Region constitution prediction method and region constitution prediction device - Google Patents

Abstract

To provide a region constitution prediction method and a region constitution prediction device, that can appropriately predict a region constitution for obtaining a desired texture.SOLUTION: The region constitution prediction method for predicting the region constitution formed by using materials for coloring multiple colors comprises: a texture prediction stage (S204) that predicts relationship between a predetermined texture and a laminated constitution in which a plurality of colored layered regions are overlapped by computer simulation; a learning stage (S206) that generates a learned model, which is a learning model that learned relationship between the laminated constitution and the texture, by machine learning; and a region constitution prediction stage that predicts the laminated constitution corresponding to the texture using a learned model generated in the learning stage, and predicts the region constitution corresponding to the desired texture based on the laminated constitution.SELECTED DRAWING: Figure 5

Description

The present invention relates to a region configuration prediction method and a region configuration prediction device.

Conventionally, a modeling device (3D printer) for modeling a modeled object using an inkjet head is known (see, for example, Patent Document 1). In such a modeling device, for example, by stacking a plurality of layers of ink formed by an inkjet head, a modeled object is modeled by a layered manufacturing method.

Japanese Unexamined Patent Publication No. 2015-071282

When modeling a modeled object with a modeling device, the texture of the modeled object may differ depending on how the regions (for example, colored layered regions) formed by the modeling material (for example, ink) overlap. For example, when multiple types of inks of different colors are used as modeling materials and areas of various colors are overlapped to express a color, the texture such as transparency on the surface of the modeled object differs depending on how the areas are overlapped. May occur.

Therefore, in such a case, it is desired to model the modeled object so that the regions of the modeling material overlap with each other in a configuration that can obtain the desired texture of the user (for example, a designer). Then, in this case, for example, it is desired to appropriately predict a configuration in which a desired texture can be obtained with respect to how the regions are overlapped (region configuration). Therefore, an object of the present invention is to provide a region configuration prediction method and a region configuration prediction device that can solve the above problems.

In order to determine the region configuration in which the desired texture can be obtained at the time of modeling the modeled object, it is usually necessary to perform trial and error by actually modeling the modeled object or the like. However, since it takes a lot of time to model a modeled object, if such trial and error is required, the time and effort required to determine the configuration of the modeled object will be greatly increased.

On the other hand, the inventor of the present application does not repeat trial and error every time each modeled object is modeled, but causes a computer to machine-learn the relationship between the laminated structure and the texture corresponding to the structure of the modeled object in advance. , I thought about outputting the corresponding laminated structure by inputting the desired texture. Further, in this case, by creating a plurality of samples in which layers of modeling materials are laminated with different configurations and measuring the samples corresponding to a predetermined texture, the relationship between the laminated configuration and the texture can be determined. I thought about letting a computer perform machine learning to acquire and learn the relationship. In addition, it has been found that by actually conducting various experiments and the like, it is possible to appropriately determine the region configuration corresponding to the desired texture by such a method. Furthermore, with respect to such a method, the applicant of the present application has filed an application as Japanese Patent Application No. 2019-63003.

When the region configuration is determined by such a method, for example, after the machine learning is completed, the region configuration can be appropriately determined in a short time. However, even in this case, a lot of time and effort is required at the stage of acquiring the data necessary for machine learning. More specifically, for example, when machine learning is performed by the above method, it is usually necessary to prepare a large number of samples and perform measurement on each sample. In this case, a lot of labor and time are required at the stage of preparing and measuring the sample.

On the other hand, the inventor of the present application considered to predict the relationship between the laminated composition and the texture by computer simulation without preparing a large number of samples by further diligent research. Further, by actually conducting various experiments and the like, it was found that the region configuration corresponding to the desired texture can be appropriately determined even by such a method.

In addition, the inventor of the present application has found the features necessary for obtaining such an effect through further diligent research, and has reached the present invention. In order to solve the above problems, the present invention is a region configuration prediction method for predicting a region configuration which is a configuration of a region formed by using a material for coloring of a plurality of colors, and a colored layered region is used. It is a stage of predicting the relationship between the laminated structure which is a plurality of overlapping structures and a predetermined texture by computer simulation, and the plurality of types of the laminated structures in which the layered regions overlap with each other are expressed by the respective laminated structures. A texture prediction stage for predicting the texture, and a stage for generating a trained model, which is a learning model in which the relationship between the laminated configuration and the texture is learned, is generated by machine learning. A learning step of generating the trained model based on the texture predicted in the texture prediction step and a step of predicting the region configuration corresponding to the desired texture, and the trained trained model generated in the learning step. It is characterized by including a region configuration prediction stage for predicting the laminated configuration corresponding to the texture using a model and predicting the region configuration corresponding to the desired texture based on the laminated configuration.

With this configuration, for example, the relationship between the laminated configuration and the texture can be appropriately predicted by computer simulation. Further, by generating a trained model in which the relationship between the laminated structure and the texture is learned by machine learning for the laminated structure of various configurations, the laminated structure corresponding to the desired texture can be appropriately predicted. Further, based on the predicted laminated structure, the area structure for obtaining a desired texture can be appropriately predicted. Further, in this case, for example, a large number of data (learning data) necessary for machine learning can be appropriately prepared without actually preparing a large number of samples or performing measurement on a large number of samples. Further, as a result, for example, the time and effort required to generate the trained model can be appropriately reduced.

In this configuration, as the material for coloring a plurality of colors, for example, it is conceivable to use a material for modeling a plurality of colors. In this case, the region configuration can be considered, for example, a configuration corresponding to at least a part of the region of the modeled object to be modeled by the modeling apparatus that models the modeled object using the modeling materials of a plurality of colors. Further, the region configuration can be considered, for example, a configuration in which a plurality of layered regions, each of which is colored, overlap in a direction parallel to the normal direction on the surface of the modeled object. With this configuration, for example, the region configuration can be appropriately predicted based on the laminated configuration. Further, this makes it possible to appropriately predict, for example, the composition of the modeled object for obtaining a desired texture.

It is also conceivable to use a material other than the modeling material as the coloring material. In this case, for example, ink or the like used in a printing apparatus that prints on a medium can be considered as a material for coloring. Further, in this case, in the printing apparatus, for example, it is conceivable to express various textures by stacking a plurality of layers of inks of various colors on the medium. Then, in this case, the way in which the ink layers formed on the medium are overlapped can be considered as a region configuration. Further, as the printing apparatus, it is conceivable to use a printing apparatus (so-called 2.5D printer) that forms a three-dimensional shape by superimposing an ink layer on the medium. In this case, for example, the configuration of the three-dimensionally shaped surface formed on the medium can be considered as the region configuration.

Further, in this configuration, in the learning stage, for example, a trained model is generated by causing a computer to perform machine learning. Further, as the texture, for example, it is conceivable to use a transparency expressed by overlapping a plurality of layered regions. In this case, in the texture prediction stage, for example, it is conceivable to use a function indicating how the light spreads as a parameter indicating the texture. Further, more specifically, as a parameter indicating such a texture, for example, a line spreading function (LSF) or the like can be preferably used. With such a configuration, for example, it is possible to appropriately predict a laminated configuration corresponding to a desired transparency.

Further, in this configuration, the region configuration prediction stage includes, for example, a parameter calculation stage and a laminated configuration acquisition stage. In this case, the parameter calculation step is, for example, a step of calculating a setting parameter which is a parameter corresponding to the texture set in the computer graphics image based on the computer graphics image in which the desired texture is set. .. Further, the laminated configuration acquisition stage is, for example, a stage of acquiring an output indicating the laminated configuration by using the setting parameters calculated in the setting parameter calculation stage as inputs to the trained model. With such a configuration, for example, it is possible to appropriately accept the designation of a desired texture and appropriately output the laminated configuration corresponding to the texture.

Further, in the texture prediction stage, for example, the texture expressed by the laminated structure is predicted by performing a computer simulation using the absorption coefficient and the scattering coefficient corresponding to each of the materials for coloring a plurality of colors. With this configuration, for example, the texture corresponding to the laminated configuration can be appropriately predicted by computer simulation. In the texture prediction stage, for example, a computer simulation having the same or similar method as a known computer simulation method can be used. For example, in the texture prediction stage, a computer simulation by a light scattering Monte Carlo simulation method (MCML method) or the like can be preferably used.

Further, the absorption coefficient and the scattering coefficient corresponding to each of the materials for coloring a plurality of colors may be obtained by, for example, measurement for a small number of samples. In this case, the region composition prediction method further includes, for example, each color sample preparation step and a coefficient determination step. Each color sample preparation step is, for example, a step of preparing a sample corresponding to a material for coloring each color. In this case, for example, it is conceivable to prepare a sample using a modeling apparatus. The coefficient determination step is, for example, a step of determining the absorption coefficient and the scattering coefficient corresponding to each of the materials for coloring a plurality of colors by measuring the optical characteristics of the sample prepared in each color sample preparation step. .. Further, in this case, in the texture prediction stage, for example, a computer simulation is performed using the absorption coefficient and the scattering coefficient determined in the coefficient determination stage. With this configuration, for example, the absorption coefficient and the scattering coefficient corresponding to each of the materials for coloring a plurality of colors can be appropriately determined. Further, as a result, for example, a computer simulation at the texture prediction stage can be appropriately performed with high accuracy.

Further, in the learning stage, for example, deep learning can be preferably used as the machine learning. In this case, the learning stage can be considered as, for example, a stage in which the learning target neural network, which is the learning target neural network, performs learning. Further, in this case, as the learning target neural network, for example, a neural network or the like in which a parameter indicating a texture is used as an input and an output and a parameter indicating a laminated structure is used as an intermediate output can be preferably used. Further, more specifically, as such a learning target neural network, for example, a neural network having an encoder unit and a decoder unit can be preferably used. In this case, the encoder unit can be considered, for example, a neural network or the like in which a parameter indicating the texture is input and a parameter indicating the laminated structure is output. Further, the decoder unit can be considered as, for example, a neural network or the like in which a parameter indicating a laminated structure is input and a parameter indicating a texture is output. With this configuration, for example, a trained model in which the relationship between the laminated configuration and the texture is trained can be appropriately generated. Further, as each of the encoder unit and the decoder unit, for example, it is preferable to use a multi-layer neural network having three or more layers.

Further, in this case, the learning target neural network can be considered as, for example, a neural network in which an encoder unit and a decoder unit are connected. Further, when such a learning target neural network is used, for example, it is conceivable to have the decoder unit perform learning first, and then make the entire learning target neural network perform learning. More specifically, in this case, the learning stage has, for example, a decoder unit learning stage and an overall learning stage. The decoder unit learning stage can be considered, for example, a stage in which the weight in the decoder unit is determined by having the decoder unit learn the relationship between the laminated structure and the texture. The overall learning stage can be considered, for example, a stage in which learning of the relationship between the laminated structure and the texture is performed on the entire learning target neural network. Further, in this case, in the overall learning stage, for example, the weight in the decoder unit determined in the decoder unit learning stage is fixed, and the learning target neural network is made to perform learning. With this configuration, for example, a trained model can be appropriately generated.

Here, the weight in the neural network is a weight set between the neurons constituting the neural network. Further, when such a learning target neural network is used, in the region configuration prediction stage, for example, a trained model created by having the training target neural network perform training is used to predict a stacked configuration corresponding to the texture. .. With such a configuration, for example, it is possible to appropriately predict the laminated configuration corresponding to a desired texture.

Further, in order to predict the laminated structure corresponding to the desired texture with higher accuracy, it is conceivable to further use the data acquired by a method other than the computer simulation. In this case, for example, it is conceivable to actually prepare a sample corresponding to various laminated configurations and perform measurement on the sample corresponding to the texture of interest. Even in this configuration, the number of required samples can be significantly reduced, for example, by using the data acquired by computer simulation.

More specifically, in this case, in the learning stage, for example, a trained model is generated using simulation acquisition data and measurement acquisition data. The simulation acquisition data is, for example, data showing the relationship between the laminated structure and the texture based on the texture predicted in the texture prediction stage. Further, the measurement acquisition data is data showing the relationship between the laminated structure and the texture acquired by measuring the optical characteristics of the sample prepared by using, for example, a modeling apparatus or the like. With this configuration, for example, a trained model that can predict the relationship between the texture and the laminated configuration with high accuracy can be appropriately generated.

Further, in this case, in the learning stage, for example, it is conceivable to generate a trained model by using the simulation acquisition data and the measurement acquisition data without distinguishing them. In this case, regarding the use of the simulation acquisition data and the measurement acquisition data without distinction, for example, at the time of learning to generate the trained model, each data is either the simulation acquisition data or the measurement acquisition data. It is treated in the same way without changing the way of handling. Further, such an operation can be considered as, for example, an operation of increasing the number of data used for learning by adding measurement acquisition data. With this configuration, for example, simulation acquisition data and measurement acquisition data can be used easily and appropriately.

It is also conceivable to use the simulation acquisition data and the measurement acquisition data in different ways so as to make the best use of their respective characteristics. More specifically, with respect to the simulation acquisition data, for example, it can be considered that a large number of data can be created more easily. Further, regarding the measurement acquisition data, for example, it can be considered that the relationship between the texture and the laminated structure is shown with higher accuracy. Then, in this case, for example, it is conceivable to first perform learning using simulation acquisition data to generate a learning model in which learning is advanced, and to have the learning model perform learning using measurement acquisition data. Be done. Further, such an operation can be considered as, for example, an operation of adjusting the learning model reflecting the simulation acquisition data by using the measurement acquisition data.

Further, in this case, the learning stage has, for example, a first learning stage and a second learning stage. In this case, in the first learning stage, for example, by having the learning model perform learning using the simulation acquisition data without using the measurement acquisition data, an intermediate generation model which is a learning model reflecting the simulation acquisition data is created. Generate. Further, in the second learning stage, for example, a trained model is generated by further learning the intermediate generative model using the measurement acquisition data. With this configuration, for example, the simulation acquisition data and the measurement acquisition data can be appropriately used so as to make the best use of their respective characteristics. Further, as a result, for example, a trained model capable of predicting the relationship between the texture and the laminated structure with higher accuracy can be appropriately generated.

Further, when a learning target neural network having an encoder unit and a decoder unit is used, it is conceivable to use simulation acquisition data and measurement acquisition data according to the configuration of the learning target neural network. More specifically, in this case, for example, in the decoder unit learning stage to be performed first, it is conceivable to have the decoder unit perform learning using the measurement acquisition data. Further, in this case, in the overall learning stage, for example, the learning target neural network is made to perform learning using the simulation acquisition data. With this configuration, for example, the characteristics of the actually prepared sample can be appropriately reflected in the trained model. Further, more specifically, in this case, in the decoder unit learning stage, for example, the decoder unit is made to perform learning using the measurement acquisition data and not using the simulation acquisition data. Further, in the overall learning stage, for example, the learning target neural network is made to perform learning by using the simulation acquisition data and the measurement acquisition data. With this configuration, for example, the characteristics of the actually prepared sample can be appropriately reflected in the trained model.

Further, in the case where a material for modeling a plurality of colors is used as a material for coloring a plurality of colors, after predicting a laminated structure corresponding to a desired texture, for example, a modeled object to be modeled is obtained based on the prediction result. It is conceivable to generate the modeling data shown. In this case, the area configuration prediction method further includes, for example, a modeling data generation stage for generating modeling data. As the modeling data, for example, it is conceivable to generate data in which the modeling material is laminated on the modeling device so that at least a part of the modeled object is formed in the region configuration predicted in the region configuration prediction stage. With this configuration, for example, the modeling apparatus can appropriately perform modeling of a modeled object that can obtain a desired texture.

Further, as the configuration of the present invention, a region configuration prediction method composed of some of the above-mentioned features can be considered. In this case, for example, a region configuration prediction method focusing on the operation of the region configuration prediction stage can be considered. Further, as the configuration of the present invention, for example, it is conceivable to use a region configuration prediction device corresponding to the above region configuration prediction method. In these cases as well, for example, the same effect as described above can be obtained.

According to the present invention, for example, a region configuration for obtaining a desired texture can be appropriately predicted.

It is a figure explaining the modeling system 10 which executes the area composition prediction method which concerns on one Embodiment of this invention. FIG. 1A shows an example of the configuration of the modeling system 10. FIG. 1B shows an example of the configuration of the main part of the modeling apparatus 12. FIG. 1C shows an example of the configuration of the head portion 102 in the modeling apparatus 12. It is a figure explaining the structure of the modeled object 50. FIG. 2A is a cross-sectional view showing an example of the configuration of the modeled object 50 modeled by the modeling apparatus 12. FIG. 2B shows an example of the configuration of the colored region 154 in the modeled object 50. It is a figure which shows an example of the texture of a modeled object expressed by using a colored region 154. FIG. 3A is a cross-sectional view showing the composition of human skin in a simplified manner. FIG. 3B shows an example of the configuration of the colored region 154 used when expressing the texture of the skin. FIG. 3C shows an example in which the number of layers of each region constituting the colored region 154 is different. It is a figure explaining the operation which predicts a region composition using a trained model. FIG. 4A shows an example of the configuration of the control PC 14. FIG. 4B is a flowchart showing an example of an operation of generating modeling data in the control PC 14. It is a figure explaining the operation which generates a trained model. FIG. 5A is a flowchart showing an example of the operation of generating the trained model. FIG. 5B is a flowchart showing the operation of step S102 in more detail. It is a figure which explains the measurement performed with respect to the ink sample in more detail. FIG. 6A is a photograph showing an example of an ink sample. FIG. 6B shows an example of a photograph taken at the time of measurement with respect to the ink sample. FIG. 6C shows a mathematical formula representing the dipole model used in this example. It is a figure explaining the simulation performed in this example in more detail. It is a figure which shows an example of the structure of the learning model used in the machine learning executed in this example. FIG. 8A shows an example of the configuration of a deep neural network (DNN) used as a learning model. FIG. 8B shows examples of inputs, outputs, and intermediate outputs for the training model. It is a figure explaining the machine learning performed in this example. FIG. 9A is a flowchart showing an example of an operation of causing the learning model to perform learning. FIG. 9B shows an example of the learning curve of the decoder unit 604. FIG. 9C shows an example of the learning curve of the encoder unit 602. It is a figure explaining the modification of the method of using the simulation acquisition data and the measurement acquisition data. FIG. 10A is a flowchart showing a modified example of the operation of generating a trained model using the simulation acquisition data and the measurement acquisition data. FIG. 10B shows a further modified example of how to use the simulation acquisition data and the measurement acquisition data.

Hereinafter, embodiments according to the present invention will be described with reference to the drawings. FIG. 1 is a diagram illustrating a modeling system 10 that executes a region configuration prediction method according to an embodiment of the present invention. FIG. 1A shows an example of the configuration of the modeling system 10. Except as described below, the modeling system 10 may have the same or similar characteristics as known modeling systems.

In this example, the modeling system 10 is a system for modeling a three-dimensional modeled object, and includes a modeling device 12 and a control PC 14. The modeling device 12 is a device that executes modeling of a modeled object, and uses ink as a modeling material, and by laminating layers of ink, the modeled object is modeled by a layered manufacturing method. In this case, the additive manufacturing method is, for example, a method of forming a modeled object by stacking a plurality of layers. The modeled object is, for example, a three-dimensional three-dimensional structure. The ink can be considered, for example, a functional liquid or the like. Further, in this example, the ink is an example of a modeling material. The ink can be considered, for example, a liquid discharged from the inkjet head. In this case, the inkjet head is an ejection head that ejects a liquid by an inkjet method.

Further, in this example, the modeling apparatus 12 uses inks of a plurality of colors different from each other to model a colored modeled object. As such a modeling device 12, for example, a known modeling device can be preferably used. More specifically, as the modeling device 12, for example, a modeling device (3D printer) manufactured by Mimaki Engineering Co., Ltd. can be preferably used. Further, in this example, as the ink, an ultraviolet curable ink (UV ink) that is cured from a liquid state by irradiation with ultraviolet rays is used.

The control PC 14 is a computer that operates according to a predetermined program, and controls the operation of the modeling device 12 by supplying modeling data indicating a modeled object to be modeled to the modeling device 12. Further, more specifically, in this example, the control PC 14 sets a desired texture or the like for the three-dimensional data indicating the shape or the like of the modeled object to be modeled, and generates the modeling data. In this case, the control PC 14 generates a computer graphic image (hereinafter referred to as a CG image) showing a modeled object based on the three-dimensional data, and while showing the CG image to a user such as a designer who created the three-dimensional data, the texture is displayed. Accept settings etc. from the user. Then, the modeling data for causing the modeling device 12 to model the modeled object with the set texture is generated. Further, in this example, the control PC 14 is an example of the area configuration prediction device. The operation of generating modeling data in the control PC 14 will be described in more detail later.

Subsequently, a more specific configuration of the modeling device 12, a configuration of a modeled object to be modeled by the modeling device 12, and the like will be described in more detail. FIG. 1B shows an example of the configuration of the main part of the modeling apparatus 12. Except as described below, the modeling apparatus 12 may have the same or similar characteristics as the known modeling apparatus. More specifically, except for the points described below, the modeling apparatus 12 is the same as or similar to a known modeling apparatus that performs modeling by ejecting droplets that are a material of the modeled object 50 using an inkjet head. May have features. In addition to the configurations shown in the figure, the modeling device 12 may further include various configurations necessary for modeling the modeled object 50, for example.

In this example, the modeling device 12 is a modeling device (3D printer) that models a three-dimensional modeled object 50 by a layered manufacturing method, and includes a head unit 102, a modeling table 104, a scanning drive unit 106, and a control unit 110. .. The head portion 102 is a portion for discharging the material of the modeled object 50. Further, as described above, in this example, an ultraviolet curable ink is used as the material of the modeled object 50. In this case, the ultraviolet curable ink is an example of an ink that is cured according to a predetermined condition. More specifically, the head portion 102 ejects ultraviolet curable ink from a plurality of inkjet heads as a material for the modeled object 50. Then, by curing the ink after landing, each layer constituting the modeled object 50 is formed in layers. Further, the head portion 102 further discharges the material of the support layer 52 in addition to the material of the modeled object 50. As a result, the head portion 102 forms a support layer 52 around the modeled object 50 and the like, if necessary. The support layer 52 is, for example, a laminated structure that supports at least a part of the modeled object 50 being modeled. The support layer 52 is formed as needed at the time of modeling the modeled object 50, and is removed after the modeling is completed.

The modeling table 104 is a trapezoidal member that supports the modeling object 50 being modeled, is arranged at a position facing the inkjet head in the head portion 102, and the modeled object 50 and the support layer 52 being modeled are placed on the upper surface. do. Further, in this example, the modeling table 104 has a configuration in which at least the upper surface can be moved in the stacking direction (Z direction in the drawing), and is driven by the scanning drive unit 106 to model the modeled object 50. At least the upper surface is moved as the process progresses. In this case, the stacking direction can be considered, for example, the direction in which the modeling materials are laminated in the additive manufacturing method. Further, in this example, the stacking direction is a direction orthogonal to the main scanning direction (Y direction in the drawing) and the sub scanning direction (X direction in the drawing) set in advance in the modeling apparatus 12.

The scanning drive unit 106 is a drive unit that causes the head unit 102 to perform a scanning operation that moves relative to the modeled object 50 being modeled. In this case, moving relative to the modeled object 50 being modeled means, for example, moving relative to the modeling table 104. Further, having the head portion 102 perform the scanning operation means, for example, causing the inkjet head of the head portion 102 to perform the scanning operation. Further, in this example, the scanning drive unit 106 causes the head unit 102 to perform a main scanning operation (Y scanning), a sub-scanning operation (X scanning), and a stacking direction scanning operation (Z scanning) as scanning operations. The main scanning operation is, for example, an operation of ejecting ink while moving in the main scanning direction relative to the modeled object 50 being modeled. Further, the sub-scanning operation is, for example, an operation of moving relative to the modeled object 50 being modeled in the sub-scanning direction orthogonal to the main scanning direction. The sub-scanning operation can be considered, for example, an operation of moving relative to the modeling table 104 in the sub-scanning direction by a preset feed amount. In this example, the scanning drive unit 106 causes the head unit 102 to perform a sub-scanning operation between the main scanning operations. Further, the stacking direction scanning operation is, for example, an operation of moving the head portion 102 in the stacking direction relative to the modeled object 50 being modeled. For example, the scanning drive unit 106 adjusts the relative position of the inkjet head with respect to the modeled object 50 being modeled in the stacking direction by causing the head unit 102 to perform a stacking direction scanning operation in accordance with the progress of the modeling operation.

The control unit 110 is configured to include, for example, the CPU of the modeling device 12, and controls the modeling operation in the modeling device 12 by controlling each unit of the modeling device 12. More specifically, the control unit 110 controls each part of the modeling device 12 based on, for example, shape information of the modeled object 50 to be modeled, color information, and the like. According to this example, the modeled object 50 can be appropriately modeled.

Further, in this example, the head portion 102 in the modeling apparatus 12 has, for example, the configuration shown in FIG. 1 (c). FIG. 1C shows an example of the configuration of the head portion 102. In this example, the head portion 102 has a plurality of inkjet heads, a plurality of ultraviolet light sources 124, and a flattening roller 126. Further, as a plurality of inkjet heads, as shown in the figure, there are an inkjet head 122s, an inkjet head 122w, an inkjet head 122y, an inkjet head 122m, an inkjet head 122c, an inkjet head 122k, and an inkjet head 122t. These plurality of inkjet heads are arranged side by side in the main scanning direction, for example, by aligning the positions in the sub-scanning direction. Further, each inkjet head has a nozzle row in which a plurality of nozzles are arranged in a predetermined nozzle row direction on a surface facing the modeling table 104. In this example, the nozzle row direction is a direction parallel to the sub-scanning direction.

Further, among these inkjet heads, the inkjet head 122s is an inkjet head that ejects the material of the support layer 52. As the material of the support layer 52, for example, a known material for the support layer can be preferably used. The inkjet head 122w is an inkjet head that ejects white (W color) ink. Further, in this example, the white ink is an example of a light-reflecting ink, and is used, for example, when forming a region having a property of reflecting light (light-reflecting region) in the modeled object 50. This light reflection region reflects light incident from the outside of the modeled object 50, for example, when the surface of the modeled object 50 is colored in full color expression. The full-color expression can be considered, for example, a color expression performed by a possible combination of the subtractive color mixing method using process color ink.

The inkjet head 122y, the inkjet head 122m, the inkjet head 122c, and the inkjet head 122k are coloring inkjet heads used when modeling the colored model 50, and are of a plurality of color inks (coloring inks) used for coloring. Each ink is ejected. More specifically, the inkjet head 122y ejects yellow (Y color) ink. The inkjet head 122m ejects magenta (M color) ink. The inkjet head 122c ejects cyan (C color) ink. Further, the inkjet head 122k ejects black (K color) ink. In this case, each color of YMCK is an example of a process color used for full-color expression. Further, in this example, the ink of each color of YMCK is an example of a material for coloring a plurality of colors and a material for modeling a plurality of colors. Further, the inkjet head 122t is an inkjet head that ejects clear ink. The clear ink is, for example, an ink that is colorless and transparent (T) with respect to visible light.

The plurality of ultraviolet light sources 124 are light sources (UV light sources) for curing the ink, and generate ultraviolet rays for curing the ultraviolet curable ink. Further, in this example, each of the plurality of ultraviolet light sources 124 is arranged on one end side and the other end side of the head portion 102 in the main scanning direction so as to sandwich an array of inkjet heads between them. As the ultraviolet light source 124, for example, a UV LED (ultraviolet LED) or the like can be preferably used. It is also conceivable to use a metal halide lamp, a mercury lamp, or the like as the ultraviolet light source 124. The flattening roller 126 is a flattening means for flattening a layer of ink formed during modeling of the modeled object 50. The flattening roller 126 flattens the ink layer by contacting the surface of the ink layer and removing a part of the ink before curing, for example, during the main scanning operation.

By using the head portion 102 having the above configuration, for example, the ink layer constituting the modeled object 50 can be appropriately formed. Further, by forming a plurality of layers of ink on top of each other, for example, the modeled object 50 can be appropriately modeled. Further, the specific configuration of the head portion 102 is not limited to the configuration described above, and can be variously modified. For example, the head portion 102 may further have an inkjet head for colors other than the above as an inkjet head for coloring. Further, the arrangement of the plurality of inkjet heads in the head portion 102 can be variously modified. For example, some inkjet heads may be displaced from other inkjet heads in the sub-scanning direction.

Subsequently, an example of the configuration of the modeled object 50 to be modeled by the modeling device 12 of this example will be described. FIG. 2 is a diagram illustrating the configuration of the modeled object 50. FIG. 2A is a cross-sectional view showing an example of the configuration of the modeled object 50 modeled by the modeling apparatus 12. FIG. 2B shows an example of the configuration of the colored region 154 in the modeled object 50. In this example, the modeling apparatus 12 models a modeled object 50 whose surface is colored at least partly. In this case, the surface of the modeled object 50 can be considered as, for example, a region where colors can be visually recognized from the outside of the modeled object 50. Further, as described above, in this example, the modeling apparatus 12 models the modeled object 50 by the additive manufacturing method. Then, in this case, the modeling apparatus 12 forms the modeled object 50 by stacking layers of ink having a predetermined thickness in the stacking direction. More specifically, in this case, the modeling apparatus 12 forms the modeled object 50 by, for example, forming the ink layers shown as the modeling unit layer 502 in the drawing in layers.

Further, in this example, as shown in the figure, for example, the modeling device 12 models a modeled object 50 having an internal area 152 and a colored area 154. The internal region 152 is a region constituting the interior of the modeled object 50, and is formed inside the modeled object 50 so as to be surrounded by the colored region 154. Further, in this example, the internal region 152 is formed as a region that also serves as a light-reflecting region by being formed of a light-reflecting ink such as white ink. In the modified example of the configuration of the modeled object 50, a light reflection region may be formed separately from the internal region 152. In this case, the internal region 152 can be formed with, for example, an ink of any color. Further, in this case, the light reflection region is formed between, for example, the internal region 152 and the colored region 154.

The colored region 154 is an region to be colored in the modeled object 50, and the thickness (depth from the outer peripheral surface) in the normal direction on the surface of the modeled object 50 is increased by using inks of a plurality of colors for coloring. By being formed on the surface of the modeled object 50 so as to be constant, it is formed in a state of being colored in various colors. Further, in this example, the modeling apparatus 12 forms a colored region 154 in the modeled object 50 by using, for example, each color of CMYK and clear ink. Further, the modeling apparatus 12 may further use, for example, W color ink as the coloring ink.

Further, in this example, as shown in FIG. 2B, the modeling apparatus 12 forms a colored region 154 having a structure in which a plurality of layered regions 202 overlap each other. Further, in this case, the modeling apparatus 12 is formed by individually coloring each layered region 202 with various colors. Further, the plurality of layered regions 202 are formed so as to overlap each other in a direction parallel to the normal direction on the surface of the modeled object 50. In this case, the normal direction on the surface of the modeled object 50 can be considered as, for example, a direction orthogonal to the surface at each position on the surface of the modeled object 50. With this configuration, for example, various textures can be expressed in the colored region 154. Further, in FIG. 2B, the difference in color of each layered region 202 is shown with different shaded patterns for convenience of illustration.

Here, in this example, the layered region 202 constituting the colored region 154 can be considered as, for example, a region having a substantially constant thickness and extending in a direction orthogonal to the normal direction. Further, as described above, in this example, the modeling apparatus 12 models the modeled object 50 by the additive manufacturing method. In this case, on the upper surface side and the lower surface side in the lamination direction in the additive manufacturing method, the modeling unit layer 502, which is a layer of each ink laminated by the additive manufacturing method, is used as it is as each layered region 202 constituting the colored region 154. It is conceivable to use it. Further, on the side surface of the modeled object 50 intersecting the upper surface and the lower surface, the colored region 154 is formed along the side surface so that the thickness in the normal direction is constant. Further, in this case, the structure of the colored region 154 is such that a plurality of layered regions 202 along the side surface are overlapped. Then, in this case, the layered region 202 can be considered as a layered region or the like different from the modeling unit layer 502. Further, the layered region 202 can be considered as, for example, a layered region for expressing the texture in the colored region 154. Further, it is preferable that each layered region 202 constituting the colored region 154 is formed with a predetermined constant thickness. Therefore, in this example, the amount of ink is appropriately supplemented by using clear ink, which is a colorless and transparent ink, so that the thickness of the layered region 202 does not differ due to the difference in color of the layered region 202. .. In this case, compensation using clear ink can be considered, for example, compensation for a difference in the amount of colored ink used due to a difference in color in each layered region 202.

Further, as described above, according to this example, by forming the colored region 154 having a structure in which a plurality of layered regions 202 are overlapped, various textures can be expressed in the colored region 154. In this case, it is conceivable to express, for example, a feeling of transparency as the texture. Further, by expressing the transparency, for example, it is conceivable to form a colored region 154 that reproduces the texture of human skin with high quality. Further, in this case, for example, it is conceivable to form a colored region 154 having a configuration described below with reference to FIG.

FIG. 3 is a diagram showing an example of the texture of a modeled object expressed by using the colored region 154, and shows an example of the configuration of the colored region 154 when expressing the texture of human skin. FIG. 3A is a cross-sectional view showing the composition of human skin in a simplified manner. As shown in the figure, human skin has a multi-layered structure in which the dermis and epidermis are layered on the subcutaneous tissue. Therefore, in this example, for example, the texture of the skin is reproduced by using the colored region 154 in which the layered regions 202 are overlapped in the region configuration (layer structure) that imitates the multi-layered structure of the skin. In this case, the region configuration can be considered, for example, a configuration corresponding to at least a part of the region of the modeled object.

FIG. 3B shows an example of the configuration of the colored region 154 used when expressing the texture of the skin. In this case, the colored region 154 has a dermis region 302, an epidermis region 304, and a clear region 306, as shown in the figure. The dermis region 302 is a region corresponding to the dermis in human skin. Further, in human skin, the dermis usually has a reddish color due to the influence of blood flowing through blood vessels in the subcutaneous tissue. Therefore, in this example, the dermis region 302 is formed by overlapping the layered regions 202 of the reddish color. The epidermis region 304 is a region corresponding to the epidermis in human skin. In this case, the epidermis can be considered as, for example, a region showing a skin color which is the surface color of human skin. Therefore, the epidermis region 304 is formed by overlapping the layered regions 202 of the skin color system.

In addition, transparency is considered to be important for the texture of human skin. Therefore, in this example, a clear region 306, which is a region formed by clear ink, is formed above (outside) the dermis region 302 and the epidermis region 304. In this case, it is conceivable that the clear region 306 is formed only with clear ink. Further, depending on the required texture of the skin, it is conceivable that the clear region 306 is formed, for example, in a state where a small amount of colored ink is mixed, mainly using clear ink. Further, in this case, various textures can be expressed by changing the number of layers of the layered region 202 constituting each of the dermis region 302, the epidermis region 304, and the clear region 306, and the color of each layered region 202. Can be done.

FIG. 3C shows an example in which the number of layers of each region constituting the colored region 154 is different. In the figure on the left side, the number of layers of the layered region 202 constituting the dermis region 302 is 4, the number of layers of the layered region 202 constituting the epidermis region 304 is 2, and the number of layers of the layered region 202 constituting the clear region 306 is 2. An example is shown in the case where the number of layers of is 3. Further, in the figure on the right side, the number of layers of the layered region 202 constituting the dermis region 302 is set to 3, the number of layers of the layered region 202 constituting the epidermis region 304 is set to 5, and the number of layers of the layered region 202 constituting the clear region 306 is set to 5. An example is shown when the number is 2. According to this example, various textures can be expressed by changing the number of layers and the color of the layered region 202 constituting each region, for example. Further, depending on the required texture, a part of these regions may be omitted.

Here, when the number of layers of the layered region 202 constituting the colored region 154 is too large, it is conceivable that the color of the colored region 154 becomes dark due to the increase in the amount of light absorbed in each layered region 202. .. Therefore, the total of the layered regions 202 in the colored region 154 is preferably about 10 layers or less. Further, in this example, the total of the layered regions 202 constituting the colored region 154 is fixed to 10 layers. Therefore, when the number of layered regions 202 constituting any of the regions increases or decreases, the number of layered regions 202 constituting any of the other regions decreases or increases. With this configuration, for example, the thickness of the colored region 154 can be kept constant even when the texture expressed by the colored region 154 changes.

Further, the color of each layered region 202 constituting the dermis region 302 and the color of each layered region 202 constituting the epidermis region 304 may be selected from a color group prepared in advance. In this case, the color group is, for example, a group including a plurality of colors. Further, the color group used in this example can be considered as, for example, a group including only a part of all the colors that can be expressed by the modeling apparatus 12 (see FIG. 1). In this case, it is conceivable to use color groups containing colors in different combinations as the color groups corresponding to each region. More specifically, for example, as for the color group including the color used in the layered region 202 constituting the dermis region 302, it is conceivable to use the color group including a plurality of reddish colors. As the color group including the color used in the layered region 202 constituting the epidermis region 304, it is conceivable to use a color group including a plurality of skin color systems.

As described above, in this example, various textures can be expressed by forming the colored region 154 having a structure in which the layered regions 202 colored with various colors are overlapped. However, in this case, it is considered that it becomes difficult to predict what kind of texture can be obtained by what kind of region configuration due to the variety of possible configurations for the region configuration in which the plurality of layered regions 202 overlap. Further, as a result, for example, it may be difficult to find a region configuration corresponding to the required texture.

On the other hand, in this example, it has been learned by preliminarily performing machine learning on the relationship between the texture and the laminated structure in which the ink layers are overlapped in the overlapping manner corresponding to the overlapping method of the layered regions 202 in the region configuration. The model is used to predict the laminated configuration corresponding to the desired texture. In addition, based on the prediction result of the laminated configuration, the region configuration corresponding to the laminated configuration is predicted. Further, in this case, as the data for generating the trained model, the data acquired by performing the computer simulation (hereinafter referred to as the simulation acquisition data) is used. Therefore, the operation of predicting the region configuration using the trained model, the operation of generating the trained model using the simulation acquisition data, and the like will be described below.

FIG. 4 is a diagram illustrating an operation of predicting a region configuration using a trained model. In this example, for example, the control PC 14 in the modeling system 10 predicts the area configuration. Then, in this case, for example, it is conceivable to use the control PC 14 having the configuration shown in FIG. 4 (a). FIG. 4A shows an example of the configuration of the control PC 14. In this example, the control PC 14 has a display unit 402, a reception unit 404, a communication I / F unit 406, and a control unit 408. Further, as described above, as the control PC 14, for example, it is conceivable to use a computer that operates according to a predetermined program. In this case, it can be considered that each part of the computer operates as each part of the control PC 14.

The display unit 402 is a display device that displays characters, images, and the like. As the display unit 402, for example, a computer monitor or the like can be preferably used. Further, in this example, the display unit 402 displays, for example, a CG image or the like showing a modeled object to be modeled by the modeling device 12 (see FIG. 1). The reception unit 404 is an input device that receives a user's instruction. As the reception unit 404, for example, a computer input device (for example, a mouse, a keyboard, etc.) can be preferably used. Further, in this example, the reception unit 404 receives an instruction from the user to set the texture of the CG image displayed on the display unit 402, for example. Further, in this case, in this example, as the texture, the setting of the texture corresponding to the transparency is accepted.

The communication I / F unit 406 is an interface unit for communicating with an external device of the control PC 14. As the communication I / F unit 406, for example, a communication interface unit in a computer or the like can be preferably used. Further, in this example, the control PC 14 communicates with the modeling device 12 and other computers via the communication I / F unit 406. In this case, in communication with the modeling device 12, for example, it is conceivable to supply modeling data. Further, as the communication with other computers, for example, communication with a computer storing the trained model can be considered. In this case, the control PC 14 communicates with another computer via the communication I / F unit 406 to obtain a prediction result obtained by predicting, for example, the texture set in the CG image using the trained model. Get from another computer. With this configuration, the control PC 14 can appropriately perform a prediction operation using, for example, a trained model. Further, in this case, the prediction operation using the trained model can be considered as, for example, the prediction operation using the result of machine learning. Further, the control PC 14 may perform prediction using the trained model by using a trained model stored in a storage unit or the like of the control PC 14 (not shown) without communicating with another computer, for example. ..

The control unit 408 is configured to control the operation of each unit in the control PC 14. As the control unit 408, for example, a computer CPU or the like can be preferably used. Further, in this example, the control PC 14 generates modeling data based on the texture set for the CG image by the user. In this case, the control unit 408 predicts the region configuration of the modeled object 50 corresponding to the texture (desired texture) specified by the user, for example, based on the result of the prediction performed using the trained model. Then, modeling data is generated based on the predicted area configuration. Further, the control unit 408 supplies (outputs) the generated modeling data to the modeling device 12 via the communication I / F unit 406. With this configuration, for example, the modeling device 12 can appropriately perform modeling of a modeled object that reproduces a desired texture.

Further, in the control PC 14, for example, by performing the operation shown in FIG. 4B, the texture setting is received from the user and the modeling data is generated. FIG. 4B is a flowchart showing an example of an operation of generating modeling data in the control PC 14. In the operation of generating modeling data in this example, the control PC 14 first displays a CG image showing a modeled object on the display unit 402, and receives an instruction to specify the texture from the user via the reception unit 404 (S102). ). Further, as a result, the control PC 14 sets the texture of the CG image. Setting the texture on the CG image may mean, for example, setting the texture on at least a part of the objects shown by the CG image. Further, in this example, as the texture to be set for the CG image, as described above, the texture corresponding to the transparency is used. Further, in this case, the control PC 14 creates, for example, a CG image showing the modeled object based on the three-dimensional data indicating the shape of the modeled object, and displays it on the display unit 402. In this case, as the three-dimensional data, for example, data showing the shape of the modeled object to be modeled in a general-purpose format can be preferably used. In a modified example of the operation of the control PC 14, for example, instead of accepting the texture setting from the user in the control PC 14, a CG image in which the texture is set in advance may be received from the outside of the control PC 14.

Further, following the operation of step S102, the control PC 14 calculates a setting parameter which is a parameter corresponding to the texture set in the CG image based on the CG image in which the texture is set (S104). In this case, the operation of step S104 is an example of the operation of the parameter calculation stage. Further, in step S104, the control PC 14 calculates the setting parameters by, for example, performing a simulation (computer simulation) of irradiating the CG image with light. More specifically, in this example, a function indicating how the light spreads is used as a setting parameter corresponding to the texture. Regarding using a predetermined function as a parameter, for example, it can be considered that a value calculated by the predetermined function is used as a parameter. Further, in this example, a line spreading function (LSF) is used as such a function.

Further, in this case, as for the operation of calculating the setting parameter, for example, the texture specified by the user is performed on the CG image, and then the LSF corresponding to the texture is acquired by simulation. Further, more specifically, in this case, for example, by performing a simulation of irradiating a CG image in which a texture is set with light (for example, a laser beam), an LS F corresponding to the texture is acquired. Further, as this simulation, for example, it is conceivable to perform a simulation of irradiating linear light in the same manner as in the case of measuring LSF on a sample (patch) or a modeled object actually created. Further, in practical use, for example, the LSF can be appropriately acquired from the transition of the pixel value by performing a simulation of irradiating the surface of the object shown by the CG image with point-like light with a spotlight. In this case, it is conceivable to calculate the point spread function (PSF) by performing a simulation of irradiating point-shaped light, and to calculate the LSF based on the point spread function.

Further, after calculating the setting parameters, the control PC 14 acquires the laminated configuration (layout) corresponding to the setting parameters using the trained model created in advance (S106). In this case, the laminated structure can be considered, for example, a structure in which a plurality of colored layered regions are overlapped. Further, this laminated structure may be considered as, for example, a structure used when creating a trained model in order to show how a plurality of layered regions 202 (see FIG. 2) constituting the colored region 154 of the modeled object 50 are overlapped. can. Further, regarding this laminated structure, for example, it corresponds to how the modeling unit layers 502 (see FIG. 2) are overlapped when each modeling unit layer 502 laminated at the time of modeling coincides with one layered region 202 in the colored region 154. It can also be thought of as a configuration. Further, more specifically, in this example, the laminated structure can be considered to be a structure in which the ink layers are overlapped in an overlapping manner corresponding to the area configuration of the colored region 154 (see FIG. 2) in the modeled object 50.

Further, the operation of step S106 can be considered as, for example, an operation of acquiring an output indicating the stacked configuration by using the setting parameters calculated in step S104 as inputs to the trained model. In this case, using the setting parameter as an input can be considered, for example, giving a setting parameter expressed in a format that can be input to the trained model. With this configuration, for example, the laminated configuration corresponding to the texture set in the CG image can be appropriately acquired by using the trained model. Further, as a result, for example, it is possible to appropriately predict the laminated structure corresponding to the desired texture.

Further, in this example, the control PC 14 predicts the region configuration in which the plurality of layered regions 202 overlap in the colored region 154 of the modeled object 50 based on the prediction result of the laminated configuration. In this case, predicting the region configuration can be considered, for example, predicting the configuration of the colored region 154 for expressing the texture specified by the user. Further, as described above, in this example, the laminated structure predicted by using the trained model has a structure in which the ink layers are overlapped in an overlapping manner corresponding to the area configuration of the colored region 154 in the modeled object 50. And so on. Then, in this case, as the region configuration, the control PC 14 predicts a configuration in which a plurality of layered regions 202 are overlapped in a stacking manner corresponding to the stacking style of the ink layers in the laminated configuration. With this configuration, for example, the region configuration can be appropriately predicted. Further, in this case, the control PC 14 can be considered to function as, for example, an area configuration prediction device. Further, in this example, the operation of step S106 is an example of the operation of the layered configuration acquisition stage. Further, the series of operations performed in steps S104 to S106 is an example of the operations in the area configuration prediction stage. In this case, the region configuration prediction stage is, for example, a stage of predicting the region configuration corresponding to a desired texture. Further, the region configuration prediction stage may be considered as, for example, a stage of predicting a laminated configuration corresponding to a texture using a trained model and predicting a region configuration corresponding to a desired texture based on the laminated configuration. can. Further, in this example, the series of operations performed in S102 to S106 can be considered as an example of the operations in the region configuration prediction stage.

Further, after predicting the region configuration corresponding to the texture specified by the user, the control PC 14 generates modeling data indicating the modeled object to be modeled based on the prediction result (S108). In this example, the operation of step S108 is an example of the operation of the modeling data generation stage. Further, in step S108, the control PC 14 generates modeling data, for example, data for laminating ink layers on the modeling apparatus 12 so that the colored region 154 is formed in the region configuration acquired in step S106. According to this example, for example, it is possible to appropriately predict the region configuration corresponding to the desired texture. Further, by generating modeling data based on the prediction result of the region configuration, for example, the modeling device 12 can appropriately perform modeling of a modeled object that can obtain a desired texture.

Next, the operation of generating the trained model and the like will be described. FIG. 5 is a diagram illustrating an operation of generating a trained model. FIG. 5A is a flowchart showing an example of the operation of generating the trained model. As explained above, in this example, the simulation acquisition data acquired by performing the simulation (computer simulation) is used as the data for generating the trained model. Further, in this case, in the simulation, a parameter indicating the characteristics of the ink used for modeling in the modeling apparatus 12 (see FIG. 1) is used. Then, in this example, the characteristics of this ink are acquired by creating an ink sample (3D printer ink sample) which is a sample corresponding to the ink of each color and performing a predetermined measurement on the ink sample.

Further, in this case, in this example, first, an ink sample is prepared for ink of at least a part of the colors used in the modeling apparatus 12, and a predetermined measurement is performed on the ink sample to acquire the characteristics of the ink (S202). ). More specifically, in this example, in step S202, the absorption coefficient and the scattering coefficient of the ink of each color are determined as the characteristics of the ink based on the result of this measurement. Further, in this case, the absorption coefficient and the scattering coefficient of the ink are not directly acquired by measurement, but are acquired by measuring a predetermined state reflecting the absorption coefficient and the scattering coefficient and performing a predetermined calculation process. do.

Then, following step S202, a simulation is executed using the characteristics of the ink acquired in step S202 (S204). Further, in this example, in this simulation, the texture expressed by each of the laminated configurations is predicted for a plurality of types of laminated configurations in which a plurality of layered regions colored with different configurations are overlapped. Further, in step S204, the relationship between the laminated structure and the predetermined texture is predicted by simulation.

Further, in this example, the operation of step S204 is an example of the operation of the texture prediction stage. In step S204, for example, it is conceivable to use a texture corresponding to the transparency as the texture. In this case, the transparency can be considered, for example, the transparency expressed by overlapping a plurality of layered regions. Further, in this example, a function indicating how the light spreads is used as a parameter indicating the texture expressed by the laminated structure. More specifically, as such a function, a line spread function (LSF) is used. Further, in this example, the data showing the relationship between the laminated configuration and the texture based on the texture predicted by the simulation performed in step S204 can be considered as the simulation acquisition data. Further, in this example, regarding the simulation, for example, it can be considered that the prediction of the state corresponding to the specified input is performed by the calculation by the computer. Further, in this case, the calculation on the computer can be considered as, for example, an operation in which the output corresponding to the input is specified in advance and the calculation is performed according to the rules. Further, this rule can be considered as, for example, a rule described by a mathematical formula.

Further, after the simulation is performed in step S204, a trained model is generated by causing a computer to perform machine learning using the result (S206). More specifically, in this example, in step S206, a trained model is generated by machine learning based on the texture predicted in step S204 for a plurality of types of laminated configurations. In this case, the trained model can be considered as, for example, a learning model in which the relationship between the laminated structure and the texture is trained. Further, in this example, the operation of step S206 is an example of the operation of the learning stage. In step S206, for example, deep learning can be preferably used as the machine learning. In this case, step S206 can be considered, for example, a stage in which the learning target neural network, which is the learning target neural network, is made to perform learning. According to this example, for example, a trained model used for predicting the region configuration can be appropriately created.

Subsequently, the operation of each step in the flowchart of FIG. 5A will be described in more detail. FIG. 5B is a flowchart showing the operation of step S102 in more detail. As described above, when measuring the characteristics of the ink, an ink sample is prepared for the ink of at least a part of the colors used in the modeling apparatus 12 (S212). Further, in this example, ink samples are prepared for the inks of each color of YMCK and the clear inks. In this case, the ink sample for each color can be considered as an example of the sample to be measured, for example. Further, the sample corresponding to the ink of each color of YMCK can be considered as an example of the sample corresponding to the material for coloring, for example. Further, in this example, the operation of step S212 is an example of the operation of each color sample preparation stage. Each color sample preparation stage can be considered, for example, a stage of preparing a sample corresponding to a material for coloring each color.

Further, after the ink sample is prepared, the characteristics of the ink of each color are acquired by performing a predetermined measurement (S214), and the absorption coefficient and the scattering coefficient of the ink of each color are determined based on the result (S216). The measurement performed on the ink sample and the method of determining the absorption coefficient and the scattering coefficient will be described in more detail later. Further, in this example, the operations performed in steps S214 and S216 are examples of operations in the coefficient determination stage. The coefficient determination stage is considered to be, for example, a stage of determining the absorption coefficient and the scattering coefficient corresponding to each of the materials for coloring of a plurality of colors by measuring the optical characteristics of the sample prepared in each color sample preparation stage. be able to. According to this example, for example, by performing measurement on a small number of ink samples, the absorption coefficient and the scattering coefficient corresponding to the inks of each color can be appropriately determined.

FIG. 6 is a diagram for explaining in more detail the measurement and the like performed on the ink sample. FIG. 6A is a photograph showing an example of an ink sample, and shows an example of an ink sample actually created by the inventor of the present application. FIG. 6B shows an example of a photograph taken at the time of measurement with respect to the ink sample. FIG. 6C shows a mathematical formula representing the dipole model used in this example.

As described above, in this example, ink samples are prepared for each color of YMCK ink and clear ink. In this case, in the modeling apparatus 12 (see FIG. 1), an ink sample of that color is created by forming a plurality of layers of ink of each color in layers. The ink sample of each color can be considered as, for example, a sample in which layers of ink of one color are stacked. Further, as the ink sample, a sample for a color other than the above may be further prepared. In this case, for example, it is conceivable to further prepare an ink sample for white.

Further, in this example, as the measurement for the ink sample, the current state of light scattering is measured by irradiating the upper surface of each ink sample with light by a point light source and taking a picture of the ink sample. .. More specifically, in this case, as the point light source, laser light of red color (R color), green color (G color), and blue color (B color) is used. Then, for example, as shown in FIG. 6B, the exposure times are 1/1000 second, 1/250 second, 1/60 second, 1/15 second, 1/4 second, 1 second, and 4 seconds, respectively. Set to to take a picture of the ink sample. As the light source of the laser light, a known smart beam laser projector or the like can be preferably used. Further, it is conceivable that the photographing is performed by a known digital camera in a dark room. Further, in this case, it is preferable to mount a polarizing plate in front of the lens of the camera to reduce the influence of the surface reflection component. Further, when irradiating the laser beam, it is conceivable to reproduce the point light source in a pseudo manner by projecting an image having a high pixel value only in the center of the image with a projector and, for example, narrowing the projected light with a slit. In this case, the resolution of the projected image may be adjusted to the resolution of the projector.

Further, in this example, the operation of taking a picture of the ink sample in this way can be considered as an operation of measuring the ink sample. Then, after measuring the ink sample in this way, the absorption coefficient and the scattering coefficient of the ink of each color are determined based on the photograph taken. Further, in this example, the absorption coefficient and the scattering coefficient are determined by applying the data shown by the photograph taken to the dipole model. More specifically, for example, in the dipole model represented by the mathematical formula shown in FIG. 6 (c), the scattering term can be considered as a term having an absorption coefficient and a scattering coefficient as parameters. Then, in this case, the absorption coefficient and the scattering coefficient can be estimated by approximating the result of measuring the scattering phenomenon with a dipole model. In this case, the operation of determining the absorption coefficient and the scattering coefficient can be considered as, for example, an operation of estimating the absorption coefficient and the scattering coefficient using a dipole model. Further, in this case, for example, it is conceivable to determine the absorption coefficient and the scattering coefficient by fixing the values such as the anisotropic parameter and the relative refractive index to predetermined values and performing fitting to the dipole model. ..

Further, at the time of fitting to the dipole model, correction may be appropriately performed as necessary. As such a correction, for example, it is conceivable to perform a correction in consideration of the reflectance of each ink sample. Further, in this case, it is conceivable to separately measure and obtain the reflectance of the ink sample. Further, in this example, the dipole model is an example of a model showing the characteristics of the ink sample using the absorption coefficient and the scattering coefficient. A model other than the dipole model may be used in the modified example of the method of determining the absorption coefficient and the scattering coefficient. Further, in this case, it is conceivable to perform the measurement for the ink sample according to the model to be used. According to this example, for example, the absorption coefficient and the scattering coefficient corresponding to the inks of each color used in the modeling apparatus 12 can be appropriately determined. In addition, this makes it possible to appropriately perform a simulation that reflects the characteristics of the ink to be used, for example, in a subsequent simulation.

FIG. 7 is a diagram for explaining the simulation performed in this example in more detail. As described above, in this example, the texture is predicted by executing a simulation using the characteristics of the ink acquired in advance. Further, in this simulation, the texture expressed by each laminated configuration is predicted for a plurality of types of laminated configurations having different configurations. Further, in this case, for example, as shown in the figure, a laminated structure in which ink layers of various colors are laminated is considered, and the texture obtained by each laminated structure (texture expressed by the laminated structure) is of each color. Calculated by computer simulation using the ink absorption coefficient and scattering coefficient.

Further, in this example, as the laminated structure, a structure in which a white layer as a base is used as the lowermost layer and a predetermined number of layers are superposed on the white layer is considered. Further, in this case, as the color of each layer other than the bottom layer, various colors that can be expressed by the combination of the plurality of colors of ink used in the modeling apparatus 12 are used. In this case, the number of layers in the laminated structure may be the same as the number of layered regions 202 (see FIG. 2) constituting the colored region 154 (see FIG. 2) of the modeled object 50. Further, it is conceivable to select the color of each layer in the laminated structure from a color group prepared in advance. As such a color group, it is conceivable to use a color group including a plurality of types of colors used as the color of the layered region 202. More specifically, as described above with reference to FIG. 3 and the like, it is conceivable to use a region in which the layered regions 202 are overlapped with a region configuration imitating the multi-layered structure of human skin as the colored region 154. .. In this case, as the laminated structure shown in FIG. 7, a structure in which layers corresponding to the layered regions 202 of the dermis region 302, the epidermis region 304, and the clear region 306 (see FIG. 3) are overlapped is used. Further, it is conceivable to select the color of the layer corresponding to the layered region 202 of the dermis region 302 and the epidermis region 304 from the colors included in the color group for the dermis region 302 and the color group for the epidermis region 304.

Further, in the operation of actually forming the colored region 154 at the time of modeling the modeled object 50, by variously changing the ratio of the ejection positions for ejecting the ink of each color, the color of each layered region 202 is area-wise. Make an expression. However, in the simulation performed in this example, if the color is to be expressed in terms of area in this way, the necessary processing is considered to be complicated. Therefore, the colors of each layer in the simulation performed in this example are expressed in volume by the ratio of the inks of each color contained in the unit volume.

Further, in the simulation performed in this example, for a laminated structure in which the colors of each layer are set, the absorption coefficient and the scattering coefficient of the inks of each color are used, and the light scattering Monte Carlo simulation method (MCML method) is used to obtain the laminated structure. Predict the texture to be produced. The light scattering Monte Carlo simulation method can be considered, for example, the Monte Carlo method for obtaining a solution of the transport equation when the target substance is a multilayer turbid medium. Further, in the light scattering Monte Carlo simulation method, for example, as shown in the figure, the scattering and absorption generated for the incident light in each layer in the laminated structure are stochastically calculated to predict the reflected light. .. The application of the light scattering Monte Carlo simulation method to the laminated structure can be performed in the same manner as or in the same manner as the known usage of the light scattering Monte Carlo simulation method.

Further, as described above, in this example, a function indicating how the light spreads is used as a parameter indicating the texture predicted by this simulation. Further, as such a function, a line spread function (LSF) is used. According to this example, for example, it is possible to appropriately predict the corresponding texture for various laminated configurations without preparing a sample corresponding to the laminated configuration or measuring the sample. Further, as a result, for example, for a large number of laminated configurations, data showing the relationship between the laminated configurations and the texture can be easily and appropriately acquired.

Subsequently, in this example, the operation of generating the trained model, the operation of using the trained model, and the like will be described in more detail. FIG. 8 shows an example of the configuration of the learning model used in the machine learning executed in this example. FIG. 8A shows an example of the configuration of a deep neural network (DNN) used as a learning model. FIG. 8B shows examples of inputs, outputs, and intermediate outputs for the training model. The learning model can be considered, for example, a model having parameters that are changed by performing machine learning. Further, the trained model can be considered as, for example, a learning model after adjusting the parameters by performing predetermined learning. Further, as described above, in this example, deep learning is used as machine learning. Then, DNN is used as the learning model. In this case, the learning model for learning can be considered as an example of the training target neural network.

Further, in this example, as a learning model, as shown in FIG. 8A, a neural network having an encoder unit 602 and a decoder unit 604 is used. In this case, the encoder unit 602 can be considered as, for example, a neural network or the like that takes a predetermined parameter as an input and outputs a predetermined parameter different from the input. Further, the decoder unit 604 can be considered as, for example, a neural network or the like in which the parameter output by the encoder unit 602 is used as an input and the same parameter as the input of the encoder unit 602 is output as an output. Further, in this case, the learning model used in this example can be considered as, for example, a neural network in which the encoder unit 602 and the decoder unit 604 are connected. As each of the encoder unit 602 and the decoder unit 604, it is conceivable to use a multi-layer neural network having three or more layers.

Further, in this example, the input to the encoder unit 602 can be considered as the input to the learning model (the entire learning model). The output of the encoder unit 602 can be considered as an intermediate output in the learning model. Further, the output of the decoder unit 604 can be considered as the output of the learning model. Further, in this example, as shown in FIG. 8B, a line spreading function (LSF) is used as an input and an output of the learning model. In this case, as shown in the figure, it is conceivable to divide the value of the line spreading function into an R component, a G component, and a B component and use them as inputs and outputs. More specifically, in this case, for example, it is conceivable that each component of RGB is represented by a vector having about 100 elements and used as an input and an output. Further, in this case, the LSF used as the input and output of the learning model can be considered as an example of the parameters indicating the texture.

Further, in this example, as the output of the encoder unit 602 which is the intermediate output of the learning model, a parameter indicating the laminated configuration (layout) is used. In this case, the laminated configuration is the laminated configuration used to use the texture prediction in the simulation described with reference to FIG. 7. Further, in this case, as a parameter indicating the laminated structure, a parameter indicating the color of each layer stacked in the laminated structure is used. More specifically, in this example, the number of layers to be laminated in the laminated structure is fixed, and the laminated structure is shown by using a vector composed of numbers (color numbers) indicating the colors of the respective layers. In this case, regarding the color number, for example, it is conceivable to prepare a color group including a plurality of colors and associate the number with each color included in the color group. Further, in FIG. 8B, for convenience of illustration, the difference in color of each layer is illustrated with different shaded patterns.

Further, the number of layers in the laminated structure may be, for example, about 10 layers. Further, in this example, as the color number, an integer value associated with each color is used. Therefore, in the encoder unit 602, a soft quantization layer is provided immediately before the output so that the output value becomes a value close to an integer value. The soft quantization layer can be considered, for example, a configuration in which the output value is brought close to an integer value by a differentiable function. As the soft quantization layer, a known function used for such a purpose can be preferably used. For example, as a soft quantization layer, it is conceivable to use a substantially stepped function that changes smoothly so as to be differentiable.

Further, in this example, as described above, such a learning model is subjected to machine learning using the simulation acquisition data acquired by the simulation described with reference to FIG. 7 and the like. .. FIG. 9 is a diagram illustrating machine learning performed in this example. FIG. 9A is a flowchart showing an example of an operation of causing the learning model to perform learning. In this example, when the learning model is trained, the decoder unit 604 is trained first, and then the entire learning model is trained.

More specifically, in this case, first, the weight in the decoder unit 604 is determined by having the decoder unit 604 learn the relationship between the laminated configuration and the texture using the simulation acquisition data (S302). In this case, regarding having the decoder unit 604 learn the relationship between the laminated configuration and the texture, for example, the correspondence between the laminated configuration and the texture is determined based on the texture calculated by simulation for various laminated configurations. It can be considered that the weight of the decoder unit 604 is adjusted so as to reproduce it. Further, the weight in the neural network such as the encoder unit 602 and the decoder unit 604 is a weight set between the neurons constituting the neural network. Regarding the operation of causing the decoder unit 604 to perform learning, for example, the learning model is trained so that the relationship between the intermediate output and the output in the entire learning model shown in FIG. 8A reflects the simulation acquisition data. It can also be thought of as an action to make it. Further, in this example, the operation of step S302 is an example of the decoder unit learning stage.

Further, after the decoder unit 604 is trained, the encoder unit 602 and the decoder unit 604 are combined (S304), and the learning of the relationship between the laminated structure and the texture is performed on the entire learning model (S304). S306). More specifically, in this example, in step S306, the weight in the decoder unit 604 determined in step S302 is fixed, and the entire learning model is trained. Further, in this case, the weight in the encoder unit 602 is adjusted so that the input to the encoder unit 602 and the output from the decoder unit 604 are the same.

With this configuration, for example, the encoder unit 602 will reflect the relationship between the laminated configuration and the texture indicated by the simulation acquisition data while maintaining the relationship between the laminated configuration and the texture learned by the decoder unit 604. The weight can be adjusted. Further, as a result, for example, the entire learning model can be appropriately trained based on the simulation acquisition data. Further, in this example, the operation of step S306 is an example of the operation of the whole learning stage. Further, in this case, the learning model trained in this way can be considered as a trained model used when predicting the layered configuration.

Here, as described above, in this example, the parameter indicating the laminated structure is used as an intermediate output in the learning model. Therefore, when predicting the laminated structure, it is conceivable to use this intermediate output as a value indicating the laminated structure. With such a configuration, for example, it is possible to appropriately predict the laminated configuration corresponding to a desired texture. Further, this intermediate output is the output of the encoder unit 602 in the learning model. Therefore, the trained encoder unit 602 may be used as the trained model when predicting the laminated configuration. Also in this case, when creating the trained model, it is preferable to use the neural network having the encoder unit 602 and the decoder unit 604 as the training target neural network. Even in such a configuration, for example, it is possible to appropriately predict the laminated configuration corresponding to the desired texture.

When the trained encoder unit 602 is used as the trained model, it seems that it is sufficient to train only the encoder unit 602 based on the simulation acquisition data without using the decoder unit 604. However, when learning based on the simulation acquisition data is directly performed by the encoder unit 602 without using the decoder unit 604, it is conceivable that it takes more time to complete the learning. Further, for example, it is conceivable that the number of required simulation acquisition data will increase significantly. Further, in this case, even if the number of simulation acquisition data is significantly increased, the learning may not be completed properly. On the other hand, according to this example, for example, by setting the weight of the decoder unit 604 first and then adjusting the weight of the encoder unit 602, the encoder unit 602 can be learned efficiently and appropriately. be able to.

Further, the inventor of the present application actually prepares 1413 sets of data as simulation acquisition data regarding the relationship between the laminated configuration (layout) and the texture (LSF), and applies the decoder unit 604 and the entire learning model to the whole. I was made to learn. In this case, 80% of the data was used as training data, and 20% of the data was used as test data. FIG. 9B shows an example of the learning curve of the decoder unit 604. FIG. 9C shows an example of the learning curve of the encoder unit 602. In FIGS. 9B and 9C, the horizontal axis is the number of epochs. The vertical axis shows the output result of the loss function. Further, in this case, the learning curve of the encoder unit 602 can be considered as, for example, the learning curve of the entire learning model in which the encoder unit 602 and the decoder unit 604 are combined.

As can be understood from the illustrated learning curve and the like, in the decoder unit 604, it can be seen that the learning has converged at an early stage. Further, it can be seen that the encoder unit 602 also converges when the number of epochs approaches 20. Further, it can be seen that both the encoder unit 602 and the decoder unit 604 can estimate the validation data with sufficiently high accuracy. Further, the loss value for the test data divided earlier was 0.1460, and it was possible to estimate the unknown data with the same accuracy as the training data. Further, it was confirmed that the loss value at the time when the learning converged on the encoder unit 602 was sufficiently small, and that the stacking configuration could be predicted appropriately.

As described above, according to this example, for example, a trained model in which the relationship between the laminated structure and the texture is appropriately trained can be appropriately generated. Further, by using such a trained model, for example, with respect to a texture such as transparency, it is possible to appropriately predict a laminated structure corresponding to a desired texture. Further, as described above, in this example, the trained model is generated by using the simulation acquisition data acquired by the simulation. Then, in this case, for example, a large number of data necessary for learning can be appropriately prepared without actually preparing a large number of samples or performing measurement on a large number of samples. Further, as a result, for example, the time and effort required to generate the trained model can be appropriately reduced.

Further, in the above, an example of an operation of generating a trained model using only simulation acquisition data as training data has been mainly described. However, in the modified example of the operation of generating the trained model, the trained model may be generated by further using the data other than the simulation acquisition data. For example, in the case of predicting a laminated structure corresponding to a desired texture with higher accuracy, data acquired by actually creating a sample (patch) and performing measurement (hereinafter referred to as measurement acquisition data). ) Can be further used. In this case, for example, measurement acquisition data is acquired by actually creating a sample corresponding to various laminated configurations and performing measurement on the sample corresponding to a texture such as transparency (texture of interest). Then, the trained model is generated by training the learning model using the simulation acquisition data and the measurement acquisition data. Even in this configuration, by using the simulation acquisition data, the number of required samples can be significantly reduced as compared with the case where a trained model is generated using only the measurement acquisition data, for example.

Further, the measurement acquisition data can be considered as, for example, data showing the relationship between the laminated structure and the texture acquired by measuring the optical characteristics of the sample prepared by using a modeling apparatus or the like. .. Further, when creating a learning model, for example, it is conceivable to generate a trained model by using the simulation acquisition data and the measurement acquisition data without distinguishing them. In this case, regarding the use of the simulation acquisition data and the measurement acquisition data without distinction, for example, at the time of learning to generate the trained model, each data is either the simulation acquisition data or the measurement acquisition data. It is treated in the same way without changing the way of handling. Further, such an operation can be considered as, for example, an operation of increasing the number of data used for learning by adding measurement acquisition data. With this configuration, for example, simulation acquisition data and measurement acquisition data can be used easily and appropriately.

When deep learning is used as machine learning, if there is a large difference in data tendency between simulation acquisition data and measurement acquisition data, if the two are used without distinction, the data tendency in the entire data will be diminished. As a result, the accuracy of learning may decrease. Therefore, when the difference in data tendency between the simulation acquisition data and the measurement acquisition data is large, it is preferable to generate the trained model using only the simulation acquisition data without using the measurement acquisition data. On the other hand, when the difference in data tendency between the simulation acquisition data and the measurement acquisition data is small, by using the simulation acquisition data and the measurement acquisition data as described above, for example, learning with high accuracy is appropriate. Can be done. Further, as a result, for example, a trained model capable of predicting the relationship between the texture and the laminated structure with high accuracy can be appropriately generated.

When the simulation acquisition data and the measurement acquisition data are used, it is conceivable to use the simulation acquisition data and the measurement acquisition data in different ways so as to make the best use of their respective characteristics. In this case, the simulation acquisition data can be considered as, for example, data that can more easily create a large number of data. Further, the measurement acquisition data can be considered as, for example, data showing the relationship between the texture and the laminated structure with higher accuracy. Then, in this case, for example, as shown in FIG. 10, it is conceivable to use the simulation acquisition data and the measurement acquisition data. FIG. 10 is a diagram illustrating a modified example of how to use the simulation acquisition data and the measurement acquisition data. FIG. 10A is a flowchart showing a modified example of the operation of generating a trained model using the simulation acquisition data and the measurement acquisition data.

In this modified example, first, learning using simulation acquisition data (initial learning) is performed to generate a learning model in which learning is advanced (S402), and learning using measurement acquisition data is performed on the learning model (S402). Additional learning) is performed (S404). Such an operation can be considered as, for example, an operation of adjusting the learning model reflecting the simulation acquisition data by using the measurement acquisition data. Further, in this case, the operation of step S402 is an example of the operation of the first learning stage. The first learning stage can be considered, for example, a stage in which learning using simulation acquisition data is performed on the learning model without using measurement acquisition data. Further, in the present modification, in step S402, an intermediate generation model, which is a learning model reflecting the simulation acquisition data, is generated by learning using the measurement acquisition data. Further, the operation of step S404 is an example of the operation of the second learning stage. The second learning stage can be considered, for example, a stage in which learning using the measurement acquisition data is performed on the learning model after the first learning stage. Further, in the present modification, in step S404, the trained model is generated by further learning the intermediate generation model generated in step S402 using the measurement acquisition data.

According to this modification, for example, the simulation acquisition data and the measurement acquisition data can be appropriately used so as to make the best use of their respective characteristics. Further, as a result, for example, a trained model that reflects the characteristics of the actually prepared sample can be efficiently created. Further, for example, a trained model capable of predicting the relationship between the texture and the laminated structure with higher accuracy can be appropriately generated. Further, the operation of generating the trained model in this modified example can be considered as, for example, the operation of performing transfer learning. In this case, the intermediate generative model generated in step S402 can be considered as, for example, the first trained model generated by learning using the simulation acquisition data. Further, the trained model generated in step S402 can be considered as a second trained model generated by transfer learning using the measurement acquisition data and the first trained model. Further, in this case, by performing transfer learning by such a method, a trained model that reflects the measured and acquired data can be efficiently and appropriately created as compared with the case of performing learning using only the measured and acquired data, for example. can do.

Further, for example, when a neural network having an encoder unit 602 and a decoder unit 604 (see FIG. 8) as shown in FIG. 8A is used as a learning model, simulation acquisition data and measurement acquisition data are used according to the configuration of the learning model. It is also possible to use it. In this case, in step S402, it is conceivable to have the decoder unit 604 perform learning using the measurement acquisition data. Further, in step S404, it is conceivable that the learning using the simulation acquisition data is performed on the entire learning model. Further, in this case, it can be considered that the operation of step S402 is also an example of the decoder unit learning stage. It can be considered that the operation of step S404 is also an example of the overall learning stage. With this configuration, for example, the simulation acquisition data and the measurement acquisition data can be appropriately used according to the configuration of the learning model. Further, as a result, for example, a trained model that reflects the characteristics of the actually prepared sample can be efficiently created.

Further, as described above, when a neural network having an encoder unit 602 and a decoder unit 604 is used, it is considered that the learning of the decoder unit 604 converges relatively early. Therefore, for example, even when only the measurement acquisition data, which is smaller in number than the simulation acquisition data, is used, it is possible to appropriately learn the decoder unit 604. Therefore, in step S404, for example, it is conceivable to have the decoder unit 604 perform learning using the measurement acquisition data and without using the simulation acquisition data. With this configuration, for example, the decoder unit 604, which determines the weight first, can be trained to more appropriately reflect the measurement acquisition data. Further, in this case, in the learning in step S404 performed after that, as described above, the weight of the decoder unit 604 is fixed and the learning is performed. Therefore, with this configuration, the learning that reflects the measurement acquisition data can be appropriately performed even in the learning performed in step S404. Further, in the learning performed on the entire learning model in step S404, it is considered that a large number of training data is preferable. Therefore, in step S404, it is conceivable to have the learning model perform learning using the simulation acquisition data and the measurement acquisition data. Further, depending on the purpose of learning and the like, in step S404, the learning model may be trained using the simulation acquisition data and without using the measurement acquisition data.

Further, the simulation acquisition data and the measurement acquisition data are not used as learning data of the same learning model, but may be used as learning data for learning models different from each other, for example, as shown in FIG. 10B. FIG. 10B shows a further modified example of how to use the simulation acquisition data and the measurement acquisition data.

In this modified example, as the trained model, a prediction model used for predicting the laminated configuration (layout) from the texture (LSF, etc.) and a confirmation model used for confirming the texture corresponding to the laminated configuration are used. Use. In this case, as the prediction model, for example, the trained model for prediction of the laminated structure described above can be preferably used. In this modified example, when the prediction model is created, the learning model is trained using the simulation acquisition data and without using the measurement acquisition data. Further, more specifically, as the prediction model, for example, a trained model created by training the learning model described with reference to FIG. 8 or the like using simulation acquisition data is preferably used. Can be done. Further, in this modification, the confirmation model is a trained model used to confirm the prediction result in the prediction model. As the confirmation model, for example, a neural network or the like corresponding to the decoder unit 604 in the learning model described with reference to FIG. 8 or the like can be preferably used. Further, in this modified example, when the confirmation model is created, the learning model is trained using the measurement acquisition data and without using the simulation acquisition data.

Further, in this modification, for example, at the time of predicting the laminated configuration, it is conceivable to predict the laminated configuration using the prediction model and then confirm the prediction result using the confirmation model. .. More specifically, in this case, for example, it is conceivable to further use the laminated configuration output by the prediction model as an input of the confirmation model. Then, it is conceivable to compare the output of the confirmation model with the desired texture (LSF or the like) used as the input of the prediction model. With this configuration, for example, it is possible to more appropriately confirm whether or not the result of the prediction of the laminated structure performed using the prediction model is the result of obtaining a desired texture. Further, in this case, if necessary, for example, it is conceivable to adjust the result of prediction of the laminated structure so as to obtain a desired texture. With this configuration, for example, a laminated configuration corresponding to a desired texture can be obtained more appropriately.

Further, in a further modification of the usage of the simulation acquisition data and the measurement acquisition data, for example, after learning to be performed using the simulation acquisition data, the learning result is used to create a sample to be measured. It is also conceivable to do. More specifically, in this case, for example, it is conceivable to confirm the amount of learning data required for learning by performing learning using simulation acquisition data. Further, in this case, it is conceivable to prepare a sample used for acquiring the measurement acquisition data according to the required amount of the confirmed learning data. With this configuration, for example, it is possible to appropriately acquire the amount of measurement acquisition data required for learning without preparing an unnecessarily large number of samples. Further, in this case, it is conceivable to create a trained model for predicting the layer configuration corresponding to the desired texture by using only the measurement acquisition data without using the simulation acquisition data. Even in this configuration, the simulation acquisition data and the measurement acquisition data can be used to appropriately predict the layer configuration corresponding to the desired texture.

Subsequently, supplementary explanations and the like regarding each configuration described above will be given. Further, in the following, for convenience of explanation, the present example will be referred to including the modified example described above. As described above, in this example, the control PC 14 (see FIG. 1) is an example of the area configuration prediction device. Further, for example, as shown in FIG. 4A, in this example, the control PC 14 includes a display unit 402, a reception unit 404, a communication I / F unit 406, and a control unit 408. In this case, the control unit 408 operates as each unit of the area configuration prediction device by operating according to a predetermined program, for example. More specifically, the control unit 408 operates as, for example, a texture prediction unit, a trained model creation unit, a region configuration prediction unit, a modeling data generation unit, and the like by operating according to a predetermined program. In this case, the texture prediction unit can be considered, for example, a configuration for executing an operation in the texture prediction stage. The trained model creation unit can be considered, for example, a configuration for executing an operation in the learning stage. The area configuration prediction unit can be considered, for example, a configuration for executing the operation of the area configuration prediction stage. Further, the modeling data generation unit can be considered, for example, a configuration for executing an operation in the modeling data generation stage. Further, the control unit 408 may operate as each configuration of the region configuration prediction device for other configurations in the control PC 14.

Further, in the above description, regarding the operation of predicting the laminated structure, mainly the operation of predicting the laminated structure corresponding to a partial area configuration of the modeled object to be modeled by the modeling apparatus 12 (see FIG. 1) has been described. .. However, in the modified example of the operation of predicting the laminated structure, the laminated structure corresponding to at least a part of the region structure of the object other than the modeled object may be predicted. For example, in a printing device (for example, an inkjet printer) that prints a two-dimensional image, it is possible to express various textures by stacking a plurality of layers of ink of various colors on a medium to be printed. Conceivable. Then, in this case, for example, the overlapping method of the ink layers formed on the medium may be considered as the region configuration, and the stacking configuration corresponding to such the region configuration may be predicted. Further, for example, when a printing device (so-called 2.5D printer) that forms a three-dimensional shape by stacking layers of ink on a medium is used, the composition of the surface of the three-dimensional shape formed on the medium is defined as an area. Considering the configuration, the laminated configuration corresponding to such a region configuration may be predicted. In these cases as well, by using the trained model created in the same manner as described above, it is possible to appropriately predict the laminated configuration and the like.

The present invention can be suitably used, for example, in a method for predicting region composition.

10 ... modeling system, 12 ... modeling device, 14 ... control PC, 50 ... modeled object, 52 ... support layer, 102 ... head part, 104 ... modeling table, 106 ... Scanning drive unit, 110 ... Control unit, 122 ... Inkjet head, 124 ... Ultraviolet light source, 126 ... Flattening roller, 152 ... Internal area, 154 ... Colored area, 202: Layered area, 302: Dermis area, 304: Epidermis area, 306: Clear area, 402: Display unit, 404: Reception unit, 406: Communication I / F Unit, 408 ... Control unit, 502 ... Modeling unit layer, 602 ... Encoder unit, 604 ... Decoder unit

Claims (19)

It is a region configuration prediction method for predicting a region configuration, which is a region configuration formed by using a material for coloring a plurality of colors.
It is a stage of predicting the relationship between a predetermined texture and a laminated structure in which a plurality of colored layered regions are overlapped by computer simulation. The texture prediction stage for predicting the texture expressed by the laminated structure of
It is a stage of generating a trained model which is a learning model in which the relationship between the laminated structure and the texture is learned by machine learning, and the texture predicted in the texture prediction stage for the plurality of types of laminated configurations is obtained. The learning stage to generate the trained model based on
It is a stage of predicting the region configuration corresponding to the desired texture, predicting the laminated configuration corresponding to the texture using the learned model generated in the learning stage, and based on the laminated configuration, the desired A region configuration prediction method comprising a region configuration prediction stage for predicting the region configuration corresponding to the texture of the above. The material for coloring the plurality of colors is a material for modeling the plurality of colors.
Claim 1 is characterized in that the region configuration corresponds to at least a part of a region of the modeled object to be modeled in a modeling apparatus that models a modeled object using the plurality of colors of modeling materials. The area composition prediction method described in 1. The region configuration prediction method according to claim 2, wherein the region configuration is a configuration in which a plurality of layered regions, each of which is colored, overlap in a direction parallel to the normal direction on the surface of the modeled object. .. The area composition prediction stage is
Based on the computer graphics image in which the desired texture is set, the setting parameter calculation step of calculating the setting parameter which is the parameter corresponding to the texture set in the computer graphics image, and the setting parameter calculation step.
Any of claims 1 to 3, wherein the set parameter calculated in the setting parameter calculation step is used as an input to the trained model, and the stack configuration acquisition step of acquiring an output indicating the stack configuration is provided. Area composition prediction method described in. The texture is a transparency expressed by overlapping a plurality of the layered regions.
The region configuration prediction method according to any one of claims 1 to 4, wherein a function indicating how light spreads is used as a parameter indicating the texture in the texture prediction stage. The region configuration prediction method according to claim 5, wherein a line spread function (LSF) is used as a parameter indicating the texture. In the texture prediction stage, the texture represented by the laminated structure is predicted by performing a computer simulation using the absorption coefficient and the scattering coefficient corresponding to each of the materials for coloring the plurality of colors. The area composition prediction method according to any one of claims 1 to 6. Each color sample preparation stage for preparing a sample corresponding to the coloring material of each color, and
Further provided with a coefficient determination step of determining the absorption coefficient and the scattering coefficient corresponding to each of the materials for coloring the plurality of colors by measuring the optical characteristics of the sample prepared in each color sample preparation step.
The region configuration prediction method according to claim 7, wherein in the texture prediction step, a computer simulation is performed using the absorption coefficient and the scattering coefficient determined in the coefficient determination step. The learning stage is a stage in which a learning target neural network, which is a learning target neural network, is made to perform learning.
The learning target neural network is a neural network in which the parameter indicating the texture is used as an input and an output, and the parameter indicating the laminated structure is used as an intermediate output.
An encoder unit that is a neural network that inputs the parameter indicating the texture and outputs the parameter indicating the laminated structure, and
The region configuration prediction method according to any one of claims 1 to 8, further comprising a decoder unit that is a neural network that receives a parameter indicating the laminated structure as an input and outputs a parameter indicating the texture as an output. The learning target neural network is a neural network connecting the encoder unit and the decoder unit.
The learning stage is
A decoder unit learning stage in which the weight in the decoder unit is determined by having the decoder unit learn the relationship between the laminated structure and the texture.
The area configuration prediction method according to claim 9, further comprising an overall learning stage in which the weight in the decoder unit determined in the decoder unit learning stage is fixed and the learning target neural network performs learning. In the learning stage
Simulation acquisition data, which is data showing the relationship between the laminated structure and the texture based on the texture predicted in the texture prediction stage, and the acquisition by measuring the optical characteristics of the prepared sample. The region configuration prediction method according to any one of claims 1 to 10, wherein the trained model is generated by using the measurement acquisition data which is the data showing the relationship between the laminated configuration and the texture. The area configuration prediction method according to claim 11, wherein in the learning stage, the learned model is generated by using the simulation acquisition data and the measurement acquisition data without distinguishing them. The learning stage is
The first learning step of generating an intermediate generation model which is a learning model reflecting the simulation acquisition data by causing the learning model to perform learning using the simulation acquisition data without using the measurement acquisition data.
The region configuration prediction method according to claim 11, further comprising learning the intermediate generative model using the measurement acquisition data to generate the trained model. .. The learning stage is a stage in which a learning target neural network, which is a learning target neural network, is made to perform learning.
The learning target neural network is
An encoder unit that is a neural network that inputs the parameter indicating the texture and outputs the parameter indicating the laminated structure, and
It is a neural network connected to a decoder unit which is a neural network in which a parameter indicating the texture is input and a parameter indicating the texture is output, and the parameter indicating the texture is input and output to show the layered configuration. The parameter is the intermediate output
The learning stage is
A decoder unit learning stage, which is a stage in which the weight in the decoder unit is determined by having the decoder unit learn the relationship between the laminated structure and the texture.
It has an overall learning stage in which the weight in the decoder unit determined in the decoder unit learning stage is fixed and the learning target neural network is made to perform learning.
In the decoder unit learning stage, the decoder unit is made to perform learning using the measurement acquisition data.
The region configuration prediction method according to claim 11, wherein in the overall learning stage, the learning target neural network is made to perform learning using the simulation acquisition data. In the decoder unit learning stage, the decoder unit is made to perform learning using the measurement acquisition data and without using the simulation acquisition data.
The region configuration prediction method according to claim 14, wherein in the overall learning stage, the learning target neural network is trained using the simulation acquisition data and the measurement acquisition data. It is further equipped with a modeling data generation stage that generates modeling data indicating the modeled object to be modeled in the modeling device.
The modeling data according to any one of claims 1 to 15, wherein the modeling data is data in which modeling materials are laminated on the modeling apparatus by a stacking method based on the region configuration predicted in the region configuration prediction stage. Area composition prediction method. It is a region configuration prediction method for predicting a region configuration, which is a region configuration formed by using a material for coloring a plurality of colors.
It is a stage of predicting the region configuration corresponding to a desired texture, and by learning the result of predicting the relationship between the predetermined texture and the laminated configuration in which a plurality of colored layered regions are overlapped by computer simulation. It is characterized by including a region configuration prediction stage in which the stacked configuration corresponding to the texture is predicted using the generated trained model, and the region configuration corresponding to the desired texture is predicted based on the stacked configuration. Area composition prediction method to be performed. It is a region configuration prediction device that predicts the region configuration, which is the configuration of the region formed by using materials for coloring of a plurality of colors.
A texture prediction unit that predicts the relationship between a predetermined texture and a laminated configuration in which a plurality of colored layered regions are overlapped by computer simulation, and a texture prediction unit.
A trained model creation unit that generates a trained model, which is a learning model trained by learning the relationship between the laminated structure and the texture, by machine learning.
A region configuration prediction unit that predicts the region configuration corresponding to the desired texture is provided.
The texture prediction unit predicts the texture expressed by each of the laminated configurations of a plurality of types in which the layered regions overlap with each other.
The trained model creation unit generates the trained model based on the texture predicted by the texture prediction unit for the plurality of types of laminated configurations.
The region configuration prediction unit predicts the laminated configuration corresponding to the texture using the trained model generated by the trained model creation unit, and based on the laminated configuration, the region corresponding to the desired texture. A region configuration predictor characterized by predicting a configuration. It is a region configuration prediction device that predicts the region configuration, which is the configuration of the region formed by using materials for coloring of a plurality of colors.
A region configuration prediction unit that predicts the region configuration corresponding to a desired texture is provided.
The region configuration prediction unit uses a trained model generated by learning the result of predicting the relationship between the predetermined texture and the laminated configuration in which a plurality of colored layered regions are overlapped by computer simulation. A region configuration predicting apparatus characterized in that the laminated configuration corresponding to the texture is predicted, and the region configuration corresponding to the desired texture is predicted based on the laminated configuration.

Images