JP7481173B2 - Information processing device, information processing method, and program - Google Patents

Description

The present invention relates to an information processing device, an information processing method, and a program.

In recent years, image recognition using artificial intelligence (AI) has become common. For example, image recognition can be used to determine what animal is in a photograph, but in order to make this determination, it is necessary to have an information processing device such as a computer learn in advance using feature quantities that represent the characteristics of the object to be recognized. Furthermore, the amount of training data (labeled learning data) used for this learning has a significant impact on the accuracy of image recognition by AI.

However, it can be difficult to prepare enough data for sufficient learning, and performing image recognition under insufficient learning conditions can result in reduced recognition accuracy and inappropriate results. In addition, for unlearned or non-recognition targets, the system will select the closest result from the learned results, making it extremely difficult to correctly identify unlearned or non-recognition targets.

Patent Document 1 discloses a technology for creating training data for face discrimination. It describes the use of a discriminator to detect faces facing in various directions in a target image, and performs left-right flipping and 90-degree rotation processing on partial images cut out sequentially from the detection target image to generate inverted/rotated images of the partial images, and uses a specified type of discriminator on these images to determine whether or not the image is of a face facing a specified facial orientation and top-to-bottom direction.

JP 2006-350704 A

However, in the technology described in Patent Document 1, images that serve as training data are generated by inverting or rotating partial images, and since there is a limit to the images that can be obtained by inverting or rotating, a sufficient amount of training data cannot always be obtained. The present invention has been made in consideration of these circumstances, and aims to create more training data and improve the recognition accuracy of image recognition.

The information processing device of the present invention is an information processing device that generates a trained model used in type determination processing by image recognition, and is characterized in that it has an image acquisition means for acquiring a plurality of first images including a target subject, a label assignment means for assigning labels for a plurality of elements related to the subject included in the first image to each of the acquired first images, a model generation means for generating the trained model for each of the elements based on the first image to which the labels have been assigned, and a determination means for determining each element of a second image including a subject for which the type determination processing is performed based on the generated trained model for each of the elements, and determining the type corresponding to the combination of the determination results of each element using determination information indicating the correspondence between the combination of the determination results of each element and the type .

The present invention makes it possible to create more training data and improve the recognition accuracy of image recognition.

FIG. 2 is a diagram illustrating an example of a hardware configuration of an information processing device according to the present embodiment. FIG. 2 is a diagram illustrating an example of a functional configuration of an information processing device according to the present embodiment. FIG. 13 is a diagram illustrating an example of a processing target. FIG. 4 is a diagram showing an example of determination information in the present embodiment. FIG. 2 is a diagram illustrating the operation of the information processing device according to the present embodiment. 4 is a flowchart showing an example of processing by the information processing device in the present embodiment. FIG. 4 is a diagram for explaining packaging type determination in the present embodiment. 4 is a flowchart showing an example of processing by the information processing device in the present embodiment.

The following describes an embodiment of the present invention with reference to the drawings.

FIG. 1 is a diagram showing an example of the hardware configuration of an information processing device 100 in one embodiment of the present invention. The information processing device 100 is, for example, an information processing device such as a personal computer (PC), a server device, or a tablet terminal. The information processing device 100 has a CPU 101, a main memory device 102, an auxiliary memory device 103, an input I/F 104, an output I/F 105, and a network I/F 106. The CPU 101, the main memory device 102, the auxiliary memory device 103, the input I/F 104, the output I/F 105, and the network I/F 106 are connected to each other via a system bus 107 so as to be able to communicate with each other.

The CPU (Central Processing Unit) 101 is a central processing unit that controls the information processing device 100. The main memory 102 is a memory device that functions as a work area for the CPU 101 and a temporary storage location for data. The main memory 102 is implemented using, for example, a RAM (Random Access Memory). The auxiliary memory 103 is a memory device that stores programs, image data, information related to various processes, etc. The auxiliary memory 103 is implemented using, for example, a ROM (Read Only Memory), a hard disk drive (HDD), a solid state drive (SSD), etc.

The input I/F 104 is an interface used to connect to input devices such as a mouse, a keyboard, a touch panel operation unit, an imaging device (camera), etc. The CPU 101 accepts input from the input device connected to the input I/F 104 via the input I/F 104. The output I/F 105 is an interface used to connect to output devices such as a monitor, a speaker, a touch panel display unit, etc. The network I/F 106 is an interface used to communicate with external servers, devices, etc. via a network such as the Internet or a LAN (Local Area Network).

The CPU 101 executes processing based on a program stored in the auxiliary storage device 103, thereby realizing each function of the information processing device 100 shown in FIG. 2 and each process shown in the flowchart described below.

FIG. 2 is a diagram showing an example of the functional configuration of the information processing device 100 in this embodiment. The information processing device 100 has an image acquisition unit 201, a label assignment unit 202, a model generation unit 203, a determination unit 204, and a storage unit 205.

The image acquisition unit 201 acquires images including a target subject captured by an imaging device or the like from an external imaging device, storage device, or the like. The images acquired by the image acquisition unit 201 include images used for learning and images to be subjected to judgment processing.

The label assignment unit 202 assigns labels relating to each element contained in the image to the image used for learning acquired by the image acquisition unit 201. The label assignment unit 202 accepts input of information on labels to be assigned to the images used for learning, and assigns the input labels to the images. The label assignment unit 202 assigns label information consisting of multiple elements, i.e. multiple items, to each image used for learning. Hereinafter, with regard to the images used for learning, images to which no labels have been assigned will also be referred to as "learning data", and images to which labels have been assigned by the label assignment unit 202 will also be referred to as "teaching data", depending on whether or not the image has been assigned a label.

The model generation unit 203 performs learning using training data, which are images used for learning and to which labels have been assigned by the label assignment unit 202, and generates a trained model. The model generation unit 203 extracts features from the images that are training data, and learns based on the extracted features to generate a trained model. The model generation unit 203 generates a trained model for each element that has been assigned a label.

The determination unit 204 performs a determination process for each element of the image to be determined, acquired by the image acquisition unit 201, using the trained model generated by the model generation unit 203. The determination unit 204 also determines the type of subject included in the image to be determined, based on a combination of the determination results of each element obtained by the determination process.

The storage unit 205 stores images acquired by the image acquisition unit 201 (images as learning data, images to be subjected to judgment processing, etc.) and images (teacher data) to which labels have been assigned by the label assignment unit 202. The storage unit 205 also stores a trained model generated by the model generation unit 203 and judgment information used to judge the type of subject by the judgment unit 204.

2 shows an example of an information processing device 100 having an image acquisition unit 201, a label assignment unit 202, a model generation unit 203, a judgment unit 204, and a storage unit 205, but is not limited to this. For example, the functional units related to learning, i.e., the image acquisition unit 201, the label assignment unit 202, and the model generation unit 203 that acquire images (learning data) used in learning, and the functional unit related to judgment, i.e., the judgment unit 204, may be configured to be included in different information processing devices. Also, the storage unit 205 may be an external storage device connected to the information processing device.

The operation of the information processing device 100 in this embodiment will be described using as an example the determination of the packaging type of a "packaged coil" in which a "coil" as shown in FIG. 3 is packed in a packing material. FIG. 3(A) shows an example of a coil. As shown in FIG. 3(A), the coil 301 is made by winding up a metal material such as a steel plate. Note that the coil is not limited to a metal material, and may be made by winding up a plate-shaped resin sheet or a rubber sheet. FIG. 3(B) shows an example of a packed coil, and FIG. 3(B) shows an example in which a coil as shown in FIG. 3(A) is covered with packing paper and is provided with a ring member and a cable tie. In FIG. 3(B), 311 is an inner ring member (inner peripheral packing member) in the packed coil, 312 is an outer ring member (outer peripheral packing member) in the packed coil, and 313 is a cable tie (hoop) in the packed coil. Note that the packed coil shown in FIG. 3(B) is an example, and is not limited to such packaging. For example, the coil may be packed using at least one of packing paper, ring members, and cable ties. In the following, the packaging type will be determined based on the number of bundles, inner ring members, and outer ring members in the packaged coil, but this is just one example, and the elements for determining the packaging type can be set as appropriate.

When the information processing device 100 of this embodiment is applied to determining the packaging type, each functional unit shown in FIG. 2 operates as follows. In the following description, "coil image" refers to an image that includes a packed coil as a subject. The image acquisition unit 201 acquires coil images as learning data and coil images to be determined (as objects of determination). The label assignment unit 202 assigns label information regarding the number of bundles, inner ring member, and outer ring member to each of the images acquired as learning data by the image acquisition unit 201. For example, the label assignment unit 202 assigns label information such as "number of bundles: 3, inner ring member: A, outer ring member: α" to an image of the learning data.

The number of ties may be the number of ties in both the vertical and horizontal directions in the packed coil, or the number of ties in one of the vertical or horizontal directions. The number of ties in the vertical and horizontal directions may be distinguished and assigned as labels. For the inner ring members and outer ring members, information indicating the color of the member or information indicating the presence or absence of the member is assigned as a label. In this embodiment, the label assignment unit 202 assigns label information regarding the number of ties, inner ring members, and outer ring members to each image as learning data, but is not limited to this and may include label information regarding other elements.

The model generation unit 203 performs learning using a learning machine based on the teacher data labeled by the labeling unit 202, and generates a learned model for each of the number of ties, the inner ring member, and the outer ring member. The model generation unit 203 extracts features from the coil image serving as the teacher data, learns based on the extracted features, and generates a learned model for each element of the number of ties, the inner ring member, and the outer ring member. The determination unit 204 performs a determination process for each element (number of ties, inner ring member, and outer ring member) in the image of the determination target acquired by the image acquisition unit 201 using the generated learned models of the number of ties, the inner ring member, and the outer ring member. The determination unit 204 also determines the packaging type of the coil to be determined based on the determination information (determination table) as shown in FIG. 4.

Figure 4 is a diagram showing an example of judgment information (judgment table) for determining the packaging type of a coil. The judgment information (judgment table) shown as an example in Figure 4 is stored in, for example, the memory unit 205, and the judgment unit 204 performs processing by referring to the stored judgment information (judgment table). Figure 4 shows judgment information (judgment table) for a case in which the number of bundles has a trained model capable of determining 3 or 4, the inner ring member has a trained model capable of determining A, B, or C, and the outer ring member has a trained model capable of determining α or β.

For example, when the determination result shows that the number of ties is 3, the inner ring member is A, and the outer ring member is α, the determination unit 204 determines that the packaging type of the packed coil is type 1. Also, when the determination result shows that the number of ties is 4, the inner ring member is A, and the outer ring member is α, the determination unit 204 determines that the packaging type of the packed coil is unknown. Also, when the determination result shows that the number of ties is 4 and the inner ring member is B, the determination unit 204 determines that the packaging type of the packed coil is unknown regardless of the determination result of the outer ring member. Note that in FIG. 4, an example is shown in which there are two types of number of ties and two types of outer ring members, and three types of inner ring members, but this is just an example, and it is sufficient to prepare determination information (determination table) according to the number of types that the learned model for each element can determine.

The operation of determining the packaging type of a packed coil in the information processing device 100 in this embodiment will be described with reference to FIG. 5. In the learning phase for generating a trained model, an image including the packed coil as a subject is captured by the imaging device 501, and the captured coil image is acquired by the image acquisition unit 201 as learning data 502. The image acquisition unit 201 acquires a plurality of coil images as the learning data 502. Note that the coil images as the learning data 502 are not limited to those acquired directly from the imaging device 501, and may be coil images that have been captured in advance and stored in a storage device or the like.

Then, the labeling unit 202 assigns labels for each element (bundle number, inner ring member, and outer ring member) to each of the coil images acquired as the training data 502, generating teacher data 503, which is a labeled image. In this manner, in this embodiment, by assigning three labels, the bundle number, the inner ring member, and the outer ring member, to an image as one training data, three teacher data can be created from one training data. For example, by assigning M types of labels to one training data, M pieces of teacher data can be generated from one training data, making it possible to obtain more teacher data.

Next, the model generation unit 203 uses the created teacher data 503 to train the learning machine 504, and generates a trained model 505 for each element included in the coil image. This generated trained model 505 is used for the judgment process in the judgment unit 204 in the judgment phase in which the judgment process is performed.

In the determination phase for determining the packaging type from the coil image of the determination target, an image including the packed coil as a subject is captured by the imaging device 511, and the captured coil image is acquired by the image acquisition unit 201 as image data 512 of the determination target. Note that the image of the determination target is not limited to one image that captures the entire packed coil, but may be images captured by multiple imaging devices 511 so that parts suitable for the determination process are captured. Next, the determination unit 204 performs a determination process for each element of the acquired image data 512 of the determination target using the determiner 506 based on the trained model, and outputs the determination result.

Next, referring to FIG. 6, an example of processing by the information processing device 100 for determining the packaging type of a packed coil will be described. First, in the learning phase for generating a trained model, in step S601, the image acquisition unit 201 acquires a plurality of coil images 611 as training data. The coil images 611 as training data acquired in step S601 are assigned labels for each element by the label assignment unit 202, and are stored as teacher data, for example, in the storage unit 205.

Next, in step S602, the model generation unit 203 uses the training data generated in step S601 to perform deep learning related to the classification of the cohesion number, and generates a trained model 612 to be used for determining the cohesion number. In step S603, the model generation unit 203 uses the training data generated in step S601 to perform deep learning related to the classification of the inner ring members, and generates a trained model 613 to be used for determining the inner ring members.

In step S604, the model generation unit 203 performs deep learning for the classification of the outer ring members using the teacher data generated in step S601, generates a trained model 614 used to determine the outer ring members, and ends the learning phase processing. The trained models 612 to 614 generated in steps S602 to S604 are stored in, for example, the storage unit 205. Note that the order of executing the processing of steps S602, S603, and S604 in the learning phase is not limited to the example shown in FIG. 6, and is arbitrary. In this embodiment, a trained model is generated by performing deep learning on each element, but it is possible to generate a trained model using other supervised machine learning methods other than deep learning. Examples of supervised machine learning for generating a trained model include neural networks, K-nearest neighbor methods, and linear regression.

In the determination phase for determining the packaging type from the coil image of the determination target, in step S651, the image acquisition unit 201 acquires the coil image 661 of the determination target. Next, in step S652, the determination unit 204 performs a determination process using the coil image 661 of the determination target and the trained model 612 for determining the number of ties, and determines the number of ties in the packed coil in the coil image 661 of the determination target. In step S653, the determination unit 204 performs a determination process using the coil image 661 of the determination target and the trained model 613 for determining the inner ring member, and determines the inner ring member in the packed coil in the coil image 661 of the determination target.

In step S654, the determination unit 204 performs a determination process using the coil image 661 to be determined and the trained model 614 for determining the outer ring member, and determines the outer ring member of the packed coil in the coil image 661 to be determined. Note that, even in the determination phase, the order of executing the processes of steps S652, S653, and S654 is not limited to the example shown in FIG. 6, and is arbitrary.

Next, in step S655, the determination unit 204 determines the packaging type of the packed coil included in the coil image to be determined based on the determination results obtained by the determination processes in steps S652 to S654 described above. The determination unit 204 compares the combination of the determination results obtained by the determination processes described above with the determination information (determination table) to determine the packaging type of the packed coil in the coil image to be determined.

Next, in step S656, it is determined whether additional learning is necessary regarding the packaging type of the packed coil. If it is determined that additional learning is necessary, in step S657, additional learning preparation processing is performed to add the coil image to be judged as learning data, and the process returns to the learning phase. If it is determined that additional learning is not necessary, the processing of the judgment phase ends.

According to this embodiment, by creating training data by assigning labels for each element contained in each of multiple images used as training data, it is possible to create more training data from the training data, and by generating a trained model using more training data, it is possible to improve the recognition accuracy of image recognition.

In addition, in this embodiment, the elements to be judged are broken down into a learnable level, a trained model is generated for each element, and image recognition is performed using these. In this way, by breaking down the elements to be judged into a learnable level and making judgments based on a combination of the judgment results of each element, it becomes possible to correctly judge images that have not been learned or are not eligible for recognition.

The labeling unit 202 may assign labels to the learning data by accepting input of label information to be assigned to each element for each element, or may hold information such as that shown in FIG. 4 and accept input of the packaging type and assign a label to each element corresponding to the input packaging type. In addition, if a person or the like is captured in the coil image, the coil image may be invalidated and not used for processing. In addition, when determining the packaging type, multiple judgments may be made using multiple images of the same object, and if the packaging types in the judgment results match, or if the degree of match exceeds a predetermined threshold, the judgment result may be determined to be correct.

Here, when the determination process by the determination unit 204 is performed in the order of the number of bundles, the inner ring member, and the outer ring member, the process of determining the packaging type of the packed coil based on the determination information exemplified in FIG. 4 is as shown in FIG. 7. In FIG. 7, the determination results in which the packaging type can be identified are shown by solid lines, and the determination results in which the packaging type cannot be identified are shown by dashed lines.

When the determination process by the determination unit 204 is performed in the order of number of bundles, inner ring member, and outer ring member, as shown in 701 in FIG. 7, if the determination result of the number of bundles is 4 and the determination result of the inner ring member is B, regardless of the determination result of the outer ring member, the packaging type of the packed coil cannot be identified and is considered to be an unknown type. Therefore, as in the example shown in FIG. 7, when the result of the determination process is such that the packaging type of the packed coil cannot be identified, it is possible to reduce the amount of processing by not performing subsequent determination processes.

In this case, the information processing device 100 may execute the process of the judgment phase, for example, as shown in FIG. 8. FIG. 8 is a flowchart showing another example of the process of the information processing device 100 in this embodiment. Note that the process of the learning phase is similar to that in the previously described embodiment, so FIG. 8 shows only the process in the judgment phase.

As shown in FIG. 8, in step S801, the image acquisition unit 201 acquires a coil image to be determined. Next, in step S802, the determination unit 204 performs a determination process using the coil image to be determined and a trained model for determining the bundling number, and determines the bundling number of the packed coil in the coil image to be determined. Next, in step S803, the determination unit 204 determines whether or not there is a candidate type for the bundling number obtained as the determination result. If the determination unit 204 determines that there is a candidate type for the bundling number obtained as the determination result, the process proceeds to step S804, and if it determines that there is no candidate type, the determination unit 204 determines that the type is unknown and proceeds to step S809.

In step S804, the determination unit 204 performs a determination process using the coil image to be determined and a trained model for determining the inner ring member, and determines the inner ring member in the packed coil in the coil image to be determined. Next, in step S805, the determination unit 204 determines whether there is a candidate type for the combination of the number of bundles and the inner ring member obtained as the determination result. If the determination unit 204 determines that there is a candidate type for the combination of the number of bundles and the inner ring member obtained as the determination result, the process proceeds to step S806, and if it determines that there is no candidate type, the determination unit 204 determines that the type is unknown and proceeds to step S809.

In step S806, the determination unit 204 performs a determination process using the coil image to be determined and a trained model for determining the outer ring member, and determines the outer ring member in the packed coil in the coil image to be determined. Next, in step S807, the determination unit 204 determines whether there is a candidate type for the combination of the number of bundles, inner ring member, and outer ring member obtained as the determination result. If the determination unit 204 determines that there is a candidate type for the combination of the number of bundles, inner ring member, and outer ring member obtained as the determination result, the process proceeds to step S808, and if it determines that there is no candidate type, the determination unit 204 determines that the type is unknown and proceeds to step S809.

In step S808, the determination unit 204 determines the packaging type of the packed coil included in the coil image to be determined based on the determination results obtained by the determination processes in steps S802, S804, and S806 described above. The determination unit 204 compares the combination of the determination results obtained by the determination processes described above with the determination information (determination table) to determine the packaging type of the packed coil in the coil image to be determined.

In step S809, it is determined whether or not additional learning is required regarding the packaging type of the packed coil. If it is determined that additional learning is required, processing related to the additional learning is performed in step S810, and if it is determined that additional learning is not required, the processing of the determination phase is terminated. In this way, if there are no candidate types that match the combination of determination results, the subsequent determination processing is not performed, thereby reducing the amount of processing.

Note that, although the judgment process has been described as being performed in the order of number of bundles, inner ring member, and outer ring member, this is merely an example, and the execution order of the judgment process can be set to an appropriate order that reduces the amount of processing, based on the judgment information of the packed coils.

A method of detecting the target packed coil as an object and extracting the detected portion from the image, or a method of detecting and extracting the packaging components as an object, may be applied to the above-described embodiment. By combining these methods, it is possible to improve the recognition accuracy of image recognition.

In addition, when learning images in which the size of the target packed coil does not affect the judgment result, images of various sizes may be trained to recognize that there is no difference due to size and that other devices appearing in the vicinity are noise. In this case, since it is necessary to prepare images of various sizes, it is possible to cut out and learn images so that the size of the target object in the image is always the same using image processing or object detection, and to perform judgment processing on images that have been similarly processed, eliminating the need to prepare images of various sizes to recognize that there is no difference due to size, and thus achieving results with less training data.

In addition, when determining the type by combining the determination result by image recognition in the information processing device 100 and the determination result by the user (person), the information processing device 100 may be configured to notify the user that the determination results are different when the determination result differs from the inputted user determination result.

The above embodiments are merely examples of the implementation of the present invention, and the technical scope of the present invention should not be interpreted in a limiting manner based on these. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features.

100 Information processing device 101 CPU
102 Main memory device 103 Auxiliary memory device 104 Input I/F
105 Output I/F
106 Network I/F
107 System bus 201 Image acquisition unit 202 Labeling unit 203 Model generation unit 204 Determination unit 205 Storage unit

Claims (13)

An information processing device that generates a trained model used in a type determination process by image recognition,
an image capture means for capturing a plurality of first images including a subject of interest;
A labeling means for assigning labels to each of the acquired first images with respect to a plurality of elements related to the subject included in the first images;
A model generation means for generating the trained model for each of the elements based on the first image to which the label is assigned ;
and a determination means for determining each element of a second image including a subject for which the type determination process is performed based on the generated trained model for each of the elements, and determining the type corresponding to the combination of the determination results of each element using determination information indicating a correspondence between the combination of the determination results of each element and the type . The information processing apparatus according to claim 1 , wherein the determining unit performs the determination process a plurality of times using a plurality of the second images including the same subject. An information processing device that generates a trained model used in a type determination process by image recognition, the trained model being used to determine each element in an image to be determined using a trained model for each element, and identifying a type from a combination of the determination results of each element obtained,
an image capture means for capturing a plurality of first images including a subject of interest;
A labeling means for assigning labels to each of the acquired first images with respect to a plurality of elements related to the subject included in the first images;
and a model generation means for generating the trained model for each of the elements based on the first image to which the label has been assigned. 4. The information processing apparatus according to claim 1 , wherein the labeling means receives input of information on a label of each element to be assigned to the first image, and assigns the label. The information processing device according to any one of claims 1 to 3, characterized in that the label assignment means accepts input of information indicating a type of the subject in the first image, and assigns the label corresponding to the input information indicating the type . The information processing device according to any one of claims 1 to 5 , characterized in that the model generation means performs supervised machine learning using the first image to which the label has been assigned to generate the trained model. The information processing device according to any one of claims 1 to 5 , wherein the model generation means performs deep learning using the first image to which the label has been assigned to generate the trained model. 8. The information processing apparatus according to claim 1, wherein the subject is a packed coil, and the type is a packing type of the packed coil. The information processing device according to claim 8 , wherein the plurality of elements include at least one of a number of bundles, an inner ring member, and an outer ring member in the packed coil. An information processing method of an information processing device that generates a trained model used in a type determination process by image recognition,
an image acquisition step of acquiring a plurality of first images including a subject of interest;
a labeling step of assigning labels to each of the acquired first images with respect to a plurality of elements related to the subject included in the first images;
A model generation process of generating the trained model for each of the elements based on the first image to which the label is assigned ;
An information processing method comprising: a determination step of determining each element of a second image including a subject for which a type determination process is performed based on the generated trained model for each of the elements; and determining the type corresponding to the combination of the determination results of each element using determination information indicating a correspondence between a combination of the determination results of each element and the type . An information processing method of an information processing device that generates a trained model used in a type determination process by image recognition, the trained model being used to determine each element in an image to be determined using a trained model for each element, and identifying a type from a combination of the determination results of each element obtained,
an image acquisition step of acquiring a plurality of first images including a subject of interest;
a labeling step of assigning labels to each of the acquired first images with respect to a plurality of elements related to the subject included in the first images;
and a model generation step of generating the trained model for each of the elements based on the first image to which the label has been assigned. A computer of an information processing device that generates a trained model used in a type determination process by image recognition,
an image acquisition step of acquiring a plurality of first images including a subject of interest;
a labeling step of assigning labels to each of the acquired first images with respect to a plurality of elements related to the subject included in the first images;
A model generation step of generating the trained model for each of the elements based on the first image to which the label is assigned ;
A program for executing a determination step of determining each element of a second image including a subject for which the type determination process is performed based on the generated trained model for each of the elements, and determining the type corresponding to the combination of the determination results of each element using determination information indicating the correspondence between the combination of the determination results of each element and the type . A computer of an information processing device generates a trained model used in a type determination process by image recognition, the trained model being used to determine each element in an image to be determined using a trained model for each element, and identifying a type from a combination of the determination results of each element obtained,
an image acquisition step of acquiring a plurality of first images including a subject of interest;
a labeling step of assigning labels to each of the acquired first images with respect to a plurality of elements related to the subject included in the first images;
and a model generation step of generating the trained model for each of the elements based on the first image to which the label has been assigned.

Images