JP7492489B2 - Information processing device, information processing method, and information processing program - Google Patents

Description

The present invention relates to an information processing device, an information processing method, and an information processing program.

Conventionally, various technologies have been provided for classifying information. For example, a technology has been provided for classifying product category expressions from each expression using a classification model trained from product category expressions that represent product categories and ground truth labels (for example, Patent Document 1).

JP 2013-257748 A

However, with the above-mentioned conventional technology, it may be difficult to appropriately determine a category for the object to be classified. For example, if a product category expression is not included in the expression, it is difficult to determine the category of the product to be classified. In addition, with the above-mentioned conventional technology, it is difficult to handle information other than text data. Therefore, it is desirable to appropriately determine a category for the object to be classified.

The present application has been made in consideration of the above, and aims to provide an information processing device, an information processing method, and an information processing program that appropriately determine categories for classification objects.

The information processing device according to the present application is characterized by comprising a classification model including a first partial model that outputs a first classification result indicating a first category corresponding to input information and a second partial model that outputs a second classification result indicating a second category corresponding to the input information, an acquisition unit that acquires classification target information to be classified, and a determination unit that inputs the classification target information as the input information to the classification model, and determines a target category to which the classification target information belongs based on a first candidate category indicated by the first classification result output by the first partial model and a second candidate category indicated by the second classification result output by the second partial model.

According to one aspect of the embodiment, it is possible to appropriately determine a category for the classification target.

FIG. 1 is a diagram illustrating an example of information processing according to an embodiment. FIG. 2 is a diagram illustrating an example of information processing according to the embodiment. FIG. 3 is a diagram illustrating an example of the configuration of an information processing system according to the embodiment. FIG. 4 is a diagram illustrating an example of the configuration of an information processing device according to an embodiment. FIG. 5 is a diagram illustrating an example of a first type information storage unit according to the embodiment. FIG. 6 is a diagram illustrating an example of a second type information storage unit according to the embodiment. FIG. 7 is a diagram illustrating an example of a model information storage unit according to the embodiment. FIG. 8 is a flowchart illustrating an example of information processing according to the embodiment. FIG. 9 is a diagram illustrating an example of an application of an information processing system. FIG. 10 is a diagram illustrating an example of a hardware configuration.

Below, the information processing device, information processing method, and information processing program according to the present application will be described in detail with reference to the drawings. Note that the information processing device, information processing method, and information processing program according to the present application are not limited to these embodiments. In addition, the same parts in the following embodiments will be denoted by the same reference numerals, and duplicated descriptions will be omitted.

(Embodiment)
[1. Information Processing]
An example of information processing according to the embodiment will be described below with reference to Fig. 1 and Fig. 2. Fig. 1 and Fig. 2 are diagrams showing an example of information processing according to the embodiment. Specifically, Fig. 1 shows an example of model learning processing among the information processing according to the embodiment. Also, Fig. 2 shows an example of category determination processing using a model among the information processing according to the embodiment.

In the following, first, the structure of classification model M1, which is a model used to determine a category, and an example of a learning process of classification model M1 will be described with reference to FIG. 1. Also, an example of a category determination process using classification model M1 will be described with reference to FIG. 2. Note that, in the following, an example will be described in which the information for which a category is to be determined (also referred to as "classification target information") is an image, but the classification target information is not limited to images and may be any type of information, such as text information or audio, as long as it is information that is the subject of determining (classifying) the category to which the information belongs.

[1-1. Learning process example]
From here, an example of the structure and learning process of the classification model M1 will be described with reference to Fig. 1. The classification model M1 includes a first partial model PM1, a second partial model PM2, and a third partial model PM3, etc. Below, the classification model M1 will be described starting from the input side.

First, the third partial model PM3 will be described. The third partial model PM3 is a partial model (feature extraction model) that performs object detection and feature extraction for classification target information such as input images of the classification model M1. For example, the third partial model PM3 functions as the backbone of the model, and corresponds to the Backbone in FIG. 9. For example, the third partial model PM3 is a sublayer that learns general features related to images. For example, the third partial model PM3 extracts features of the input information IN1 that has been input and outputs them as feature information. For example, the third partial model PM3 outputs feature information extracted from the input image.

The first partial model PM1 is a partial model that receives the output of the third partial model PM3 and outputs a first classification result OT1. For example, the first partial model PM1 functions as the head of the model and corresponds to the first Head in FIG. 9. For example, the first partial model PM1 is a sublayer in which learning specialized for the task of performing classification (hereinafter also referred to as "first classification" is performed). For example, the first partial model PM1 (head) performs a linear transformation according to the task and converts the information (vector) output by the third partial model PM3 (backbone).

The first partial model PM1 is trained to output a classification result for a category corresponding to a first type of dataset (hereinafter also referred to as the "first category") through a learning process using a first type of dataset described below. That is, the first partial model PM1 is trained to perform a first classification corresponding to the first category.

The first partial model PM1 receives image feature information extracted by the third partial model PM3 and outputs a first classification result OT1 for the image. For example, the first partial model PM1 outputs the score (value) of each category of the first category group corresponding to the first type of dataset as the first classification result OT1. For example, the first partial model PM1 has a softmax layer. In this case, the first partial model PM1 performs softmax processing, converts the score of each category so that the sum of the scores of each category is 1 (100%), and outputs the score of each category.

The second partial model PM2 is a partial model that receives the output of the third partial model PM3 and outputs a second classification result OT2. For example, the second partial model PM2 functions as the head of the model and corresponds to the second Head in FIG. 9. For example, the second partial model PM2 is a sublayer in which learning specialized for the task of performing classification (hereinafter also referred to as "second classification" is performed). For example, the second partial model PM2 (head) performs a linear transformation according to the task and converts the information (vector) output by the third partial model PM3 (backbone).

The second partial model PM2 is trained to output a classification result for a category corresponding to a second type of dataset (hereinafter also referred to as the "second category") through a learning process using a second type of dataset described below. That is, the second partial model PM2 is trained to perform a second classification corresponding to the second category.

The second partial model PM2 receives image feature information extracted by the third partial model PM3 and outputs a second classification result OT2 for the image. For example, the second partial model PM2 outputs the score (value) of each category of the second category group corresponding to the second type of dataset as the second classification result OT2. For example, the second partial model PM2 has a softmax layer. In this case, the second partial model PM2 performs softmax processing, converts the score of each category so that the sum of the scores of each category is 1 (100%), and outputs the score of each category.

Note that the above structure is merely an example, and any structure can be adopted for the classification model M1. For example, the classification model M1 may have any structure as long as the first partial model PM1 outputs information indicating a first category corresponding to the input information, and the second partial model PM2 outputs information indicating a second category corresponding to the input information. The classification model M1 may be configured using a structure such as Faster R-CNN (Region-based Convolutional Neural Network). For example, the classification model M1 may be a function that receives an image as input and outputs the coordinates (x, y, w, h) of an object in the image and an object category label.

Next, the learning of the classification model M1 will be described. In FIG. 1, a case where the information processing device 100 (see FIG. 4) learns the classification model M1 will be described as an example, but the learning of the classification model M1 may be performed by a device other than the information processing device 100.

The information processing device 100 uses multiple types of datasets to train the classification model M1. The information processing device 100 trains the classification model M1 using two types of datasets: a first type of dataset and a second type of dataset.

1 shows a case where public data DS1 is used as the first type of dataset. Details of the public data will be described later, but the public data DS1 is a dataset in which a plurality of images are associated with a first category to which each image belongs. For example, the public data DS1 is a dataset in which each of a plurality of images including objects such as a car and a dog is associated with a category (car type, dog breed, etc.) to which the object included in each image belongs. Also, the example of FIG. 1 shows a case where in-house data DS2 is used as the second type of dataset. Details of the in-house data will be described later, but the in-house data DS2 is a dataset in which a plurality of images are associated with a second category to which each image belongs. Note that FIG. 1 shows an example in which the first type of dataset is public data DS1 and the second type of dataset is in-house data DS2, but any dataset may be used for the first type of dataset and the second type of dataset.

The information processing device 100 trains the classification model M1 using public data DS1, which is a first type of dataset. First, in training using the public data DS1, which is a first type of dataset, the information processing device 100 performs a training process (hereinafter also referred to as the "first training process") on the first partial model PM1 and the third partial model PM3 (step S1).

For example, in the first learning process using the public data DS1, the information processing device 100 learns the classification model M1 by performing backpropagation on the path from the first partial model PM1 to the third partial model PM3. That is, in learning using the public data DS1, the information processing device 100 updates the parameters (weights) corresponding to the first partial model PM1 and the third partial model PM3, excluding the second partial model PM2 from the learning target. As a result, in learning using the first type of dataset, the information processing device 100 performs a learning process such that the third partial model PM3 learns to extract features from the data of the first type of dataset, and the first partial model PM1 outputs information indicating the first category corresponding to the input data.

For example, when the information processing device 100 inputs an image in the public data DS1 into the classification model M1, the learning process is performed so that the first partial model PM1 outputs the first category (category CZ) to which the image (image Z) is associated. When the information processing device 100 inputs image Z into the classification model M1, the learning process is performed so that, among the scores of each category output by the first partial model PM1, the score of the category CZ to which image Z is associated becomes larger. For example, when the information processing device 100 inputs image Z into the classification model M1, the learning process is performed so that the score of the category CZ output by the first partial model PM1 approaches 1.

The information processing device 100 trains the classification model M1 using the in-house developed data DS2, which is a second type of dataset. For example, the in-house developed data DS2 is a dataset in which each of a plurality of images including objects such as cars and dogs is associated with a category to which the object included in each image belongs (the category to which cars belong, the category to which dogs belong, etc.). Here, in training using the in-house developed data DS2, which is a second type of dataset, the information processing device 100 performs a learning process (hereinafter also referred to as the "second learning process") on the second partial model PM2 and the third partial model PM3 (step S2).

For example, in the second learning process using the in-house data DS2, the information processing device 100 learns the classification model M1 by performing backpropagation on the path from the second partial model PM2 to the third partial model PM3. That is, in learning using the in-house data DS2, the information processing device 100 excludes the first partial model PM1 from the learning target and updates the parameters (weights) corresponding to the second partial model PM2 and the third partial model PM3. As a result, in learning using the second type of dataset, the information processing device 100 performs a learning process in which the third partial model PM3 learns to extract features from the data of the second type of dataset, and the second partial model PM2 outputs information indicating the second category corresponding to the input data.

For example, when the information processing device 100 inputs an image in the in-house data DS2 into the classification model M1, the learning process is performed so that the second partial model PM2 outputs the first category (category CY) to which the image (image Y) is associated. When the information processing device 100 inputs image Y into the classification model M1, the learning process is performed so that the score of the category CY to which the image Y is associated becomes larger among the scores of each category output by the second partial model PM2. For example, when the information processing device 100 inputs image Y into the classification model M1, the learning process is performed so that the score of the category CY output by the second partial model PM2 approaches 1. For example, the information processing device 100 switches between the first learning process and the second learning process described above according to the input data. For example, the information processing device 100 may switch between the first learning process and the second learning process based on information indicating which type of data the input data is. For example, the switching between the first learning process and the second learning process may be performed by an if statement or the like in the program.

As described above, the information processing device 100 can generate a model including multiple partial models that perform accurate category classification according to each dataset by switching the target of the learning process depending on the type of dataset used for learning. Furthermore, the information processing device 100 can perform learning that takes into account all datasets for the commonly used third partial model PM3, which enables the third partial model PM3 to extract features with high accuracy regardless of the type of data.

[1-2. Example of decision process]
Next, an example of a category determination process using the classification model M1 will be described with reference to Fig. 2. In Fig. 2, an example will be described in which an image X is classification target information TG1, and the information processing device 100 (see Fig. 4) performs a category determination process for the classification target information TG1 using the classification model M1. For example, the information processing device 100 acquires the classification model M1 from the storage unit 120 (see Fig. 4), and acquires the classification target information TG1 from the terminal device 10 (see Fig. 3).

First, the information processing device 100 inputs the classification target information TG1 to the classification model M1 (step S11). The classification model M1 to which the classification target information TG1 has been input extracts the classification target information TG1 using the third partial model PM3, and inputs the feature information extracted by the classification target information TG1 to the first partial model PM1 and the second partial model PM2.

The first partial model PM1, to which the feature information of the classification target information TG1 output by the third partial model PM3 has been input, outputs the score of each category of the first category group (step S12). The information processing device 100 uses the output of the first partial model PM1 to determine a first category (hereinafter also referred to as the "first candidate category") that is a candidate to which the classification target information TG1 belongs. For example, the information processing device 100 determines the category of the first category group that has the largest score output by the first partial model PM1 as the first candidate category FR1.

The second partial model PM2, to which the feature information of the classification target information TG1 output by the third partial model PM3 has been input, outputs the score of each category of the second category group (step S13). The information processing device 100 uses the output of the second partial model PM2 to determine a second category (hereinafter also referred to as the "second candidate category") that is a candidate to which the classification target information TG1 belongs. For example, the information processing device 100 determines the category of the second category group that has the largest score output by the second partial model PM2 as the second candidate category SR1.

Here, the first category group and the second category group may include categories that are not included in the other. For example, the first category group and the second category group may include overlapping categories, but may not completely match, and the hierarchy of categories (concepts) may be different. For example, the second category group may include a category "dog" and the first category group may include categories for each dog breed. In this case, the first category group and the second category group may be a higher or lower concept than the other. Therefore, the information processing device 100 performs a conversion process so that the first candidate category FR1 and the second candidate category SR1 correspond to each other (step S14). In FIG. 2, the information processing device 100 uses the category map CM1 to convert the first candidate category FR1 into the converted first category TFR1 that corresponds to the second category.

For example, the category map CM1 is list information in which each category (first category) in the first category group is associated with a corresponding second category. For example, in the category map CM1, a category for each dog breed in the first category group is associated with the category "dog" in the second category group. Using the category map CM1, the information processing device 100 maps the first candidate category FR1 to the corresponding second category, thereby generating a converted first category TFR1.

Based on the second candidate category SR1 and the first converted category TFR1, the information processing device 100 determines the category to which image X, which is the classification target information TG1, belongs (hereinafter also referred to as the "target category") (step S15). In FIG. 2, the second candidate category SR1 and the first converted category TFR1 match, so the information processing device 100 determines the second candidate category SR1 as the target category TC1 to which image X belongs.

Details of the case where the second candidate category SR1 and the converted first category TFR1 are different will be described later. Also, when the second candidate category SR1 and the first candidate category FR1 match, the information processing device 100 may determine the second candidate category SR1 as the target category TC1 to which image X belongs without performing the conversion process of step S14.

[1-3. Effects, etc.]
As described above, the information processing device 100 can appropriately determine the category of the classification target by determining the category to which the classification target information belongs by taking into account the two classification results output by the classification model M1. For example, the information processing device 100 can enable the combined use of multiple data sets by including multiple partial models (multiple heads) that output classification results, such as the first partial model PM1 and the second partial model PM2, in the classification model M1. Furthermore, the information processing device 100 can absorb differences between multiple data sets by using a category map, and can determine a category using multiple classification results output by multiple heads.

[1-4. Other (data, etc.)]
In the above example, the in-house data DS2, which is the second type of dataset, is data required for providing e-commerce services such as auction services and shopping. For example, the in-house data DS2 includes images required for providing the e-commerce services and information indicating which category (second category) the image belongs to in the category classification (second classification) for the e-commerce services. For example, the in-house data DS2 is generated by collecting images required for providing the e-commerce services and adding correct answer data for learning such as annotations to the collected images, so that the generation of the in-house data DS2 is costly.

On the other hand, the public data DS1, which is the first type of dataset in the above example, is data (open data) that is permitted to be used for model learning and is widely available on the Internet, etc. For example, the public data DS1 includes an image and information indicating which category (first category) the image belongs to in the category classification (first classification). For example, the public data DS1 is general image data for research purposes, and the Open Images Dataset (https://storage.googleapis.com/openimages/web/index.html) is used. For example, the public data DS1 is collected for research purposes from tags on SNS (Social Networking Service). The public data DS1 can be acquired at a lower cost than the in-house data DS2.

In addition, the public data DS1 and the in-house developed data DS2 differ in the granularity of their category classifications and the range they cover. By using the in-house developed data DS2 and the public data DS1 together, the information processing device 100 can cover categories that have not yet been collected with the in-house developed data DS2.

Note that the public data DS1 and in-house data DS2 shown in the above example are merely examples, and any dataset can be used as the first type dataset and the second type dataset.

Furthermore, the processes and models shown in Figures 1 and 2 are merely examples, and the processes and models are not limited to those described above. For example, the configuration of the classification model M1 is not limited to the configuration shown above. For example, the partial models that perform classification in the classification model M1 are not limited to two, the first partial model PM1 and the second partial model PM2, and there may be three or more.

For example, when there are three types of datasets, the classification model M1 may include a fourth partial model in addition to the first partial model PM1 and the second partial model PM2. In this case, the fourth partial model is trained to output a classification result corresponding to the new third type of dataset. Similar to the first partial model PM1 and the second partial model PM2, the output of the third partial model PM3 is input to the fourth partial model. This allows the information processing device 100 to train the third partial model PM3 using three types of datasets.

In the example of FIG. 2, a case has been described in which the second candidate category SR1 and the first converted category TFR1 match, but when the second candidate category SR1 and the first converted category TFR1 differ, the information processing device 100 determines the target category TC1 to which image X belongs based on a predetermined criterion. An example of this point is described below.

If one of the second candidate category SR1 and the first converted category TFR1 is a category indicating that the category is not determined (specific) (hereinafter also referred to as an "unclassified category"), the information processing device 100 determines the other category as the target category TC1 to which image X belongs. For example, if the second candidate category SR1 is an unclassified category, the information processing device 100 determines the first converted category TFR1 as the target category TC1 to which image X belongs. Also, for example, if the first converted category TFR1 is an unclassified category, the information processing device 100 determines the second candidate category SR1 as the target category TC1 to which image X belongs.

In addition, if neither the second candidate category SR1 nor the first converted category TFR1 is an unclassified category, the information processing device 100 determines the target category TC1 by various methods. In this case, for example, the information processing device 100 may determine the second candidate category SR1 as the target category TC1 to which image X belongs. In addition, for example, the information processing device 100 may determine the first converted category TFR1 as the target category TC1 to which image X belongs.

In addition, when the second candidate category SR1 and the first category TFR1 after conversion are different, the information processing device 100 may determine the target category TC1 to which the image X belongs using the outputs of the first partial model PM1 and the second partial model PM2. For example, the information processing device 100 may determine the target category TC1 to which the image X belongs using the score output by the first partial model PM1 and the score output by the second partial model PM2.

The information processing device 100 may determine the target category TC1 to which the image X belongs by using a score (hereinafter also referred to as the "first score") corresponding to the first candidate category FR1 (first category after conversion TFR1) among the scores output by the first partial model PM1. The information processing device 100 may also determine the target category TC1 to which the image X belongs by using a score (hereinafter also referred to as the "second score") corresponding to the second candidate category SR1 among the scores output by the second partial model PM2. The information processing device 100 may determine the target category TC1 to which the image X belongs based on a comparison between the first score corresponding to the first category after conversion TFR1 and the second score corresponding to the second candidate category SR1.

When the second score corresponding to the second candidate category SR1 is equal to or greater than the first score corresponding to the converted first category TFR1, the information processing device 100 determines the second candidate category SR1 as the target category TC1 to which image X belongs. Alternatively, when the first score corresponding to the converted first category TFR1 is greater than the second score corresponding to the second candidate category SR1, the information processing device 100 determines the converted first category TFR1 as the target category TC1 to which image X belongs. Note that the above is merely an example, and the information processing device 100 may determine the target category TC1 to which image X belongs by appropriately using various information.

2. Configuration of Information Processing System
Next, a configuration example of the information processing system 1 will be described with reference to Fig. 3. Fig. 3 is a diagram showing a configuration example of the information processing system according to the embodiment. As shown in Fig. 3, the information processing system 1 includes a terminal device 10, an information providing device 50, and an information processing device 100. The terminal device 10, the information providing device 50, and the information processing device 100 are connected to each other via a predetermined network N so as to be able to communicate with each other by wire or wirelessly. Note that the information processing system 1 shown in Fig. 3 may include a plurality of terminal devices 10, a plurality of information providing devices 50, and a plurality of information processing devices 100.

The terminal device 10 is an information processing device used by a user. The terminal device 10 is realized, for example, by a smartphone, a tablet terminal, a notebook PC (Personal Computer), a desktop PC, a mobile phone, a PDA (Personal Digital Assistant), etc. In the following, the terminal device 10 may be referred to as a user. In other words, in the following, the user can also be read as the terminal device 10.

The terminal device 10 accepts operations by the user. In addition, the terminal device 10 transmits various information to the information processing device 100 and the information providing device 50 in response to the user's operations. For example, the terminal device 10 transmits information requesting content such as text information to the information processing device 100 and the information providing device 50. In addition, the terminal device 10 receives various information from the information processing device 100 and the information providing device 50. For example, the terminal device 10 receives a classification result from the information processing device 100. For example, the terminal device 10 displays the classification result received from the information processing device 100.

The information providing device 50 is a server device that provides the information processing device 100 with information (data) that can be used for model learning. For example, the information providing device 50 transmits open data (public data), which is data that is permitted to be used for model learning, to the information processing device 100. For example, the information providing device 50 publishes the public data on the Internet, and in response to a request from an external device such as the information processing device 100, transmits the requested public data to the external device that is the request destination. For example, the information providing device 50 receives request information requesting public data from the information processing device 100, and in response to receiving the request information, transmits public data corresponding to the request information to the information processing device 100.

For example, the information providing device 50 transmits information associating an image with a category to which the image belongs to as public data to the information processing device 100. For example, the public data provided by the information providing device 50 may be the above-mentioned Open Images Dataset. Note that the above is merely an example, and the public data provided by the information providing device 50 may be any data that can be used by the information processing device 100 as a first type of dataset and can be used for learning the first partial model PM1.

The information processing device 100 is a computer that determines the category to which the information to be classified belongs. For example, the information processing device 100 is a determination device that, for example, treats an image as a classification target and determines the category to which the image belongs based on the output of a model. As shown in FIG. 2, the information processing device 100 inputs the information to be classified into a model and determines the category to which the information to be classified belongs based on the classification result output by the model.

For example, the information processing device 100 is a learning device that performs learning processing. As shown in FIG. 1, the information processing device 100 learns a model using multiple types of datasets.

The information processing device 100 may provide information indicating the determined category to the terminal device 10. The information processing device 100 may distribute content including the images to be classified and the classification results of those images to the terminal device 10. In this case, the information processing device 100 may obtain various information from an external information processing device that provides a content distribution service, and perform information processing or provide information to the external information processing device based on the obtained various information.

3. Configuration of information processing device
Next, the configuration of the information processing device 100 according to the embodiment will be described with reference to Fig. 4. Fig. 4 is a diagram showing an example of the configuration of the information processing device 100 according to the embodiment. As shown in Fig. 4, the information processing device 100 has a communication unit 110, a storage unit 120, and a control unit 130. Note that the information processing device 100 may have an input unit (e.g., a keyboard, a mouse, etc.) that accepts various operations from an administrator of the information processing device 100, and a display unit (e.g., a liquid crystal display, etc.) that displays various information.

(Communication unit 110)
The communication unit 110 is realized by, for example, a network interface card (NIC) etc. The communication unit 110 is connected to a network via a wired or wireless connection, and transmits and receives information between the terminal device 10 and the information providing device 50.

(Memory unit 120)
The storage unit 120 is realized by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. As shown in FIG. 4 , the storage unit 120 according to the embodiment has a first type information storage unit 121, a second type information storage unit 122, a model information storage unit 123, and a category map storage unit 124.

(First type information storage unit 121)
The first type information storage unit 121 according to the embodiment stores various information related to the first type. Fig. 5 is a diagram illustrating an example of the first type information storage unit according to the embodiment. For example, the first type information storage unit 121 stores teacher data related to the first type used for generating a model. The first type information storage unit 121 illustrated in Fig. 5 includes items such as "data ID", "data", and "first category".

"Data ID" indicates identification information for identifying each data. Furthermore, "Data" indicates data identified by the Data ID. "Data" indicates image data, etc. to be classified. Figure 5 shows an example in which conceptual information such as "FDT1" is stored in "Data", but in reality, the image data, etc. to be classified, or a file path name indicating its storage location, etc., is stored.

"First category" refers to the classification result (first category) that is the correct answer corresponding to the corresponding data (image). "First category" refers to the classification result that is expected to be output by the model when the corresponding data is input to the model. Specifically, "first category" refers to the classification result (first category) that is expected to be output by the first partial model PM1 of the classification model M1 when the corresponding image is input to the classification model M1.

In the example of Figure 5, data FDT1 identified by data ID "FID1" indicates that the first category is "FC1". In the example of Figure 5, "FC1" to "FC8" are illustrated abstractly, but the "first category" is assumed to be a first category (value) that indicates a classification result related to the recognition target. For example, "FC1" to "FC8" are any of the first categories into which data (images) are classified.

The first type information storage unit 121 may store various information according to the purpose, not limited to the above. For example, the first type information storage unit 121 stores information indicating the type of teacher data, such as image or audio, in association with each piece of data. For example, the first type information storage unit 121 stores information indicating the type of data in association with each piece of data. In the example of FIG. 5, information indicating that the type of data is "image" is stored in association with each piece of data.

For example, the first type information storage unit 121 may store each piece of data in a manner that allows it to be identified as learning data, evaluation data, etc. For example, the first type information storage unit 121 stores learning data and evaluation data in a manner that allows it to be distinguished. The first type information storage unit 121 may store information that identifies whether each piece of data is learning data or evaluation data. The information processing device 100 learns a model based on each piece of data used as learning data and the first category (correct answer data). The information processing device 100 measures the accuracy of the model based on each piece of data used as evaluation data and the first category. The information processing device 100 measures the accuracy of the model by collecting a result of comparing the output result (classification result) output by the model when evaluation data is input with the first category.

(Second type information storage unit 122)
The second type information storage unit 122 according to the embodiment stores various information related to the second type. Fig. 6 is a diagram illustrating an example of the second type information storage unit according to the embodiment. For example, the second type information storage unit 122 stores teacher data related to the second type used for generating a model. The second type information storage unit 122 illustrated in Fig. 6 includes items such as "data ID", "data", and "second category".

"Data ID" indicates identification information for identifying each data. Furthermore, "Data" indicates data identified by the Data ID. "Data" indicates image data, etc. to be classified. Figure 6 shows an example in which conceptual information such as "SDT1" is stored in "Data", but in reality, the image data, etc. to be classified, or a file path name indicating its storage location, etc., is stored.

"Second category" refers to the correct classification result (second category) corresponding to the corresponding data (image). "Second category" refers to the classification result that is expected to be output by the model when the corresponding data is input to the model. Specifically, "second category" refers to the classification result (second category) that is expected to be output by the second partial model PM2 of the classification model M1 when the corresponding image is input to the classification model M1.

In the example of Figure 6, data SDT1 identified by data ID "SID1" indicates that the second category is "SC1". In the example of Figure 6, "SC1" to "SC3" are illustrated abstractly, but the "second category" is assumed to be a second category (value) that indicates the classification result related to the recognition target. For example, "SC1" to "SC3" are any of the second categories into which data (images) are classified.

The second type information storage unit 122 may store various information according to the purpose, not limited to the above. For example, the second type information storage unit 122 stores information indicating the type of teacher data, such as image or audio, in association with each piece of data. For example, the second type information storage unit 122 stores information indicating the type of data in association with each piece of data. In the example of FIG. 6, the data stores information indicating that the type is "image" in association with each piece of data.

For example, the second type information storage unit 122 may store data in a manner that allows each data item to be identified as learning data, evaluation data, etc. For example, the second type information storage unit 122 stores learning data and evaluation data in a manner that allows them to be distinguished. The second type information storage unit 122 may store information that identifies whether each data item is learning data or evaluation data. The information processing device 100 learns a model based on each data item used as learning data and the second category (correct answer data). The information processing device 100 measures the accuracy of the model based on each data item used as evaluation data and the second category. The information processing device 100 measures the accuracy of the model by collecting the results of comparing the output results (classification results) output by the model when evaluation data is input with the second category.

(Model information storage unit 123)
The model information storage unit 123 according to the embodiment stores information about a model. For example, the model information storage unit 123 stores information (model data) about a trained model (model) trained (generated) by a learning process. Fig. 7 is a diagram illustrating an example of a model information storage unit according to the embodiment. In the example shown in Fig. 7, the model information storage unit 123 includes items such as "model ID", "purpose", and "model data".

"Model ID" indicates identification information for identifying a model. "Use" indicates the use of the corresponding model. "Model data" indicates the data of the model. Figure 7 etc. shows an example in which conceptual information such as "MDT1" is stored in "model data", but in reality, various information that constitutes the model is included, such as information on the model configuration (network configuration) and information on parameters. For example, "model data" includes information including the nodes in each layer of the network, the functions employed by each node, the connection relationships between the nodes, and the connection coefficients set for the connections between the nodes.

In the example shown in FIG. 7, the model identified by the model ID "M1" (classification model M1) indicates that its use is "image classification." In other words, classification model M1 indicates that it is a model that outputs information indicating the classification of input image data. In addition, it indicates that the model data of classification model M1 is model data MDT1.

The model information storage unit 123 may store various information according to the purpose, not limited to the above. For example, the model information storage unit 123 may store information indicating details of the configuration within the model. For example, the model information storage unit 123 may store information indicating that the classification model M1 includes a first partial model PM1, a second partial model PM2, and a third partial model PM3, or information indicating the connection relationship of each partial model. For example, the model information storage unit 123 stores information on the network configuration indicating that data is input to the third partial model PM3 in the classification model M1, and the output of the third partial model PM3 is input to the first partial model PM1 and the second partial model PM2. Note that models such as the classification model M1 may be expected to be used as a program module that is part of artificial intelligence software. Models such as the classification model M1 may be programs.

(Category map storage unit 124)
The category map storage unit 124 according to the embodiment stores information related to the category map. For example, the category map storage unit 124 stores a category map used to convert a first category into a second category.

For example, the category map storage unit 124 stores a category map indicating the correspondence between a first category group corresponding to a first partial model and a second category group corresponding to a second partial model. For example, the category map storage unit 124 stores a list of first categories and a category map in which a second category corresponding to each first category in the list is associated.

The category map may be any information that can convert a first category into a second category. For example, the category map may be information that indicates a rule-based conversion rule for converting a first category into a second category. The category map may also be a conversion model that receives a first category as input and outputs a second category that corresponds to the input first category.

The information processing device 100 may generate a category map. For example, the information processing device 100 may generate a category map using the first category output by the first partial model PM1 for an image to which a second category has been assigned. For example, the information processing device 100 may apply the first head to an image to which an in-house data category has been assigned (no box has been assigned). For example, the information processing device 100 may generate a category map based on the correlation between the first category of the image determined (predicted) by the first partial model PM1 and the correct label (second category) assigned to the image. For example, the information processing device 100 may examine the correlation between the predicted first category of the image and the second category assigned to the image, and generate a category map based on the correlation.

(Control unit 130)
Returning to the explanation of Fig. 4, the control unit 130 is a controller, and is realized, for example, by a CPU (Central Processing Unit) or an MPU (Micro Processing Unit) executing various programs (corresponding to examples of information processing programs such as a learning program and a generation program) stored in a storage device inside the information processing device 100 using a RAM as a working area. The control unit 130 is also a controller, and is realized, for example, by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

As shown in FIG. 4, the control unit 130 has an acquisition unit 131, a learning unit 132, a determination unit 133, and a provision unit 134, and realizes or executes the functions and actions of the information processing described below. Note that the internal configuration of the control unit 130 is not limited to the configuration shown in FIG. 4, and may be other configurations that perform the information processing described below.

(Acquisition unit 131)
The acquisition unit 131 acquires various information. The acquisition unit 131 acquires various information from an external information processing device such as the terminal device 10 or the information providing device 50. The acquisition unit 131 may receive various information indicated in the first type information storage unit 121, the second type information storage unit 122, the model information storage unit 123, the category map storage unit 124, or the like from the external information processing device.

The acquisition unit 131 acquires various information from the storage unit 120. The acquisition unit 131 acquires various information from the first type information storage unit 121, the second type information storage unit 122, the model information storage unit 123, the category map storage unit 124, etc.

The acquisition unit 131 acquires first character information, which is a first type of character information, and second character information, which is a second type of character information different from the first type. The acquisition unit 131 acquires selection information for selecting the type of character information to be output by the conversion model. The acquisition unit 131 acquires selection information for selecting a learning type, which is a type corresponding to the learning character information to be input to the conversion model.

The acquisition unit 131 acquires a classification model including a first partial model that outputs a first classification result indicating a first category corresponding to the input information, and a second partial model that outputs a second classification result indicating a second category corresponding to the input information. The acquisition unit 131 acquires classification target information to be classified. The acquisition unit 131 acquires a classification model including a third partial model that outputs feature information of the input information in response to input of the input information, a first partial model that outputs a first classification result using the output of the third partial model, and a second partial model that outputs a second classification result using the output of the third partial model. The acquisition unit 131 acquires classification target information, which is an image to be classified.

(Learning Unit 132)
The learning unit 132 learns various information. The learning unit 132 generates various models through a learning process. For example, the learning unit 132 performs the learning process based on information stored in the first type information storage unit 121, the second type information storage unit 122, etc. The learning unit 132 generates various models based on information stored in the first type information storage unit 121, the second type information storage unit 122, etc.

For example, the learning unit 132 learns the classification model M1 using the learning data stored in the first type information storage unit 121 and the second type information storage unit 122.

The learning unit 132 uses multiple types of datasets to learn the classification model M1. The learning unit 132 uses two types of datasets, a first type of dataset and a second type of dataset, to learn the classification model M1.

The learning unit 132 learns the classification model M1 using public data DS1, which is a first type of dataset. In learning using the public data DS1, the learning unit 132 learns the classification model M1 by performing backpropagation on the path from the first partial model PM1 to the third partial model PM3.

The learning unit 132 learns the classification model M1 using the in-house data DS2, which is a second type of dataset. In learning using the in-house data DS2, the learning unit 132 learns the classification model M1 by performing backpropagation on the path from the second partial model PM2 to the third partial model PM3.

(Determination unit 133)
The determination unit 133 determines various information. The determination unit 133 determines the category to which the classification target information belongs by a determination process. For example, the determination unit 133 performs the determination process based on information stored in the model information storage unit 123 or the like. The determination unit 133 determines the category to which the classification target information belongs by using a model stored in the model information storage unit 123. The classification target information is input to the classification model M1, and the category to which the classification target information belongs is determined by using the output of the classification model M1.

The determination unit 133 inputs the classification target information as input information into the classification model, and determines a target category to which the classification target information belongs based on the first candidate category indicated by the first classification result output by the first partial model and the second candidate category indicated by the second classification result output by the second partial model. The determination unit 133 uses feature information of the classification target information output by the third partial model to determine a target category to which the classification target information belongs based on the first candidate category indicated by the first classification result output by the first partial model and the second candidate category indicated by the second classification result output by the second partial model.

The determination unit 133 determines the target category to which the classification target information belongs based on the converted first category obtained by converting the first candidate category indicated by the first classification result output by the first partial model into a format corresponding to the second partial model, and the second candidate category. The determination unit 133 determines the target category to which the classification target information belongs based on the converted first category into which the first candidate category is converted, and the second candidate category, using a category map indicating the correspondence between the first category group corresponding to the first partial model and the second category group corresponding to the second partial model. The determination unit 133 determines the target category to which the classification target information belongs based on the converted first category into which the first candidate category is converted, and the second candidate category, using the category map used for converting the first category into the second category.

When the converted first category and the second candidate category are different, the determination unit 133 determines the target category to which the classification target information belongs based on a predetermined criterion. When the converted first category and the second candidate category are different, the determination unit 133 determines the target category to which the classification target information belongs based on a comparison between the first score corresponding to the converted first category and the second score corresponding to the second candidate category.

If the second score is equal to or greater than the first score, the determination unit 133 determines the second category as the target category to which the classification target information belongs. If the first score is greater than the second score, the determination unit 133 determines the converted first category as the target category to which the classification target information belongs.

When the first category after conversion is different from the second candidate category, the determination unit 133 determines the second candidate category as the target category to which the classification target information belongs. When the first category after conversion is different from the second candidate category, the determination unit 133 determines the first category after conversion as the target category to which the classification target information belongs.

When the first category after conversion matches the second candidate category, the determination unit 133 determines the second candidate category as the target category to which the classification target information belongs.When the first candidate category matches the second candidate category, the determination unit 133 determines the second candidate category as the target category to which the classification target information belongs.

(Providing Unit 134)
The providing unit 134 provides various information. For example, the providing unit 134 provides various information to the terminal device 10 and the information providing device 50. The providing unit 134 transmits various information to the terminal device 10 and the information providing device 50. The providing unit 134 provides various information based on the various information acquired by the acquiring unit 131. The providing unit 134 provides various information based on the various information learned by the learning unit 132. The providing unit 134 provides various information based on the model learned by the learning unit 132. The providing unit 134 provides various information based on the various information determined by the determining unit 133.

For example, the providing unit 134 provides information indicating the classification result of the classification target information to the recipient of the classification target information. When the classification target information is provided from the terminal device 10, the providing unit 134 transmits information indicating the classification result of the classification target information to the terminal device 10.

[4. Information processing flow]
Next, a flow of information processing by the information processing device 100 will be described with reference to Fig. 8. Fig. 8 is a flowchart showing an example of information processing according to the embodiment.

As shown in FIG. 8, the information processing device 100 acquires a classification model including a first partial model that outputs a first classification result and a second partial model that outputs a second classification result (step S101). For example, the information processing device 100 acquires a classification model M1 from the model information storage unit 123.

The information processing device 100 acquires classification target information to be classified (step S102). For example, the information processing device 100 receives classification target information TG1 to be classified from the terminal device 10.

Then, the information processing device 100 inputs the classification target information as input information to the classification model (step S103). For example, the information processing device 100 inputs the classification target information TG1 to the classification model M1.

Then, the information processing device 100 determines a target category to which the classification target information belongs based on the first candidate category indicated by the first classification result output by the first partial model and the second candidate category indicated by the second classification result output by the second partial model (step S102). For example, the information processing device 100 determines a target category to which the classification target information TG1 belongs based on the converted first candidate category obtained by converting the first candidate category using the category map CM1 and the second candidate category.

[5. Application examples of information processing systems]
An application example of the above-mentioned information processing system 1 will now be described with reference to Fig. 9. Fig. 9 is a diagram showing an example of an application of the information processing system. Each element in Fig. 9 will now be described.

The first processing group PS11, labeled "Training (Multi-task learning)" in FIG. 9, indicates the configuration and data related to the learning process, and indicates that the classification model M1 is trained using OpenImages data and in-house data. The part labeled "Detector" in FIG. 9 indicates the model, and corresponds to the classification model M1 trained by the first processing group PS11.

The second processing group PS12, labeled "Inference" in Figure 9, shows the configuration and data related to the category determination process (inference process), and indicates that the inference process is performed using the classification model M1, category map, etc. learned by the first processing group PS11.

The OpenImages data in FIG. 9 shows data used as a first type of dataset. The OpenImages data in FIG. 9 corresponds to the public data DS1 in FIG. 1.

The in-house data in FIG. 9 shows data used as a second type of dataset. The in-house data in FIG. 9 corresponds to the in-house data DS2 in FIG. 1.

The first head in FIG. 9 corresponds to the first partial model PM1. For example, the first head in FIG. 9 is a sublayer in which learning specialized for the task of classifying images into a first category (first classification) is performed. The first head in FIG. 9 is trained with OpenImages data.

The second head in FIG. 9 corresponds to the second partial model PM2. For example, the second head in FIG. 9 is a sublayer in which learning specialized for the task of classifying images into a second category (second classification) is performed. The second head in FIG. 9 is trained with in-house data.

Backbone in Figure 9 corresponds to the third partial model PM3. For example, Backbone in Figure 9 is a sublayer that learns general features about images. Backbone in Figure 9 is trained on both OpenImages data and in-house data.

The category map in FIG. 9 corresponds to the category map CM1 in FIG. 2. For example, the category map in FIG. 9 is information used to map a first category determined based on the output result of the first head to a second category.

The corresponding category in FIG. 9 corresponds to the target category TC1 in FIG. 2. For example, the corresponding category in FIG. 9 is information indicating a category determined by taking into account both the output results of the first head and the output results of the second head.

6. Effects
As described above, the information processing device 100 according to the embodiment includes the acquisition unit 131 and the determination unit 133. The acquisition unit 131 acquires a classification model including a first partial model that outputs a first classification result indicating a first category corresponding to the input information and a second partial model that outputs a second classification result indicating a second category corresponding to the input information, and classification target information to be classified. The determination unit 133 inputs the classification target information as input information to the classification model, and determines a target category to which the classification target information belongs, based on a first candidate category indicated by the first classification result output by the first partial model and a second candidate category indicated by the second classification result output by the second partial model.

In this way, the information processing device 100 according to the embodiment can determine the category to which the classification target information belongs based on multiple classification results by using a classification model including a first partial model that outputs a first classification result and a second partial model that outputs a second classification result, and therefore can appropriately determine the category for the classification target.

In addition, in the information processing device 100 according to the embodiment, the acquisition unit 131 acquires a classification model including a third partial model that outputs feature information of the input information in response to the input of the input information, a first partial model that outputs a first classification result using the output of the third partial model, and a second partial model that outputs a second classification result using the output of the third partial model.

In this way, the information processing device 100 according to the embodiment can determine the category to which the information to be classified belongs based on appropriately extracted information by using a classification model including a third partial model that outputs feature information of the input information, and can therefore appropriately determine the category for the classification target.

In addition, in the information processing device 100 according to the embodiment, the determination unit 133 uses the feature information of the classification target information output by the third partial model to determine the target category to which the classification target information belongs, based on the first candidate category indicated by the first classification result output by the first partial model and the second candidate category indicated by the second classification result output by the second partial model.

In this way, the information processing device 100 according to the embodiment can appropriately determine a category for the classification target because both the first partial model and the second partial model use the feature information of the classification target information output by the third partial model, and the feature information can be used in common.

In addition, in the information processing device 100 according to the embodiment, the acquisition unit 131 acquires classification target information, which is the image to be classified.

In this way, the information processing device 100 according to the embodiment can appropriately determine a category for an image.

In addition, in the information processing device 100 according to the embodiment, the determination unit 133 determines the target category to which the classification target information belongs based on the converted first category obtained by converting the first candidate category indicated by the first classification result output by the first partial model into a format corresponding to the second partial model, and the second candidate category.

In this way, the information processing device 100 according to the embodiment uses the converted first category obtained by converting the first candidate category indicated by the first classification result into a format corresponding to the second partial model, thereby resulting in a classification result based on a common category classification, and thus making it possible to appropriately determine a category for the classification target.

In addition, in the information processing device 100 according to the embodiment, the determination unit 133 determines the target category to which the classification target information belongs based on the converted first category into which the first candidate category is converted and the second candidate category, using a category map indicating the correspondence between a first category group corresponding to the first partial model and a second category group corresponding to the second partial model.

In this way, the information processing device 100 according to the embodiment can appropriately map the first category to the second category by using a category map that indicates the correspondence between the first category group and the second category group, and can therefore appropriately determine the category for the classification target.

In addition, in the information processing device 100 according to the embodiment, the determination unit 133 uses a category map used to convert the first category into the second category to determine the target category to which the classification target information belongs based on the converted first category into which the first candidate category is converted and the second candidate category.

In this way, the information processing device 100 according to the embodiment can use the category map to convert the first candidate category into the first category after conversion that corresponds to the second category, and therefore can appropriately determine the category for the classification target.

In addition, in the information processing device 100 according to the embodiment, when the first category after conversion is different from the second candidate category, the determination unit 133 determines the target category to which the classification target information belongs based on a predetermined criterion.

In this way, when the converted first category and the second candidate category are different, the information processing device 100 according to the embodiment takes both into consideration when determining the target category to which the classification target information belongs, thereby making it possible to appropriately determine the category for the classification target.

In addition, in the information processing device 100 according to the embodiment, when the converted first category and the second candidate category are different, the determination unit 133 determines the target category to which the classification target information belongs based on a comparison between the first score corresponding to the converted first category and the second score corresponding to the second candidate category.

In this way, the information processing device 100 according to the embodiment can appropriately determine the category for the classification target by comparing the first score corresponding to the first category after conversion with the second score corresponding to the second candidate category to determine the target category to which the classification target information belongs, taking into account the confidence of the classification results of each partial model.

In addition, in the information processing device 100 according to the embodiment, if the second score is equal to or greater than the first score, the determination unit 133 determines the second category as the target category to which the classification target information belongs.

In this way, when the second score is equal to or greater than the first score, the information processing device 100 according to the embodiment determines the second category as the target category to which the classification target information belongs, thereby taking into account the confidence of the classification results of each partial model and appropriately determining the category for the classification target.

In addition, in the information processing device 100 according to the embodiment, if the first score is greater than the second score, the determination unit 133 determines the converted first category as the target category to which the classification target information belongs.

In this way, when the first score is greater than the second score, the information processing device 100 according to the embodiment determines the first category after conversion as the target category to which the classification target information belongs, thereby taking into account the confidence of the classification results of each partial model and appropriately determining the category for the classification target.

In addition, in the information processing device 100 according to the embodiment, when the first category after conversion is different from the second candidate category, the determination unit 133 determines the second candidate category as the target category to which the classification target information belongs.

In this way, when the first category after conversion differs from the second candidate category, the information processing device 100 according to the embodiment determines the second candidate category as the target category to which the classification target information belongs, thereby placing emphasis on the category that does not require category conversion and appropriately determining the category for the classification target.

In addition, in the information processing device 100 according to the embodiment, when the first converted category differs from the second candidate category, the determination unit 133 determines the first converted category as the target category to which the classification target information belongs.

In this way, when the first category after conversion differs from the second candidate category, the information processing device 100 according to the embodiment determines the second candidate category as the target category to which the classification target information belongs, thereby placing emphasis on the output of a partial model trained with a large amount of data and appropriately determining a category for the classification target.

In addition, in the information processing device 100 according to the embodiment, when the first category after conversion matches the second candidate category, the determination unit 133 determines the second candidate category as the target category to which the classification target information belongs.

In this way, when the first category after conversion matches the second candidate category, the information processing device 100 according to the embodiment determines the matching category as the target category to which the classification target information belongs, thereby making it possible to appropriately determine the category for the classification target.

In addition, in the information processing device 100 according to the embodiment, when the first candidate category and the second candidate category match, the determination unit 133 determines the second candidate category as the target category to which the classification target information belongs.

In this way, when the first candidate category and the second candidate category match, the information processing device 100 according to the embodiment determines the matching category as the target category to which the classification target information belongs, thereby making it possible to appropriately determine the category for the classification target.

7. Hardware Configuration
The information processing device 100 and the terminal device 10 according to the above-described embodiment are realized by a computer 1000 having a configuration as shown in Fig. 10, for example. The information processing device 100 will be described below as an example. Fig. 10 is a diagram showing an example of a hardware configuration. The computer 1000 is connected to an output device 1010 and an input device 1020, and has a configuration in which a calculation device 1030, a primary storage device 1040, a secondary storage device 1050, an output I/F (Interface) 1060, an input I/F 1070, and a network I/F 1080 are connected by a bus 1090.

The arithmetic device 1030 operates based on programs stored in the primary storage device 1040 and the secondary storage device 1050, programs read from the input device 1020, and the like, and executes various processes. The arithmetic device 1030 is realized, for example, by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or the like.

The primary storage device 1040 is a memory device such as a RAM (Random Access Memory) that primarily stores data used by the arithmetic device 1030 for various calculations. The secondary storage device 1050 is a storage device in which data used by the arithmetic device 1030 for various calculations and various databases are registered, and is realized by a ROM (Read Only Memory), a HDD (Hard Disk Drive), a SSD (Solid State Drive), a flash memory, or the like. The secondary storage device 1050 may be an internal storage device or an external storage device. The secondary storage device 1050 may be a removable storage medium such as a USB memory or a SD (Secure Digital) memory card. The secondary storage device 1050 may be a cloud storage device (online storage device), a NAS (Network Attached Storage), a file server, or the like.

The output I/F 1060 is an interface for transmitting information to be output to an output device 1010 that outputs various types of information, such as a display, projector, printer, etc., and is realized by a connector conforming to a standard such as USB (Universal Serial Bus), DVI (Digital Visual Interface), or HDMI (registered trademark) (High Definition Multimedia Interface). The input I/F 1070 is an interface for receiving information from various input devices 1020, such as a mouse, keyboard, keypad, button, scanner, etc., and is realized by a USB, etc.

In addition, the output I/F 1060 and the input I/F 1070 may be wirelessly connected to the output device 1010 and the input device 1020, respectively. That is, the output device 1010 and the input device 1020 may be wireless devices.

The output device 1010 and the input device 1020 may be integrated together, such as a touch panel. In this case, the output I/F 1060 and the input I/F 1070 may also be integrated together as an input/output I/F.

The input device 1020 may be a device that reads information from, for example, an optical recording medium such as a CD (Compact Disc), a DVD (Digital Versatile Disc), or a PD (Phase change rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

The network I/F 1080 receives data from other devices via the network N and sends it to the computing device 1030, and also transmits data generated by the computing device 1030 to other devices via the network N.

The arithmetic unit 1030 controls the output device 1010 and the input device 1020 via the output I/F 1060 and the input I/F 1070. For example, the arithmetic unit 1030 loads a program from the input device 1020 or the secondary storage device 1050 onto the primary storage device 1040 and executes the loaded program.

For example, when the computer 1000 functions as the information processing device 100, the arithmetic unit 1030 of the computer 1000 executes a program loaded onto the primary storage device 1040 to realize the functions of the control unit 130. The arithmetic unit 1030 of the computer 1000 may also load a program acquired from another device via the network I/F 1080 onto the primary storage device 1040 and execute the loaded program. The arithmetic unit 1030 of the computer 1000 may also cooperate with other devices via the network I/F 1080 and use the functions and data of a program by calling them from other programs of the other devices.

[8. Other]
Furthermore, among the processes described in the above embodiments and modifications, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by a known method. In addition, the information including the processing procedures, specific names, various data and parameters shown in the above documents and drawings can be changed arbitrarily unless otherwise specified. For example, the various information shown in each drawing is not limited to the illustrated information.

In addition, each component of each device shown in the figure is a functional concept, and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc.

The above-described embodiments and variations can be combined as appropriate to the extent that they do not cause inconsistencies in the processing content.

In addition, the above-mentioned "section, module, unit" can be read as "means" or "circuit." For example, an acquisition unit can be read as an acquisition means or an acquisition circuit.

REFERENCE SIGNS LIST 1 Information processing system 100 Information processing device 121 First type information storage unit 122 Second type information storage unit 123 Model information storage unit 130 Control unit 131 Acquisition unit 132 Learning unit 133 Determination unit 134 Providing unit 10 Terminal device 50 Information providing device N Network

Claims (15)

an acquisition unit that acquires classification models including a first partial model that is trained by a learning process using a first type of dataset so that, among first scores for each of a plurality of first categories included in a first category group corresponding to the first type of dataset, a first score for a first category corresponding to input information becomes large, and outputs, as a first classification result, a plurality of first scores corresponding to each of the plurality of first categories corresponding to the input information; and a second partial model that is trained by a learning process using a second type of dataset different from the first type of dataset so that, among second scores for each of a plurality of second categories included in a second category group corresponding to the second type of dataset and having a value comparable to the first score , a second score for a second category corresponding to input information becomes large, and outputs, as a second classification result, a plurality of second scores corresponding to each of the plurality of second categories corresponding to the input information; and
a determination unit that, by inputting the classification target information as the input information into the classification model, converts the first candidate category into a category included in the second category group using a first candidate category which is the first category having a maximum value among the multiple first scores which are the first classification result output by the first partial model, and a second candidate category which is the second category having a maximum value among the multiple second scores which are the second classification result output by the second partial model, and determines the second candidate category as a target category which is a category to which the classification target information belongs , when the second candidate category matches a converted first category;
Equipped with
The determination unit is
an information processing device that determines the target category to which the classification target information belongs based on whether the converted first category into which the first candidate category is converted matches the second candidate category using a category map used to convert the first category into the second category . The acquisition unit is
The information processing device according to claim 1, further comprising: a classification model that includes a third partial model that outputs feature information of the input information in response to input of the input information; the first partial model that outputs the first classification result using an output of the third partial model; and the second partial model that outputs the second classification result using an output of the third partial model. The determination unit is
3. The information processing device according to claim 2, further comprising: determining, using feature information of the classification target information output by the third partial model, the target category to which the classification target information belongs, based on the first candidate category indicated by the first classification result output by the first partial model and the second candidate category indicated by the second classification result output by the second partial model. The acquisition unit is
4. The information processing apparatus according to claim 1, further comprising: acquiring the classification target information which is an image to be classified. The determination unit is
The information processing device according to any one of claims 1 to 4, characterized in that the target category to which the classification target information belongs is determined based on whether the converted first category into which the first candidate category is converted matches the second candidate category, using the category map indicating a correspondence relationship between a first category group corresponding to the first partial model and a second category group corresponding to the second partial model. The determination unit is
An information processing device according to any one of claims 1 to 5, characterized in that when the converted first category and the second candidate category are different, the target category to which the classification target information belongs is determined based on a comparison of information relating to the converted first category and information relating to the second candidate category. The determination unit is
The information processing device according to claim 6, characterized in that, when the converted first category and the second candidate category are different, the target category to which the classification target information belongs is determined based on a comparison of the magnitude relationship between the first score of the first candidate category and the second score of the second candidate category among the multiple first scores output by the first partial model. The determination unit is
8. The information processing device according to claim 7, further comprising: determining, when the second score of the second candidate category is equal to or greater than the first score of the first candidate category, the second category as the target category to which the classification target information belongs. The determination unit is
9. The information processing device according to claim 7, wherein when the first score of the first candidate category is greater than the second score of the second candidate category , the converted first category is determined to be the target category to which the classification target information belongs. The determination unit is
The information processing device according to any one of claims 1 to 6 , characterized in that, when the first category after the conversion is different from the second candidate category, the second candidate category is determined to be the target category to which the classification target information belongs. The determination unit is
An information processing device according to any one of claims 1 to 6, characterized in that when the converted first category and the second candidate category are different, the converted first category is determined to be the target category to which the classification target information belongs. The determination unit is
An information processing device according to any one of claims 1 to 6, characterized in that if the first category after the conversion matches the second candidate category, the second candidate category is determined to be the target category to which the classification target information belongs. The determination unit is
The information processing device according to any one of claims 1 to 12 , characterized in that, when the first candidate category and the second candidate category match, the second candidate category is determined to be the target category to which the classification target information belongs. 1. A computer-implemented information processing method, comprising:
an acquisition step of acquiring a classification model including a first partial model that is trained by a learning process using a first type of dataset so that, among first scores of each of a plurality of first categories included in a first category group corresponding to the first type of dataset, a first score of a first category corresponding to input information becomes large, and outputs, as a first classification result, a plurality of first scores corresponding to each of the plurality of first categories corresponding to the input information; and a second partial model that is trained by a learning process using a second type of dataset different from the first type of dataset so that, among second scores of each of a plurality of second categories included in a second category group corresponding to the second type of dataset and having a value comparable to the first score , a second score of a second category corresponding to the input information becomes large, and outputs, as a second classification result, a plurality of second scores corresponding to each of the plurality of second categories corresponding to the input information; and classification target information to be classified;
a determination process for inputting the classification target information as the input information into the classification model, and using a first candidate category which is the first category having the largest value among the plurality of first scores which are the first classification result output by the first partial model, and a second candidate category which is the second category having the largest value among the plurality of second scores which are the second classification result output by the second partial model, to convert the first candidate category into a category included in the second category group, when the second candidate category matches the converted first category; and
Including,
The determining step includes:
Using a category map used for converting the first category into the second category, the target category to which the classification target information belongs is determined based on whether the converted first category into which the first candidate category is converted matches the second candidate category.
23. An information processing method comprising: an acquisition step of acquiring a classification model including a first partial model that is trained by a learning process using a first type of dataset so that among first scores of each of a plurality of first categories included in a first category group corresponding to the first type of dataset, a first score of a first category corresponding to input information becomes large, and outputs a plurality of first scores corresponding to each of the plurality of first categories corresponding to the input information as a first classification result, and a second partial model that is trained by a learning process using a second type of dataset different from the first type of dataset so that among second scores, which are second scores of each of a plurality of second categories included in a second category group corresponding to the second type of dataset and have values comparable to the first scores , a second score of a second category corresponding to input information becomes large, and outputs a plurality of second scores corresponding to each of the plurality of second categories corresponding to the input information as a second classification result, and classification target information to be classified;
a determination step of inputting the classification target information as the input information into the classification model, and using a first candidate category which is the first category having the largest value among the plurality of first scores which are the first classification result output by the first partial model, and a second candidate category which is the second category having the largest value among the plurality of second scores which are the second classification result output by the second partial model, to convert the first candidate category into a category included in the second category group, when the second candidate category matches the converted first category; and determining the second candidate category as a target category which is the category to which the classification target information belongs.
Run the following on your computer :
The determination procedure includes:
an information processing program for determining the target category to which the information to be classified belongs based on whether or not the converted first category into which the first candidate category is converted matches the second candidate category using a category map used to convert the first category into the second category .

Images