JP6784039B2 - Data estimation device, data estimation method, and data estimation program - Google Patents

Description

The present invention relates to a data estimation device or the like that estimates information about data.

The deep learning method is an example of machine learning that achieves practical accuracy, and is utilized (or examined and implemented) in various fields. Among machine learning, the supervised learning method provides a learning model that expresses the relationship between the value of the explanatory variable and the value of the objective variable with respect to the teacher data in which the value of the explanatory variable and the value (label) of the objective variable are associated with each other. create. The teacher data is created by the user based on, for example, the accumulated information or the collected information.

Machine learning can create a learning model for data such as image data and text data, and is used when detecting a person or an object in a certain area. Image data is created, for example, by a surveillance camera that captures an area. In general, since a certain area is constantly imaged by a plurality of surveillance cameras, a huge amount of image data to be created is accumulated.

For example, an example of machine learning is disclosed in Patent Document 1 and Patent Document 2.

The classification model learning device disclosed in Patent Document 1 calculates the distance between the prediction data to be predicted and each teacher data, and for each prediction data, the teacher data in which the calculated distance is within a predetermined range is obtained. Identify. Next, the classification model learning device predicts the label related to the predicted data based on the value (label) of the objective variable related to the specified teacher data.

The causal relationship model generator disclosed in Patent Document 2 creates a causal relationship model that represents a causal relationship between an event of interest with respect to an observation target and a causal event that is the cause of the event. In the causal relationship model generator, among the components constituting the observation target, the components related to the causal event (or the event of interest) and the inclusion relationship with others are hierarchically expressed. It is expressed as data with the above attributes. Based on the data, the causal relationship model generator creates a causal relationship model representing a causal relationship between an event of interest with respect to an observation target and the causal event.

JP-A-2009-228686 Japanese Unexamined Patent Publication No. 2004-126641

However, regardless of whether the apparatus disclosed in Patent Document 1 or Patent Document 2 is used, if the target search condition representing the condition for identifying the target data is strict, the target data can be obtained with high accuracy. It is not always possible to judge. The reason for this is that when the target search condition is a strict condition, it is generally difficult to create a model that estimates with high accuracy whether or not the target search condition is satisfied.

Therefore, a main object of the present invention is to provide a data estimation device or the like capable of correctly determining target data even if the target search conditions are strict.

In order to achieve the above object, in one aspect of the invention, the data estimation device
By applying the first model to a certain data according to the first model that can estimate whether or not the first search condition is satisfied by calculating the first target data according to a predetermined first processing procedure. A first calculation means for calculating the first calculation result calculated by the predetermined first processing procedure, and
According to a second model in which it is possible to estimate whether or not a second search condition different from the first search condition is satisfied by performing an operation on the second target data according to a predetermined second processing procedure, for the certain data. By applying the second model, a second calculation means for calculating the second calculation result calculated in the predetermined second processing procedure, and
The calculation result calculated in the first model when the first model is applied to the third target data, and calculated in the second model when the second model is applied to the third target data. By performing a calculation on the calculation result according to a predetermined third processing procedure, the first search condition for the first calculation result and the second calculation result for the third target data. And, by applying the third model that can estimate whether or not both of the second search conditions are satisfied, whether the first search condition and the second search condition are satisfied with respect to the certain data. It is provided with a third calculation means for estimating whether or not it is.

In addition, as another viewpoint of the present invention, the data estimation method is
By applying the first model to a certain data according to the first model that can estimate whether or not the first search condition is satisfied by calculating the first target data according to a predetermined first processing procedure. Calculate the first calculation result calculated by the predetermined first processing procedure,
According to a second model in which it is possible to estimate whether or not a second search condition different from the first search condition is satisfied by performing an operation on the second target data according to a predetermined second processing procedure, for the certain data. By applying the second model, the second calculation result calculated by the predetermined second processing procedure is calculated.
The calculation result calculated in the first model when the first model is applied to the third target data, and calculated in the second model when the second model is applied to the third target data. By performing a calculation on the calculation result according to a predetermined third processing procedure, the first search condition for the first calculation result and the second calculation result for the third target data. And, by applying the third model that can estimate whether or not the second search condition is satisfied, whether or not the first search condition and the second search condition are satisfied with respect to the certain data. Estimate.

Also, from another point of view of the present invention, the data estimation program
By applying the first model to a certain data according to the first model that can estimate whether or not the first search condition is satisfied by calculating the first target data according to a predetermined first processing procedure. The first calculation function for calculating the first calculation result calculated by the predetermined first processing procedure, and
According to a second model in which it is possible to estimate whether or not a second search condition different from the first search condition is satisfied by performing an operation on the second target data according to a predetermined second processing procedure, for the certain data. By applying the second model, a second calculation function for calculating the second calculation result calculated in the predetermined second processing procedure, and
The calculation result calculated in the first model when the first model is applied to the third target data, and calculated in the second model when the second model is applied to the third target data. By performing a calculation on the calculation result according to a predetermined third processing procedure, the first search condition for the first calculation result and the second calculation result for the third target data. And, by applying the third model that can estimate whether or not the second search condition is satisfied, whether or not the first search condition and the second search condition are satisfied with respect to the certain data. The computer is provided with a third calculation function for estimating the data.

Further, the same purpose is realized by a computer-readable recording medium for recording the program.

According to the data estimation device or the like according to the present invention, the target data can be correctly determined even if the target search condition is strict.

It is a block diagram which shows the structure which the data estimation apparatus which concerns on 1st Embodiment of this invention has. It is a flowchart which shows the outline of the processing flow in the data estimation apparatus which concerns on 1st Embodiment. It is a flowchart which shows the detailed flow of the process in step S101 and step S102 shown in FIG. It is a flowchart which shows the detailed flow of the process in step S103 and step S104 shown in FIG. It is a flowchart which shows the detailed flow of the process in step S104 and step S105 shown in FIG. It is a figure which conceptually represents the inclusion relation about the data for which a target search condition is satisfied. It is a figure which conceptually represents the learning image information stored in the learning image information storage part. It is a figure which conceptually represents the display image information stored in the display image information storage part. It is a figure which conceptually represents the unprocessed image information stored in the unprocessed image information storage part. It is a figure which conceptually represents an example of the structure which a user interface has. It is a figure which conceptually represents an example of the label information stored in the label information storage part. It is a figure which conceptually represents an example of the image label information stored in the image label storage part. It is a figure which conceptually represents an example of model information. It is a figure which conceptually represents the feature information stored in the feature information storage part. It is a figure which conceptually represents the feature information stored in the feature information storage part. It is a figure which conceptually represents an example of the structure which CNN has. It is a figure which conceptually represents an example of the image label information stored in the image label storage part. It is a figure which conceptually represents an example of the goodness-of-fit learning information stored in the goodness-of-fit learning information storage part. It is a figure which conceptually represents an example of the goodness-of-fit model information stored in the goodness-of-fit model information storage part. It is a figure which conceptually represents an example of the goodness-of-fit determination information stored in the goodness-of-fit determination information storage part. It is a figure which conceptually represents an example of the goodness-of-fit information stored in the goodness-of-fit information storage part. It is a figure which conceptually represents an example of the prediction label information stored in the prediction label information storage part. It is a block diagram which shows the structure which the data estimation apparatus which concerns on 2nd Embodiment of this invention has. It is a block diagram which shows schematic the hardware configuration example of the calculation processing apparatus which can realize the data estimation apparatus which concerns on each embodiment of this invention.

Next, an embodiment of the present invention will be described in detail with reference to the drawings.

<First Embodiment>
The configuration of the data estimation device 600 according to the first embodiment of the present invention will be described in detail with reference to FIG. FIG. 1 is a block diagram showing a configuration of the data estimation device 600 according to the first embodiment of the present invention.

The data estimation device 600 according to the first embodiment of the present invention is roughly classified into an image information storage unit 100, a label input unit 200, a feature amount creation unit 310, a conformity degree creation unit 320, and a frame information creation unit. It has a 410, a conformity feature amount creating unit 420, and a label selection unit 500.

The image information storage unit 100 includes a learning image information storage unit 101, a display image information storage unit 102, and an unprocessed image information storage unit 103. The label input unit 200 includes a control unit 201, a label information storage unit 202, and an image label storage unit 203. The feature amount creation unit 310 includes a feature amount model creation unit 311, a model information storage unit 312, a feature amount calculation unit 313, and a feature amount information storage unit 314. The goodness-of-fit creation unit 320 is compatible with the goodness-of-fit learning information creation unit 321, the goodness-of-fit learning information storage unit 322, the goodness-of-fit model creation unit 323, the goodness-of-fit model information storage unit 324, and the goodness-of-fit calculation unit 325. It has a goodness-of-fit information storage unit 326. The frame information creation unit 410 includes a frame creation unit 411 and a frame information storage unit 412. The goodness-of-fit feature amount creating unit 420 has a goodness-of-fit feature amount creating unit 421 and a goodness-of-fit determination information storage unit 422. The label selection unit 500 includes a label determination unit 501 and a prediction label information storage unit 502.

Next, an example of information input to the data estimation device 600 according to the first embodiment will be described with reference to FIGS. 7 to 9. FIG. 7 is a diagram conceptually representing the learning image information stored in the learning image information storage unit 101. FIG. 8 is a diagram conceptually representing the display image information stored in the display image information storage unit 102. FIG. 9 is a diagram conceptually representing the unprocessed image information stored in the unprocessed image information storage unit 103.

With reference to the training image information illustrated in FIG. 7, the training image information is estimated (predicted, determined) with respect to the moving image data identifier (hereinafter, the identifier is referred to as “ID”) capable of identifying the moving image data and the moving image data. It is associated with an attribute that represents the content. As the learning image information, the moving image data ID may be further associated with the moving image data represented by the moving image data ID.

For example, in the learning image information illustrated in FIG. 7, the moving image data ID “V1002” and the attribute “determines whether or not there is a person wearing a hat” are associated with each other. For the video data represented by the video data ID "V1002", a label indicating whether or not the search condition "whether there is a person wearing a hat" is satisfied is input as described later with reference to FIG. In this case, it indicates that the moving image data is displayed by the control unit 201.

When a frame (image data) is searched for a search condition (hereinafter, referred to as "target search condition") that holds for the search target in the learning image information storage unit 101, the search condition is relaxed. It contains video data that is the basis for creating a model that can estimate labels related to conditions. By adding a label indicating "whether or not there is a person wearing a hat" to the learning image information, for example, as a basis for calculating a calculated value as described with reference to FIG. 14 or FIG. Data is calculated.

The learning image information may be further associated with different items, and is not limited to the above-mentioned example. Further, the learning image information may include the label ID in the label information illustrated in FIG. 11, and in this case, the label related to the label ID stored in the label information storage unit 202 is input. Hereinafter, for convenience of explanation, it is assumed that the learning image information includes moving image data. Further, it is assumed that the target for which the search condition is satisfied is not the moving image data but the frame (image data) included in the moving image data.

With reference to the display image information illustrated in FIG. 8, the display image information will be given a video data ID that can identify the video data and a label indicating whether or not the target search condition is satisfied, but still It is associated with an attribute that indicates that it has not been granted. Further, as the display image information, the moving image data ID may be associated with the moving image data represented by the moving image data ID.

For example, in the display image information illustrated in FIG. 8, the moving image data ID “V3002” and the attribute “determine whether or not the target search condition is satisfied” are associated with each other. This is because the video data ID "V3002" is represented by the video data ID "V3002" when a label indicating whether or not the target search condition is satisfied is input as described later with reference to FIG. Indicates that the image information related to "V3002" includes the image data displayed by the control unit 201.

The displayed image information may be further associated with different items, and is not limited to the above-mentioned example. Further, the display image information may include the label ID in the label information illustrated in FIG. 11, and in this case, the label related to the label ID stored in the label information storage unit 202 is input. Hereinafter, for convenience of explanation, it is assumed that the displayed image information includes moving image data.

With reference to the unprocessed image information illustrated in FIG. 9, the unprocessed image information is associated with a moving image data ID that can identify the image data and an attribute indicating that no label is attached. Further, as the display image information, the moving image data ID may be associated with the moving image data represented by the moving image data ID.

For example, in the unprocessed image information illustrated in FIG. 9, the moving image data ID “V4002” and the attribute “unlabeled image data” are associated with each other. This represents a state in which the label is not estimated (predicted) with respect to the moving image data represented by the moving image data ID “V4002”, as will be described later with reference to FIG.

The unprocessed image information may be further associated with different items, and is not limited to the above-mentioned example. The unprocessed image information may be further associated with different items and is not limited to the examples described above. Further, the unprocessed image information may include the label ID in the label information shown in FIG. 11, and in this case, the label related to the label ID stored in the label information storage unit 202 is input. Hereinafter, for convenience of explanation, it is assumed that the unprocessed image information includes moving image data.

Next, the target search condition will be described with reference to FIG. FIG. 6 is a diagram conceptually showing the inclusion relationship regarding the data for which the target search condition is satisfied.

For example, suppose that the target search condition is "Are there people wearing sunglasses and red clothes?" In this case, the search conditions A to C shown below are search conditions calculated by relaxing the target search conditions. That is,
○ Search condition A: Is there a person?
○ Search condition B: Is there anyone wearing red clothes?
○ Search condition C: Is there anyone wearing sunglasses?

Referring to FIG. 6, the data for which the target search condition is satisfied is an intersection of the data for which the search condition B is satisfied and the data for which the search condition C is satisfied. Further, the data for which the search condition A is satisfied includes data for which the search condition B is satisfied and data for which the search condition C is satisfied. Therefore,
○ The data for which the search condition B is satisfied, the data for which the search condition C is satisfied, and the data for which the target search condition is satisfied are a subset of the data for which the search condition A is satisfied.
○ The data for which the target search condition is satisfied is a subset of the data for which the search condition B is satisfied and the data for which the search condition C is satisfied.
○ There is no inclusion relationship between the data for which the search condition B is satisfied and the data for which the search condition C is satisfied.

Hereinafter, the search conditions in which the target search conditions are relaxed are referred to as "first search condition" (for example, the above-mentioned search condition B) and "second search condition" (for example, search condition C). The search conditions for which the target search conditions are relaxed do not necessarily have to be two, and may be many search conditions. When the target search condition is "is there a person wearing a hat and a jumper?", The first search condition is, for example, "is there a person wearing a hat?", And the second search condition is, for example, "Is there anyone wearing a jumper?" In this case, the target search condition is a condition to which the logical product operation is applied to the first search condition and the second search condition.

Next, with reference to FIG. 2, the outline of the processing flow in the data estimation device 600 according to the first embodiment will be described. FIG. 2 is a flowchart showing an outline of a processing flow in the data estimation device 600 according to the first embodiment.

The process in the data estimation device 600 according to the first embodiment is roughly classified into the processes shown in steps S101 to S105.

○ Step S101: Input a label indicating whether or not the first search condition is satisfied and a label indicating whether or not the second search condition is satisfied.
-Step S102: The relationship between the label related to the first search condition and the frame (referred to as "first model" for convenience of explanation) and the label related to the second search condition and the frame. Create a model to represent (referred to as "second model" for convenience of explanation),
○ Step S103: Enter a label indicating whether or not the target search condition is satisfied.
○ Step S104: A label relating to the target search condition, a calculated value calculated according to the first model (described later with reference to FIG. 14), and a calculated value calculated according to the second model (described later with reference to FIG. 15). Create a model (referred to as "goodness-of-fit model" for convenience of explanation) that represents the relationship between
-Step S105: Based on the created goodness-of-fit model, the label is estimated (predicted, determined) with respect to the image data included in the unprocessed image information.

Hereinafter, each process shown in steps S101 to S105 will be described in detail with reference to FIGS. 3 to 5.

First, the user interface 2010 through the input / output by the control unit 201 will be described with reference to FIG. 10, and then the label information input by the control unit 201 will be described with reference to FIG. After that, the processing in steps S101 to S105 will be described with reference to FIGS. 3 to 5. FIG. 10 is a diagram conceptually showing an example of the configuration of the user interface 2010.

The user interface 2010 includes an area 2011 in which the moving image data can be displayed, an area 2012 in which the displayed moving image data can be operated, and an area 2013 representing a label assigned to the image data included in the moving image data. , The area 2014 in which the displayed moving image data can be operated is included. Further, the user interface 2010 includes an area 2015 in which each label can be input to the image data included in the moving image data.

The rectangle 2016 represents a position related to the frame shown in the area 2011 among all the frames (image data) included in the moving image data. The moving image data displayed in the area 2011 changes as the rectangle 2016 is slid laterally. In the area 2013, labels input via the area 2015 are displayed for the image data included in the moving image data shown in the area 2011 among all the frames. The horizontal direction in the area 2012 and the horizontal direction in the area 2013 correspond to each other, and each represents the position of the frame shown in the area 2011 among all the frames (image data) included in the moving image data. ing. For example, in the example shown in FIG. 10, depending on the position of the rectangle 2016 with respect to the area 2012, a person wearing a hat is displayed in the area 2011, and further, the label related to the frame is “C01” in the area 2013. It is displayed that there is.

In the area 2014, in order from the left, an area in which an operation of returning the moving image data, an operation of playing the moving image data, an operation of stopping the moving image data, and an operation of fast-forwarding the moving image data can be input is arranged.

Next, the label information input by the control unit 201 will be described with reference to FIG. FIG. 11 is a diagram conceptually showing an example of label information stored in the label information storage unit 202.

In the label information storage unit 202, for example, with respect to the image data included in the moving image data, label information in which a label indicating whether or not a certain search condition is satisfied and a label ID capable of identifying the label are associated with each other is provided. It is stored. In the label information storage unit 202, for example, regarding the first search condition "is there a person wearing a hat", the label "there is a person wearing a hat" indicating the case where the first search condition is satisfied is expressed. The label information associated with the label ID "C01" is stored. Further, in the label information storage unit 202, for example, regarding the first search condition "whether there is a person wearing a hat", the label "no person wearing a hat" indicating the case where the first search condition is not satisfied is stated. , Label information associated with the label ID "C02" representing the label is stored.

In the label information storage unit 202, the label "There is a person who wears a jumper" and the label "No person who wears a jumper" are further related to the second search condition "Is there a person who wears a jumper?" It is stored. In the label information shown in FIG. 11, the label "There is a person wearing a jumper" is associated with the label ID "C03". Further, the label "No one wears a jumper" is associated with the label ID "C04".

Next, the processes in steps S101 and S102 shown in FIG. 2 will be described with reference to FIG. FIG. 3 is a flowchart showing a detailed flow of processing in step S101 and step S102 shown in FIG.

In the label input unit 200, the control unit 201 reads one learning image information stored in the learning image information storage unit 101 (exemplified in FIG. 7), and reads the moving image data included in the read learning image information. Displayed on a display device (not shown) via a user interface 2010 as illustrated in FIG. 10 (step S201). Further, the control unit 201 identifies a label related to the read learning image information (exemplified in FIG. 7) based on the label information (exemplified in FIG. 11), and displays the identified label in, for example, the area 2015 in the user interface 2010. To do.

The control unit 201 inputs a label relating to the image data, for example, via the user interface 2010 (step S202). In this case, the control unit 201 inputs the label related to the first search condition and the label related to the second search condition, respectively.

The control unit 201 is an image in which the moving image data ID, the frame ID representing the frame in the moving image data represented by the moving image data ID, and the input label (that is, the label input for the frame) are associated with each other. Label information (exemplified in FIG. 12) is created (step S203). FIG. 12 is a diagram conceptually representing an example of image label information stored in the image label storage unit 203. The control unit 201 stores the created image label information in the image label storage unit 203. That is, the control unit 201 has a label related to the first search condition (for example, label “C01” and label “C02”) in which the target search condition is relaxed, and a label related to the second search condition (for example, label “C03” and label). Create image label information including "C04").

In the feature amount creation unit 310, the feature amount model creation unit 311 is assigned a label relating to the first search condition and the label based on the image label information (exemplified in FIG. 12) stored in the image label storage unit 203. The weight included in the model representing the relationship with the label is calculated (step S204). Similarly, the feature amount model creation unit 311 has a label related to the second search condition and a frame to which the label is assigned (image) based on the image label information (exemplified in FIG. 12) stored in the image label storage unit 203. The weight included in the model representing the relationship with the data) is calculated (step S205).

In step S204, the feature amount model creation unit 311 calculates a feature amount (for example, a HOG feature amount) representing the feature of the frame according to a predetermined procedure, and the calculated feature amount and a label relating to the first search condition are used. The weights constituting the model representing the relationship between the above are calculated by a machine learning method such as the CNN method (step S204). The HOG feature amount represents an abbreviation for Histograms_of_Oriented_Gradients. CNN represents an abbreviation for Convolutional_Neural_Newwork. The feature amount model creating unit 311 calculates the weight included in the model representing the relationship between the calculated feature amount and the label related to the second search condition by the same method (step S205). That is, the feature amount model creation unit 311 creates a first model in which the label related to the first search condition can be estimated and a second model in which the label related to the second search condition can be estimated.

Next, the feature quantity model creation unit 311 creates model information in which the model ID representing the created model and the label estimated according to the model are associated with each other (step S206), and the created model information (FIG. 13). (Example) is stored in the model information storage unit 312. FIG. 13 is a diagram conceptually representing an example of model information. The model information may also be information that represents the model or may be associated with the model.

With reference to the model information illustrated in FIG. 13, the model ID “M101”, the model “there is a person wearing a hat”, and the label IDs “C01, C02” are associated with each other. This refers to the model represented by the model ID "M101" with respect to the label relating to the first search condition "whether there is a person wearing a hat" (the label represented by the label ID "C01" and the label represented by the label ID "C02"). Indicates that the feature quantity model creation unit 311 has created the label. With reference to the model information illustrated in FIG. 13, the model ID “M102”, the model “someone wearing a jumper”, and the label IDs “C03, C04” are associated with each other. This refers to the model represented by the model ID "M102" with respect to the label relating to the second search condition "whether there is a person wearing a jumper" (the label represented by the label ID "C03" and the label represented by the label ID "C04"). Indicates that the feature quantity model creation unit 311 has created the label.

Next, an example of the model created by CNN will be described with reference to FIG. The model includes weights that are the basis for calculating the features created by the processes in step S306 shown in FIG. 3 and step S404 shown in FIG. FIG. 16 is a diagram conceptually showing an example of the configuration of CNN. CNN is an example of a predetermined first processing procedure and a predetermined second processing procedure described later.

The CNN has an input layer 71, an intermediate layer 72, and an output layer 73. The input layer 71, the intermediate layer 72, and the output layer 73 each include a plurality of nodes 74 representing a certain process. The node 74 in the input layer 71 represents, for example, a process of inputting a feature amount such as the above-mentioned HOG feature amount. The node 76 in the intermediate layer 72 and the output layer 73 represents a process of executing a certain operation on the input value. Further, the edge 75 represents a process of multiplying the weight in the direction of the arrow. The weight for each edge is obtained from the relationship between the feature amount input to the input layer 71 and the label by repeatedly executing the convolution process, the polling process, and the like. The CNN may include more nodes. That is, CNN is not limited to the example shown in FIG.

The predetermined first processing procedure and the predetermined second processing procedure do not necessarily have to be one method, and are not limited to the above-mentioned examples.

Next, the processes in steps S103 and S104 shown in FIG. 2 will be described with reference to FIG. FIG. 4 is a flowchart showing a detailed flow of processing in step S103 and step S104 shown in FIG.

The control unit 201 reads the display image information (exemplified in FIG. 8) to which the label related to the target search condition is given from the display image information storage unit 102, and obtains moving image data related to the read display image information, for example, FIG. Display on a display device (not shown) via the user interface 2010 exemplified in (step S301). The control unit 201 inputs, for example, a label input via the user interface 2010 illustrated in FIG. 10 (that is, a label indicating whether or not the target search condition is satisfied) (step S302).

The frame creation unit 411 reads the moving image data in the display image information (exemplified in FIG. 8) stored in the display image information storage unit 102, and creates a frame related to the read moving image data (step S303). For example, the frame creation unit 411 creates a frame (image data) for each moving image data whose moving image data ID is in the V3000 series. The frame creation unit 411 calculates a frame ID that can identify the frame with respect to the moving image data ID, creates frame information in which the created frame is associated with the calculated frame ID, and uses the created frame information as frame information. It is stored in the storage unit 412. In this case, the frame creation unit 411 selects a label for the created frame based on the input label.

The control unit 201 is an image in which the moving image data ID, the frame ID representing the frame in the moving image data represented by the moving image data ID, and the input label (that is, the label input for the frame) are associated with each other. Label information (exemplified in FIG. 17) is created (step S304), and the created image label information is stored in the image label storage unit 203. FIG. 17 is a diagram conceptually representing an example of image label information stored in the image label storage unit 203.

The feature amount calculation unit 313 can estimate a model related to the first search condition related to the target search condition and a model capable of estimating the label related to the second search condition related to the target search condition. , Is specified based on the model information (exemplified in FIG. 13) stored in the model information storage unit 312 (step S305). For example, since the label IDs representing the labels related to the first search condition are the label ID “C01” and the label ID “C02”, the feature amount calculation unit 313 uses the model information stored in the model information storage unit 312. Based on (exemplified in FIG. 13), the model ID “M101” associated with the label ID is specified. Similarly, since the label IDs representing the labels related to the second search condition are the label ID “C03” and the label ID “C04”, the feature amount calculation unit 313 is a model stored in the model information storage unit 312. Based on the information (exemplified in FIG. 13), the model ID "M102" associated with the label ID is specified.

Next, the feature amount calculation unit 313 applies the specified model to the frame stored in the frame information storage unit 412. By this processing, for example, at each node 74 in the intermediate layer 72 illustrated in FIG. 16, the sum of the values in which the data input to the node is weighted is calculated.

For example, the feature amount calculation unit 313 identifies the model ID “M101” representing a model in which the label related to the first search condition can be estimated based on the model information exemplified in FIG. Further, the feature amount calculation unit 313 generates a frame ID “1” representing a frame included in the moving image data represented by the moving image data ID “V3001” in which the label is input, based on the image label information illustrated in FIG. Identify. Next, the feature amount calculation unit 313 applies the model represented by the model ID “M101” to the frame represented by the frame ID “1”. As a result, each node 74 in the intermediate layer 72 illustrated in FIG. 16 executes a process for the data input to the node, and as a result of the process, a calculated value is calculated. For example, the goodness-of-fit learning information creation unit 321 applies the model represented by the model ID “M101” to the frame ID “1” included in the moving image data represented by the moving image data ID “V3001”. By this processing, the goodness-of-fit learning information creation unit 321 is, for example, calculated values “2, 10, 182, 128, ...” Including the values calculated at the node 74 in the intermediate layer 72 illustrated in FIG. (Example in FIG. 14) is calculated. Similarly, the goodness-of-fit learning information creation unit 321 applies the model represented by the model ID “M102” to the frame ID “1” included in the moving image data represented by the moving image data ID “V3001”. By this process, the goodness-of-fit learning information creation unit 321 is, for example, calculated values “10, 10, 175, 131, ...” Including the values calculated at the node 74 in the intermediate layer 72 illustrated in FIG. (Example in FIG. 15) is calculated. 14 and 15 are diagrams conceptually representing the feature information stored in the feature information storage unit 314. The conformity learning information creation unit 321 creates feature information in which the calculated calculated value, the frame ID related to the calculated value, the moving image data ID, and the model ID are associated with each other, and stores the created feature information in the feature information storage. It is stored in the unit 314 (exemplified in FIGS. 14 and 15).

The goodness-of-fit learning information creation unit 321 sets the feature vector in accordance with a predetermined procedure with respect to a certain moving image data ID and a calculated value associated with a certain frame ID among the feature information stored in the feature information storage unit 314. Create (step S306). Next, the conformity learning information creation unit 321 uses the image label information (exemplified in FIG. 17) stored in the image label storage unit 203 to display the moving image data ID and the label ID associated with the frame ID. Is specified (step S307). The conformity learning information creation unit 321 has correct answer information (for example, "1" indicating that the moving image data ID, the frame ID, the created feature vector, the specified label ID, and the label representing the label ID are correct. The conformity learning information (exemplified in FIG. 18) associated with) is created (step S308), and the created conformity learning information is stored in the conformity learning information storage unit 322. Similarly, the conformity learning information creation unit 321 has a label ID relating to the target search condition and representing a label different from the specified label, the moving image data ID, the frame ID, and the created feature vector. , Creates conformity learning information (exemplified in FIG. 18) associated with correct answer information (for example, “0”) indicating that the label representing the label ID is incorrect (step S308). The goodness-of-fit learning information creation unit 321 stores the created goodness-of-fit learning information in the goodness-of-fit learning information storage unit 322. FIG. 18 is a diagram conceptually representing an example of the goodness-of-fit learning information stored in the goodness-of-fit learning information storage unit 322.

The goodness-of-fit learning information creation unit 321 is, for example, a calculated value associated with the model ID “M101” based on the feature information (exemplified in FIG. 14) with respect to the frame ID “1” in the image information related to the video data ID “V3001”. Identify "2, 10, 183, 128, ...". The model represented by the model ID "101" is, for example, a model capable of estimating the first search condition "whether there is a person wearing a hat". Similarly, the goodness-of-fit learning information creation unit 321 has, for example, in the image information related to the moving image data ID “V3001”, regarding the frame represented by the frame ID “1”, based on the feature information (exemplified in FIG. 15), the model ID “M102”. The calculated values "10, 10, 175, 131, ..." Associated with "" are calculated.

Further, the model represented by the model ID "102" is, for example, a model capable of estimating the second search condition "whether there is a person wearing a jumper". The goodness-of-fit learning information creation unit 321 creates a feature vector "2, 10, 183, 128, ..., 10, 10, 175, 131, ..." Containing the two read calculated values. As illustrated in FIG. 18, the goodness-of-fit learning information creation unit 321 associates the created feature vector with the moving image data ID “V3001”, the frame ID “1”, and the label ID “C05”. The goodness-of-fit learning information is created (step S308).

The order in which the values included in the calculated values are arranged with respect to the predetermined procedure may be any predetermined order. For example, the model ID "M102" is next to the calculated values calculated for the model represented by the model ID "M101". The calculated values calculated for the model represented by "" may be arranged in an order. Alternatively, the order may be, for example, an order in which the calculated values calculated for the model represented by the model ID "M102" are arranged next to the calculated values calculated for the model represented by the model ID "M101". Good. Further, the predetermined procedure does not necessarily include all the values processed by all the nodes 74 included in the intermediate layer 72 illustrated in FIG. 16, even if it is a part of the values. Good. That is, the predetermined procedure is not limited to the above-mentioned example.

The goodness-of-fit learning information creation unit 321 identifies the moving image data ID “V3001” and the label ID “C05” associated with the frame ID “1” from the image label storage unit 203 (exemplified in FIG. 17). In this case, the goodness-of-fit learning information creation unit 321 creates correct answer information "1" indicating that the answer is correct with respect to the label ID "C05". On the other hand, the goodness-of-fit learning information creation unit 321 creates correct answer information "0" indicating that the answer is not correct with respect to the label ID "C06" paired with the label represented by the label ID "C05".

The goodness-of-fit learning information creation unit 321 executes the above-described processing for each image data.

Next, the goodness-of-fit model creation unit 323 is a model that represents the relationship between the feature vector and the label related to the target search condition based on the information stored in the goodness-of-fit learning information storage unit 322, for example, according to the SSI method. Is created (step S309). In addition, SSI represents an abbreviation for Supervised_Semantic_Indexing. SSI is an example of a predetermined third processing procedure described later. The predetermined third processing procedure does not necessarily have to be SSI, and may be a predetermined first processing procedure or a processing procedure similar to the predetermined second processing procedure.

Next, the goodness-of-fit learning information creation unit 321 creates goodness-of-fit model information (exemplified in FIG. 19) in which the model ID representing the created model and the label ID are associated with each other (step S310), and the created goodness-of-fit degree. The model information is stored in the goodness-of-fit model information storage unit 324. The goodness-of-fit model information may be further associated with the label or the model created. FIG. 19 is a diagram conceptually representing an example of goodness-of-fit model information stored in the goodness-of-fit model information storage unit 324.

For example, the goodness-of-fit learning information creation unit 321 determines that the first search condition "is there a person wearing a hat" regarding the target search condition and the second search condition "is there a person wearing a jumper" regarding the target search condition? Create a model that can estimate the labels that hold both.

Next, with reference to FIG. 5, the processing in step S104 and step S105 shown in FIG. 2 will be described in detail. FIG. 5 is a flowchart showing a detailed flow of processing in step S104 and step S105 shown in FIG.

The frame creation unit 411 reads the unprocessed image information (exemplified in FIG. 9) stored in the unprocessed image information storage unit 103, and creates a frame for the moving image data included in the read unprocessed image information (exemplified in FIG. 9). Step S401), the created frame is stored in the frame information storage unit 412. For example, the frame creation unit 411 creates a frame for all the unprocessed image information stored in the unprocessed image information storage unit 103 (exemplified in FIG. 9), and stores the created frame in the frame information storage unit 412. To do.

The feature amount calculation unit 313 estimates a model capable of estimating a label related to the first search condition and a label related to the second search condition based on the model information stored in the model information storage unit 312 (exemplified in FIG. 13). Identify a possible model (step S402). Next, the feature amount calculation unit 313 applies the specified model to each frame stored in the frame information storage unit 412 (step S403). By this process, the calculated value is calculated in the intermediate layer 72 as illustrated in FIG. For example, the calculated values "93, 72, 112, 165, ..." Are calculated according to the model represented by the model ID "M101" (that is, the model in which the label related to the first search condition can be estimated). For example, the calculated values "32, 43, 112, 87, ..." Are calculated according to the model represented by the model ID "M102" (that is, the model in which the label related to the second search condition can be estimated).

The feature amount creation unit 421 calculates the feature vector related to the calculated value by executing the process according to a predetermined procedure (step S404), and the calculated feature vector and the moving image related to the frame which is the basis for calculating the feature vector. The goodness-of-fit determination information (exemplified in FIG. 20) in which the data ID and the frame ID are associated is created (step S405). FIG. 20 is a diagram conceptually representing an example of goodness-of-fit determination information stored in the goodness-of-fit determination information storage unit 422.

The feature amount creation unit 421 stores the created goodness-of-fit determination information in the feature information storage unit 314. For example, the feature amount calculation unit 313 refers to the feature vector "(93, 72, 112, 165, ..., 32, 43, 112" with respect to the frame "1" included in the image information regarding the moving image data ID "V4001". , 87, ...) ”Is calculated.

The goodness-of-fit calculation unit 325 applies a model capable of estimating a label related to the target search condition to each feature vector stored in the goodness-of-fit determination information storage unit 422, thereby applying the label to the label with respect to the feature vector. The label goodness of fit, which indicates the degree of conformity (degree), is calculated (step S406). The goodness-of-fit calculation unit 325 has a goodness-of-fit information (FIG. 21) in which the calculated label goodness of fit, the moving image data ID related to the label goodness of fit, the frame ID, the model ID, and the label ID related to the label goodness of fit are associated. (Example) is created, and the created goodness-of-fit information is stored in the goodness-of-fit information storage unit 326. FIG. 21 is a diagram conceptually representing an example of goodness-of-fit information stored in the goodness-of-fit information storage unit 326. Specifically, the goodness-of-fit calculation unit 325 applies the model represented by the model ID “M201” to the input feature vector, so that the label represented by the label ID “C05” and the label ID “C06” are generated. Calculate the label conformity that indicates the degree of conformity with the label. The label goodness of fit represents, for example, the probability of being a label and is any value from 0 to 1.

The label determination unit 501 selects a label related to a certain frame based on the goodness of fit information (exemplified in FIG. 21) stored in the goodness of fit information storage unit 326 (step S407). The label determination unit 501 creates expected label information (exemplified in FIG. 22) in which the selected label, the frame ID representing the certain frame, and the moving image data ID representing the moving image data including the frame are associated (exemplified in FIG. 22). S408), the created expected label information is stored in the predicted label information storage unit 502. FIG. 22 is a diagram conceptually representing an example of the predicted label information stored in the predicted label information storage unit 502.

For example, the label determination unit 501 compares the label conformity calculated for the label ID “C05” with the label conformity calculated for the label ID “C06” with respect to the frame ID representing a certain frame, and performs a large label conformity. A label of goodness of fit is given to the frame. In the case of the goodness-of-fit information illustrated in FIG. 21, among the data associated with the moving image data ID “V4001” and the frame ID “1”, the label goodness of fit associated with the label ID “C05” is 0. It is 85, and the label goodness of fit associated with the label ID “C06” is 0.15. Therefore, since the label conformity with respect to the label ID "C05" is a value larger than the label conformity with respect to the label ID "C06", the label determination unit 501 has the frame ID "1" regarding the video data represented by the video data ID "V4001". The label represented by the label ID “C05” is assigned to the frame represented by “”. Similarly, the label determination unit 501 also executes a process of assigning labels to other frames included in the goodness-of-fit information.

Next, the effect of the data estimation device 600 according to the first embodiment will be described.

According to the data estimation device 600 according to the first embodiment, the target data can be correctly determined even if the target search condition is strict.

The reason for this is that the data estimation device 600 executes the process according to the tendency found by the inventor of the present application. More specifically, the inventor of the present application has found that the second viewpoint tends to have higher classification accuracy than the first viewpoint shown below.

○ First viewpoint: The label is determined based on the model for determining whether or not the first search condition and the second search condition (that is, the target search condition) are satisfied.
○ Second viewpoint: A first model that can estimate whether or not the first search condition is satisfied and a second model that can estimate whether or not the second search condition is satisfied are created. Next, a model capable of estimating whether or not the first search condition and the second search condition are satisfied is created based on the value calculated based on the first model and the value calculated based on the second model. To do. Next, the label is determined based on the created model.

Since the data estimation device 600 performs processing based on the model created according to the second viewpoint, the data estimation device 600 according to the present embodiment correctly determines the target data even if the target search condition is strict. can do.

That is, according to the data estimation device 600 according to the first embodiment, the target data can be correctly determined even if the target search condition is strict.

According to the data estimation device 600 according to the first embodiment, there is an effect that the user can easily input the label even if the target search condition is strict. The reason for this is to enter the labels related to the first search condition and the second search condition, which are the conditions for which the target search condition is relaxed, and enter the label related to the target search condition according to the model created based on the entered label. Because it does. That is, the user inputs the label relating to the first search condition and the second search condition, which are relatively easy to detect, and then inputs the label relating to the target search condition step by step. As a result, the user-labeling load is reduced.

<Second embodiment>
Next, a second embodiment of the present invention will be described.

The configuration of the data estimation device 801 according to the second embodiment of the present invention will be described in detail with reference to FIG. 23. FIG. 23 is a block diagram showing a configuration of the data estimation device 801 according to the second embodiment of the present invention.

The data estimation device 801 according to the second embodiment includes a first calculation unit 802, a second calculation unit 803, and a third calculation unit 804.

The data estimation device 801 is connected to a model storage unit 805 in which the first model to the third model as shown below are stored.

○ First model: A model that can estimate whether or not the first search condition is satisfied by calculating the first target data according to a predetermined first processing procedure.
○ Second model: A model that can estimate whether or not the second search condition is satisfied by performing calculations on the second target data according to a predetermined second processing procedure.
○ Third model: Regarding the calculation result calculated in the first model applied to the third target data and the calculation result calculated in the second model applied to the third target data. A model capable of estimating whether or not the first search condition and the second search condition are satisfied with respect to the third target data by performing an operation according to a predetermined third processing procedure.

The target search condition shown in the first embodiment is, for example, a logical product of the first search condition and the second search condition. The third model represents, for example, a model in which the label relating to the target search condition shown in the first embodiment can be estimated. The predetermined first processing procedure and the predetermined second processing procedure are, for example, the CNN illustrated in FIG. The predetermined third processing procedure is, for example, the SSI method shown in the first embodiment.

Next, the processing in the data estimation device 801 according to the second embodiment will be described.

The first calculation unit 802 reads the first model from the model storage unit 805 and applies the read first model to a certain data. That is, the first calculation unit 802 calculates the first calculation result calculated by the processing by executing the processing on the certain data according to the predetermined first processing procedure.

The second calculation unit 803 reads the second model from the model storage unit 805, and applies the read second model to a certain data. That is, the second calculation unit 803 calculates the second calculation result calculated by the processing by executing the processing on the certain data according to the predetermined second processing procedure.

Next, the third calculation unit 804 reads the third model from the model storage unit 805. The third calculation unit 804 has read about the first calculation result calculated by the first calculation unit 802 for the certain data and the second calculation result calculated by the second calculation unit 803 for the certain data. The third model is applied. That is, the third calculation unit 804 executes the processing of the first calculation result related to the certain data and the second calculation result related to the certain data according to a predetermined third processing procedure, thereby performing the first search condition. , And the degree to which the second search condition is satisfied is calculated.

The first calculation unit 802 can be realized by using the functions of the feature amount creation unit 310 and the like shown in FIG. The second calculation unit 803 can be realized by using the functions of the feature amount creation unit 310 and the like shown in FIG. The third calculation unit 804 can be realized by using the functions of the goodness-of-fit feature amount creating unit 420, the goodness-of-fit creating unit 320, and the like shown in FIG. That is, the data estimation device 801 can be realized by using the function of the data estimation device 600 shown in FIG.

Next, the effect of the data estimation device 801 according to the second embodiment will be described.

According to the data estimation device 801 according to the second embodiment, the target data can be correctly determined even if the target search condition is strict.

The reason for this is that the data estimation device 801 executes the process according to the tendency found by the inventor of the present application. Explaining the reason in detail, the inventor of the present application has found a tendency that the classification accuracy is often higher in the second viewpoint than in the first viewpoint described above. Since the data estimation device 801 performs processing based on the model created according to the second viewpoint, and according to the data estimation device 801 according to the present embodiment, the target data can be obtained even if the target search condition is strict. It can be judged correctly.

(Hardware configuration example)
An example of a configuration of hardware resources in which the data estimation device according to each embodiment of the present invention described above is realized by using one calculation processing device (information processing device, computer) will be described. However, such a data estimation device may be physically or functionally realized by using at least two calculation processing devices. Further, the data estimation device may be realized as a dedicated device.

FIG. 24 is a block diagram schematically showing a hardware configuration example of a calculation processing device capable of realizing the data estimation device according to the first embodiment or the second embodiment. The calculation processing device 20 includes a central processing unit (Central_ProceSSIng_Unit, hereinafter referred to as “CPU”) 21, a memory 22, a disk 23, a non-volatile recording medium 24, a communication interface (hereinafter referred to as “communication IF”) 27, and It has a display 28. The calculation processing device 20 may be connectable to the input device 25 and the output device 26. The calculation processing device 20 can transmit / receive information to / from other calculation processing devices and the communication device via the communication IF 27.

The non-volatile recording medium 24 is, for example, a computer-readable compact disc (Compact_Disc) or a digital versatile disc (Digital_Versail_Disc). Further, the non-volatile recording medium 24 may be a universal serial bus memory (USB memory), a solid state drive (Solid_State_Drive), or the like. The non-volatile recording medium 24 holds the program and makes it portable without supplying power. The non-volatile recording medium 24 is not limited to the above-mentioned medium. Further, instead of the non-volatile recording medium 24, the program may be carried via the communication IF27 and the communication network.

That is, the CPU 21 copies the software program (computer program: hereinafter, simply referred to as "program") stored in the disk 23 to the memory 22 when executing the software program, and executes the arithmetic processing. The CPU 21 reads the data required for program execution from the memory 22. When display is required, the CPU 21 displays the output result on the display 28, for example, the user interface illustrated in FIG. 10. When output to the outside is required, the CPU 21 outputs the output result to the output device 26. When the program is input from the outside, the CPU 21 reads the program from the input device 25. The CPU 21 interprets and executes a data estimation program (FIGS. 2 to 5) in the memory 22 corresponding to the function (process) represented by each part shown in FIG. 1 or FIG. 23 described above. The CPU 21 sequentially executes the processes described in the above-described embodiments of the present invention.

That is, in such a case, it can be considered that the present invention can also be achieved by the data estimation program. Further, it can be considered that the present invention can be achieved by using a non-volatile recording medium in which the data estimation program is recorded and which can be read by a computer.

The present invention has been described above using the above-described embodiment as a model example. However, the present invention is not limited to the above-described embodiments. That is, the present invention can apply various aspects that can be understood by those skilled in the art within the scope of the present invention.

100 Image information storage unit 101 Learning image information storage unit 102 Display image information storage unit 103 Unprocessed image information storage unit 200 Label input unit 201 Control unit 202 Label information storage unit 203 Image label storage unit 310 Feature quantity creation unit 311 Feature quantity model Creation unit 312 Model information storage unit 313 Feature amount calculation unit 314 Feature information storage unit 320 Conformity level creation unit 321 Conformity learning information creation unit 322 Conformity learning information storage unit 323 Conformity model creation unit 324 Conformity model information storage unit 325 Conformity calculation unit 326 Conformity information storage unit 410 Frame information creation unit 411 Frame creation unit 412 Frame information storage unit 420 Conformity feature amount creation unit 421 Feature amount creation unit 422 Conformity judgment information storage unit 500 Label selection unit 501 Label judgment Part 502 Prediction Label Information Storage 600 Data Estimator 2010 User Interface 2011 Area 2012 Area 2013 Area 2014 Area 2015 Area 2016 Rectangular 71 Input Layer 72 Intermediate Layer 73 Output Layer 74 Nodes 75 Edge 76 Nodes 801 Data Estimator 802 1st Calculation Unit 803 Second calculation unit 804 Third calculation unit 805 Model storage unit 20 Calculation processing device 21 CPU
22 Memory 23 Disk 24 Non-volatile recording medium 25 Input device 26 Output device 27 Communication IF
28 display

Claims (10)

By applying the first model to a certain data according to the first model that can estimate whether or not the first search condition is satisfied by calculating the first target data according to a predetermined first processing procedure. A first calculation means for calculating the first calculation result calculated by the predetermined first processing procedure, and
According to a second model in which it is possible to estimate whether or not a second search condition different from the first search condition is satisfied by performing an operation on the second target data according to a predetermined second processing procedure, for the certain data. By applying the second model, a second calculation means for calculating the second calculation result calculated in the predetermined second processing procedure, and
The calculation result calculated in the first model when the first model is applied to the third target data, and calculated in the second model when the second model is applied to the third target data. By performing a calculation on the calculation result according to a predetermined third processing procedure, the first search condition for the first calculation result and the second calculation result for the third target data. And, by applying the third model that can estimate whether or not both of the second search conditions are satisfied, whether the first search condition and the second search condition are satisfied with respect to the certain data. Equipped with a third calculation means for estimating whether or not
The predetermined first processing procedure and the predetermined second processing procedure are Convolutional Neural Networks.
The predetermined third processing procedure is a data estimation device that is Supervised Semantic Indexing . The third model capable of estimating the relationship between the first calculation result and the second calculation result and the third label indicating whether or not the first search condition and the second search condition are satisfied. The data estimation device according to claim 1, further comprising a goodness-of-fit model calculating means to be created. A claim further comprising a first feature quantity creating means for creating the first model capable of estimating the relationship between the first target data and the first label indicating whether or not the first search condition is satisfied. 2. The data estimation device according to 2. A claim further comprising a second feature quantity creating means for creating the second model capable of estimating the relationship between the second target data and the second label indicating whether or not the second search condition is satisfied. The data estimation device according to 3. A label input means for inputting the first label, the second label, and the third label is further provided.
The first feature quantity creating means creates the first model with respect to the input first label.
The second feature quantity creating means creates the second model with respect to the input second label.
The goodness-of-fit model calculating means refers to the first calculation result calculated for the first model created by the first feature quantity creating means for the third target data and the third target data. The data according to claim 4, wherein the third model representing the relationship between the second calculation result calculated for the second model created by the second feature amount creating means and the third label is created. Estimator. The first target data, the second target data, and the third target data are all image data.
The data estimation device according to claim 5, wherein the label input means includes a user interface including the first label, the second label, or an area in which the third label can be input. Further provided with a frame creation means for creating the image data representing the frame included in the moving image data,
The user interface is
A display area capable of displaying the moving image data and
Includes the first label, the second label, or an input area into which the third label is input.
The label input means displays the moving image data in the display area, and is input to the image data displayed in the display area when the moving image data is input to the input area. The data estimation device according to claim 6, wherein label information corresponding to the input area is input. By applying the first model to a certain data according to the first model that can estimate whether or not the first search condition is satisfied by calculating the first target data according to a predetermined first processing procedure. Calculate the first calculation result calculated by the predetermined first processing procedure,
According to a second model in which it is possible to estimate whether or not a second search condition different from the first search condition is satisfied by performing an operation on the second target data according to a predetermined second processing procedure, for the certain data. By applying the second model, the second calculation result calculated by the predetermined second processing procedure is calculated.
The calculation result calculated in the first model when the first model is applied to the third target data, and calculated in the second model when the second model is applied to the third target data. By performing a calculation on the calculation result according to a predetermined third processing procedure, the first search condition for the first calculation result and the second calculation result for the third target data. And, by applying the third model that can estimate whether or not the second search condition is satisfied, whether or not the first search condition and the second search condition are satisfied with respect to the certain data. Estimate and
The predetermined first processing procedure and the predetermined second processing procedure are Convolutional Neural Networks.
The predetermined third processing procedure is a data estimation method, which is Supervised Semantic Indexing . By applying the first model to a certain data according to the first model that can estimate whether or not the first search condition is satisfied by calculating the first target data according to a predetermined first processing procedure. The first calculation function for calculating the first calculation result calculated by the predetermined first processing procedure, and
According to a second model in which it is possible to estimate whether or not a second search condition different from the first search condition is satisfied by performing an operation on the second target data according to a predetermined second processing procedure, for the certain data. By applying the second model, a second calculation function for calculating the second calculation result calculated in the predetermined second processing procedure, and
The calculation result calculated in the first model when the first model is applied to the third target data, and calculated in the second model when the second model is applied to the third target data. By performing a calculation on the calculation result according to a predetermined third processing procedure, the first search condition for the first calculation result and the second calculation result for the third target data. And, by applying the third model that can estimate whether or not the second search condition is satisfied, whether or not the first search condition and the second search condition are satisfied with respect to the certain data. The third calculation function to estimate the data is realized in the computer ,
The predetermined first processing procedure and the predetermined second processing procedure are Convolutional Neural Networks.
The predetermined third processing procedure is a data estimation program that is Supervised Semantic Indexing .
The third model capable of estimating the relationship between the first calculation result and the second calculation result and the third label indicating whether or not the first search condition and the second search condition are satisfied. The data estimation program according to claim 9, further comprising a goodness-of-fit model calculation function to be created.

Images