JP2019139479A - Program, device, and method for estimating context using a plurality of recognition engines - Google Patents

Abstract

To provide a program, a device, and a method, capable of enhancing context recognition accuracy by automatically selecting one or more optimal recognition engines in accordance with input data.SOLUTION: A recognition device 1 includes: a selection engine 11 for selecting recognition engines corresponding to input data to be estimated using a learning model which has learned by training data in which input data is associated with identifiers of recognition engines, and outputting the input data into the selected recognition engines; and a recognition score determination section 13 for determining, for the input data, recognition engines whose recognition score calculated by recognition engines 121, 122 is equal to or greater than a recognition threshold, and allowing the selected engines to get feedback of the identifiers of the recognition engines. The selection engine 11 learns the learning model again by training data in which the input data is associated with the identifiers of the recognition engines received the feedback.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a technique for estimating a context using a plurality of recognition engines.

In recent years, recognition accuracy in object recognition and human action recognition has been dramatically improved by using deep learning.
For example, there is a technique for inputting a specific data set and comparing machine learning algorithm candidates (see, for example, Patent Document 1). According to this technique, it is possible to automatically compare the evaluations of the machine learning models by counting the performance results for each machine learning model.

As a recognition engine for video data, for example, there is an object recognition technique for detecting an object reflected in an RGB image (see, for example, Non-Patent Document 1).
There is also a moving object recognition technique for detecting the movement of an object from a moving feature amount (optical flow) (see, for example, Non-Patent Document 2). For example, according to Two-stream ConvNets, the appearance characteristics of objects and backgrounds in an image and the horizontal direction of an optical flow using spatial-direction CNN (Spatial stream ConvNet) and time-series CNN (Temporal stream ConvNet) Extract both the component and motion features in the sequence of vertical components. By integrating both features, the behavior is recognized with high accuracy.
Further, there is a technique for recognizing a person's action from a three-dimensional video (see, for example, Non-Patent Document 3).
Further, there is a technique for recognizing a person's action by extracting skeleton information of a human joint and its associated part (see, for example, Non-Patent Document 4).

  As another application field, there is a technology for executing reinforcement learning of a hierarchical learning model according to the autonomous operation of a robot (for example, see Non-Patent Document 5). There is also a technique for selecting an expert network, which is a machine learning network used by a gate network, according to input data (see, for example, Non-Patent Document 6).

  FIG. 1 is a system configuration diagram having a recognition device.

  According to the system of FIG. 1, the recognition apparatus 1 functions as a server connected to the Internet. The recognition apparatus 1 has a recognition engine in which a learning model is built in advance using teacher data. When the recognition engine recognizes a person's action, the teacher data is obtained by associating in advance video data in which a person's action is reflected and an action target (context).

Each terminal 2 is equipped with a camera, and transmits video data obtained by photographing a human action to the recognition device 1. The terminal 2 is a smartphone or a mobile terminal possessed by each user, and is connected to an access network such as a mobile phone network or a wireless LAN.
Of course, the terminal 2 is not limited to a smartphone or the like, and may be, for example, a Web camera installed in a home. Alternatively, video data captured by a Web camera may be recorded on an SD card, and the recorded video data may be input to the recognition device 1.

Specifically, for example, a user photographs his / her behavior with his / her smartphone camera. The smartphone transmits the video data to the recognition device 1. The recognition apparatus 1 estimates a human action from the video data, and uses the estimation result in various applications.
Note that each function of the recognition device 1 may be incorporated in the terminal 2.

JP 2017-004509 A

Shaoqing Ren, Kaiming He, Ross Girshick, Jian Sun: Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks, [online], [Search January 24, 2018], Internet <URL: https: / /arxiv.org/pdf/1506.01497.pdf> Karen Simonyan and Andrew Zisserman, “Two-Stream Convolutional Networks for Action Recognition in Videos,” in NIPS 2014, [online], [searched January 24, 2018], Internet <URL: https://arxiv.org/ abs / 1406.2199.pdf> Hernandez Ruiz, Alejandro & Porzi, Lorenzo & Rota Bul ?, Samuel & Moreno-Noguer, Francesc: 3D CNNs on Distance Matrices for Human Action Recognition, [online], [Search January 24, 2018], Internet <URL: https://www.researchgate.net/publication/320543521_3D_CNNs_on_Distance_Matrices_for_Human_Action_Recognition> Zhe Cao, Tomas Simon, Shih-En Wei, Yaser Sheikh: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields., [Online], [Search January 28, 2018], Internet <https: // arxiv. org / pdf / 1611.08050.pdf> Yasutake Takahashi, Satoshi Asada, “Acquisition of Action by Hierarchical Construction of Multiple Learning Mechanisms”, [online], [Search January 24, 2018], Internet <URL: http: //www.er.ams. eng.osaka-u.ac.jp/Paper/1999/Takahashi99c.pdf> Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton and Jeff Dean: OUTRAGEOUSLY LARGE NEURAL NETWORKS: THE SPARSELY-GATED MIXTURE-OF-EXPERTS LAYER, [online], [January 24, 2018 Search ] Internet <URL: https: //openreview.net/pdf? Id = B1ckMDqlg> OpenPose, [online], [Search January 24, 2018], Internet <URL: https://github.com/CMU-Perceptual-Computing-Lab/openpose> “I tried OpenPose, which can detect bones from videos and photos”, [online], [Search January 24, 2018], Internet <URL: http://hackist.jp/?p=8285> “OpenPose has been upgraded and 3d pose estimation can be tried now,” [online], [searched January 24, 2018], Internet <URL: http://izm-11.hatenablog.com / entry / 2017/08/01/140945>

According to the techniques of Non-Patent Documents 1 to 4, it is necessary to determine in advance a learning model that will have the highest recognition accuracy. Therefore, depending on the input data, a learning model that is not optimal as a result may be selected.
According to the technique of Patent Document 1, in order to compare machine learning models, it is necessary to input input data to all machine learning models. The more machine learning models, the more server resources are required.
According to the technique of Non-Patent Document 5, reinforcement learning of a hierarchical learning model is executed, and context is not recognized using a plurality of recognition engines.
According to the technique of Non-Patent Document 6, when the learning of an expert who selects a network is insufficient, the score desired by the user may not be reached.

All of the above-described conventional techniques are based on the premise that a learning engine having an optimally constructed learning model is used, and an optimum learning model recognition engine is determined in advance according to input data. It is necessary to keep.
On the other hand, the inventors of the present invention can increase the recognition accuracy of the context (recognition result) by automatically selecting one or more optimal recognition engines according to the input data. I thought.

  Therefore, an object of the present invention is to provide a program, an apparatus, and a method that can improve context recognition accuracy by automatically selecting one or more optimal recognition engines according to input data. .

According to the present invention, in a recognition program for causing a computer to function to estimate a context from input data using a plurality of recognition engines,
A selection engine that selects a recognition engine for input data to be estimated using a learning model learned from teacher data in which input data is associated with an identifier of the recognition engine, and outputs the input data to the selected recognition engine When,
Determining a recognition engine for which the recognition score calculated by the recognition engine for the input data is equal to or greater than a recognition threshold, and causing the computer to function as a recognition score determination unit that feeds back the identifier of the recognition engine to the selection engine;
The selection engine causes the computer to function so as to re-learn the learning model based on the teacher data in which the input data is associated with the fed back identifier of the recognition engine.

According to another embodiment of the recognition program of the present invention,
The recognition engine is based on a classification that calculates a recognition score for each class.
The recognition engine also preferably causes the computer to function as a recognition score by calculating a statistical value of any one of a maximum value, a minimum value, an average value, and an addition value in a plurality of scores of a plurality of classes.

According to another embodiment of the recognition program of the present invention,
The recognition score determination means preferably causes the computer to function so as to feed back only the identifier of the recognition engine whose processing time required for the input data is equal to or less than a predetermined threshold time to the selection engine.

According to another embodiment of the recognition program of the present invention,
The selection engine is based on a class classification that calculates a selection score for each class,
The selection engine preferably causes the computer to function so as to output the input data to a recognition engine in which the selection score for the input data to be estimated is equal to or higher than the first selection threshold.

According to another embodiment of the recognition program of the present invention,
Preferably, the selection engine further causes the computer to function to output the input data to a recognition engine whose selection score is less than the first selection threshold and equal to or greater than the second selection threshold.

According to another embodiment of the recognition program of the present invention,
When the difference between the selection score of one recognition engine that is equal to or greater than the first selection threshold and the recognition score of the other recognition engine that is less than the first selection threshold is equal to or less than a predetermined difference, It is also preferable to further cause the computer to function so as to output the input data to the other recognition engine.

According to another embodiment of the recognition program of the present invention,
The input data is video data,
Multiple recognition engines are different from each other,
An object recognition engine based on RGB images;
Motion recognition engine based on optical flow and / or
It is also preferred to make the computer function so that it is one of the human joint recognition engines based on skeleton information.

According to the present invention, in a recognition apparatus that estimates a context from input data using a plurality of recognition engines,
A selection engine that selects a recognition engine for input data to be estimated using a learning model learned from teacher data in which input data is associated with an identifier of the recognition engine, and outputs the input data to the selected recognition engine When,
A recognition score determining means for determining a recognition engine for which the recognition score calculated by the recognition engine for the input data is equal to or greater than a recognition threshold, and feeding back the identifier of the recognition engine to the selection engine;
The selection engine is characterized in that the learning model is re-learned by using teacher data in which the input data is associated with the fed back identifier of the recognition engine.

According to the present invention, in a recognition method for an apparatus for estimating a context from input data using a plurality of recognition engines,
The device
First, a recognition engine for input data to be estimated is selected using a learning model learned from teacher data in which input data is associated with an identifier of a recognition engine, and the input data is output to the selected recognition engine. And the steps
A second step of determining a recognition engine having a recognition score calculated by the recognition engine for the input data equal to or greater than a recognition threshold;
A third step of re-learning the learning model is performed using the teacher data in which the input data is associated with the identifier of the recognition engine determined to be true in the second step.

  According to the program, apparatus, and method of the present invention, it is possible to improve context recognition accuracy by automatically selecting one or more optimal recognition engines according to input data.

It is a system block diagram which has a recognition apparatus. It is a functional block diagram of the recognition apparatus in this invention. It is a specific 1st processing flow in this invention. It is a concrete 2nd processing flow in this invention. It is a specific 3rd processing flow in this invention. It is a concrete 4th processing flow in the present invention. It is a specific 5th processing flow with respect to video data. It is a flowchart based on FIG.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 2 is a functional configuration diagram of the recognition device according to the present invention.

The recognition apparatus 1 estimates a context (for example, an object, a moving object, a human action, etc.) from input data using a plurality of recognition engines.
According to FIG. 2, the recognition apparatus 1 includes a selection engine 11, a plurality of recognition engines 12 (a first recognition engine 121 and a second recognition engine 122), and a recognition score determination unit 13. These functional components can be realized by executing a program that causes a computer installed in the apparatus to function. Further, the processing flow of these functional components can be understood as a device recognition method for input data.

[Selection engine 11]
The selection engine 11 is based on a class classification, and is a machine learning engine that gives a class (identifier of the recognition engine 12) to input data to be estimated. The selection engine 11 builds a learning model in advance based on teacher data in which input data is associated with an identifier of a recognition engine.
<Teacher data>
Input Data <-> Recognition Engine Identifier The selection engine 11 specifically calculates a score (recognition accuracy) for each recognition engine (class). In general, one recognition engine having the highest score is selected as the estimation result. However, according to the present invention, the number of recognition engines is not limited to one and may be plural. An embodiment of the selection method of the selection engine 11 will be described later with reference to FIGS.
Then, the selection engine 11 selects a recognition engine 12 for the input data to be estimated using the learning model, and outputs the input data to the selected recognition engine 12.

  Note that the selection engine 11 of the present invention does not need to build a complete learning model in advance, and re-learns by feedback from a recognition score determination unit 13 described later. “Relearning” means that the learning model is further trained using the input data and the fed back recognition engine identifier as teacher data.

[Recognition engine 12]
The recognition engine 12 selected by the selection engine 11 inputs input data from the selection engine 11. The recognition engine 12 is also based on the class classification, and calculates a recognition score (recognition accuracy) for each class (estimable context). Generally, one context having the highest recognition score is output as an estimation result.

  According to the present invention, it has a plurality of different types of recognition engines 12. For example, different types of recognition engines are combined, such as an engine that mainly recognizes objects, an engine that mainly recognizes rough actions, and an engine that mainly recognizes fine actions. Each recognition engine builds a learning model in advance based on different teacher data depending on its type.

According to FIG. 2, it has two recognition engines (the 1st recognition engine 121 and the 2nd recognition engine 122). The recognition score calculated by the recognition engine 12 may be a “statistical value” of any one of the highest value, the lowest value, the average value, and the added value of the plurality of recognition scores in the plurality of contexts.
Each recognition engine 12 then outputs a recognition score calculated for each context to the recognition score determination unit 13.

[Recognition score determination unit 13]
The recognition score determination unit 13 determines whether or not the recognition score calculated for each context of each recognition engine 12 with respect to the input data is greater than or equal to the “recognition threshold”.
Here, when it is determined to be true (recognition score ≧ recognition threshold), the identifier of the recognition engine 12 is fed back to the selection engine 11. On the other hand, the selection engine 11 re-learns the learning model as teacher data in which the input data is associated with the identifier of the recognition engine.
Further, among the recognition scores calculated by each recognition engine 12, a context that is equal to or higher than the recognition threshold is output to the application as an estimation result.
The recognition threshold can be arbitrarily set by the operator.

  As a result, the selection engine 11 re-learns the learning model based on the feedback from the recognition score determination unit 13, and then selects the most suitable recognition engine 12 for the input data to be estimated thereafter. It becomes like this.

  FIG. 3 is a specific first processing flow in the present invention.

According to FIG. 3, it is assumed that the selection engine 11 calculates a selection score for each recognition engine for input data to be estimated as follows.
[Recognition engine ID] [Selected score]
S1-> 0.7
S2-> 0.6
<Selection engine 11> * first selection threshold = 0.6
Here, the selection engine 11 outputs the input data to both the recognition engines 121 and 122 whose selection score is equal to or higher than the first selection threshold (for example, 0.6).
Note that the first selection threshold can be arbitrarily set by the operator.

Next, the first recognition engine 121 and the second recognition engine 122 output a recognition score for each context with respect to the input data to the recognition score determination unit 13. Here, the “highest value” in a plurality of recognition scores in a plurality of contexts is a statistical value.
<First recognition engine 121> (Context): (Recognition score)
c11: 0.5
c12: 0.2
c13: 0.1
* Maximum value (statistical value) = 0.5
<Second recognition engine 122> (context): (recognition score)
c21: 0.7
c22: 0.3
c23: 0.3
* Maximum value (statistical value) = 0.7
<Recognition score determination unit 13> * Recognition threshold = 0.6

The recognition score determination unit 13 determines whether or not the recognition score is a recognition threshold (0.6) or more. Here, since only the second recognition engine 122 has a recognition score of 0.6 or more, the identifier (ID: 122) of the second recognition engine 122 is fed back to the selection engine 11.
As a result, the selection engine 11 re-learns the learning model based on the teacher data in which the input data is associated with the fed back identifier of the second recognition engine 122.
Although the statistical value has been described as being the highest value according to FIG. 3, it may be the lowest value, the average value, or the added value.

  FIG. 4 is a specific second processing flow in the present invention.

According to FIG. 4, it is assumed that the selection engine 11 calculates a selection score for each recognition engine for input data to be estimated as follows, as compared with FIG.
[Recognition engine ID] [Selected score]
S1-> 0.7
S2-> 0.6
<Selection engine 11> * First selection threshold = 0.7
Here, the selection engine 11 outputs the input data only to the first recognition engine 121 whose selection score is equal to or higher than the first selection threshold (for example, 0.7). In this case, input data is not output to the second recognition engine 122.

Next, the first recognition engine 121 outputs a recognition score for each context for the input data to the recognition score determination unit 13. Again, the “highest value” in the plurality of recognition scores of the context is used as the statistical value.
<First recognition engine 121> (Context): (Recognition score)
c11: 0.5
c12: 0.2
c13: 0.1
* Maximum value (statistical value) = 0.5

Then, the recognition score determination unit 13 feeds back to the selection engine 11 an identifier (ID: 121) of the first recognition engine 121 whose recognition score is equal to or greater than a recognition threshold (for example, 0.5).
Thereby, the selection engine 11 further re-learns the learning model as teacher data in which the input data is associated with the identifier of the first recognition engine 121.

  FIG. 5 is a specific third processing flow in the present invention.

According to FIG. 5, as in FIG. 3, the recognition score calculated by the recognition engine 12 is calculated by using the highest value of a plurality of scores in a plurality of contexts as a statistical value.
<First recognition engine 121> (Context): (Score)
c11: 0.6
c12: 0.2
c13: 0.1
* Maximum value (statistical value) = 0.6
* Processing time = 100ms
<Second recognition engine 122> (context): (score)
c21: 0.7
c22: 0.3
c23: 0.3
* Maximum value (statistical value) = 0.7
* Processing time = 500ms
<Recognition score determination unit 13> * Recognition threshold = 0.6
* Predetermined threshold time = 200 ms

The recognition score determination unit 13 determines whether or not the recognition score is a recognition threshold (0.6) or more. Here, the recognition scores of both the recognition engines 121 and 122 are 0.6 or more.
The recognition score determination unit 13 determines whether the processing time required for the input data is equal to or shorter than a predetermined threshold time (200 ms). Here, the processing time of the second recognition engine 122 is 500 ms, which is false. In this case, the recognition score determination unit 13 feeds back only the identifier (ID: 121) of the first recognition engine 121 to the selection engine 11.
As a result, the selection engine 11 re-learns the learning model based on the teacher data in which the input data is associated with the identifier of the first recognition engine 121.
Thus, it is preferable from the viewpoint of processing resources to re-learn the learning model of the selection engine 11 based not only on the recognition score but also on the “processing time” of the recognition engine.

  FIG. 6 is a specific fourth processing flow in the present invention.

According to FIG. 6, the learning model of the selection engine 11 is considered in the learning omission. That is, even if the recognition engine 12 has a selection score that is less than the first selection threshold, the recognition engine 12 to be selected cannot be selected because the learning model of the selection engine 11 is incompletely learned. There is sex. In this case, the recognition score of the recognition engine 12 is determined again to determine whether or not to use the learning engine for the learning model again.
According to FIG. 6, two embodiments are described.

<First Embodiment>
According to the selection engine 11 described above, the first recognition engine 121 having a selection score equal to or higher than a first selection threshold (for example, 0.6) is selected.
On the other hand, the selection engine 11 also selects the second recognition engine 122 whose selection score is less than the first selection threshold (for example, 0.6) and equal to or higher than the second selection threshold (for example, 0.5). .
Then, the selection engine 11 outputs input data to both of the selected recognition engines 121 and 122.
Next, the recognition engines 121 and 122 output a recognition score for each context for the input data to the recognition score determination unit 13.
Then, the recognition score determination unit 13 feeds back to the selection engine 11 the identifiers of the recognition engines 121 and 122 whose recognition score is equal to or greater than the recognition threshold (for example, 0.6).
As a result, the selection engine 11 re-learns the learning model as teacher data in which the input data and the identifiers of the recognition engines 121 and 122 are associated with each other.

<Second Embodiment>
According to the selection engine 11 described above, the first recognition engine 121 having a selection score equal to or higher than a first selection threshold (for example, 0.6) is selected.
On the other hand, the selection engine 11 further selects the selection score (for example, 0.7) of the selected first recognition engine 121 and the selection score (for example, the second recognition engine 122 that is less than the first selection threshold). It is determined whether or not the difference from 0.5) is a predetermined difference (for example, 0.2) or less. If it is determined to be true, the selection engine 11 also selects the second recognition engine 122 whose selection score is a first selection threshold (for example, less than 0.6).
Then, the selection engine 11 outputs input data to both of the selected recognition engines 121 and 122.
Next, the recognition engines 121 and 122 output a recognition score for each context for the input data to the recognition score determination unit 13.
Then, the recognition score determination unit 13 feeds back to the selection engine 11 the identifiers of the recognition engines 121 and 122 whose recognition score is equal to or greater than the recognition threshold (for example, 0.6).
As a result, the selection engine 11 re-learns the learning model as teacher data in which the input data and the identifiers of the recognition engines 121 and 122 are associated with each other.

  According to FIGS. 3 to 6 described above, the recognition scores calculated by all the recognition engines 12 are determined by one recognition threshold value. On the other hand, as another embodiment, each recognition engine 12 may be determined by a different recognition threshold.

  In addition, when the recognition score calculated by all the recognition engines 12 is less than a recognition threshold value, you may be made to recognize separately or with a specific recognition engine, and the said input data may be without a recognition engine. As the associated teacher data, the learning model of the selection engine 11 may be relearned.

FIG. 7 is a specific fifth processing flow for video data.
FIG. 8 is a flowchart based on FIG.

Assume that the recognition device 1 inputs video data in which a human action is reflected as input data and estimates an action recognition result (context).
According to FIG.7 and FIG.8, it has three mutually different recognition engines.
(1) Object recognition engine based on RGB image (2) Motion recognition engine based on optical flow (3) Human joint recognition engine based on skeleton information Each of these recognition engines has an action result in a large amount of video data in which a person is reflected. A learning model is generated in advance using the associated teacher data. In object recognition, moving object recognition, and joint recognition, even if the same video data is recognized, the contexts as behavior results may be different.

(1) The object recognition engine based on RGB recognition specifically estimates an object (target object) reflected in a captured image using a neural network such as CNN (Convolutional Neural Network).
For example, when an object is reflected in video data such as “cop”, “smartphone”, “television”, and “building”, the object is recognized with high accuracy.

(2) The moving object recognition engine based on the optical flow extracts a part where the same feature point is moving between frames, and represents the movement of the object in the captured video as a “vector”.
For example, when a person's movement is reflected in the video data, such as “gripping”, “shaking”, “punch”, “kick”, the moving object is recognized with high accuracy.

(3) The human joint recognition engine based on the skeleton information specifically extracts a feature point of a human joint using a skeleton model such as OpenPose (registered trademark) (for example, Non-Patent Document 7). To 9). OpenPose is software that can detect multiple human body / hand / face keypoints in real time from images, and is published by GitHub. For example, 15 key points can be detected in the whole body of a person shown in a captured image.
For example, if the movement of a person based on the angle or position of a person's joint is reflected in the video data, such as “drink”, “eat”, “run”, or “fold”, the movement of the person's joint is recognized with high accuracy. .

  As for human action recognition, in general, moving object recognition and joint recognition have higher recognition accuracy than object recognition. Also, in the case of motion recognition of a person's body, joint recognition has higher recognition accuracy than motion recognition.

According to FIG.7 and FIG.8, it processes as follows.
(S10) The recognition apparatus 1 inputs “video data”.
(S11) According to FIG. 7, it is assumed that the selection engine 11 has selected all the recognition engines 12. In this case, the selection engine 11 outputs video data to each recognition engine 12.
(S12) Each recognition engine 12 outputs the following context and recognition score.
<RGB Recognition Engine 121> (Context): (Score)
Cup: 0.7
Smartphone: 0.4
TV: 0.1
* Maximum value (statistical value) = 0.7
<Optical flow recognition engine 122> (context): (score)
Grasping: 0.4
Shake: 0.2
Punch: 0.1
* Maximum value (statistical value) = 0.4
<Skeleton recognition engine 123> (context): (score)
Drinking: 0.6
Eat: 0.2
Run: 0.0
* Maximum value (statistical value) = 0.6
(S13) The recognition score determination unit 13 determines whether or not the recognition score is greater than or equal to the recognition threshold (0.6). Here, the recognition engines 121 and 123 have a recognition score of 0.6 or more.
Further, among the recognition scores calculated by the recognition engines 121 and 123, the contents “cup” and “drink” that are equal to or higher than the recognition threshold are output to the application as estimation results.
(S14) The identifiers (ID: 121, 123) of the recognition engines 121 and 123 are fed back to the selection engine 11.
As a result, the selection engine 11 re-learns the learning model based on the teacher data in which the input data is associated with the fed back identifiers of the recognition engines 121 and 123.

  As another embodiment, the recognition engine of the present invention is not limited to the one based on video data, but may be a character recognition one or a recognition object dedicated to a specific object (for example, a type of flower). Also good.

As described above in detail, according to the program, apparatus, and method of the present invention, the context recognition accuracy is improved by automatically selecting one or more optimum recognition engines according to input data. Can do.
According to the present invention, since the recognition engine for the input data is switched by the selection engine having the learning model, it is not necessary to determine in advance.
In particular, according to the present invention, the learning model of the selection engine can be relearned not only at the learning stage but also at the operation stage.

  Various changes, modifications, and omissions of the above-described various embodiments of the present invention can be easily made by those skilled in the art. The above description is merely an example, and is not intended to be restrictive. The invention is limited only as defined in the following claims and the equivalents thereto.

DESCRIPTION OF SYMBOLS 1 Recognition apparatus 11 Selection engine 12 Recognition engine 121 1st recognition engine 122 2nd recognition engine 13 Recognition score determination part 2 Terminal

Claims (9)

In a recognition program that allows a computer to function to infer context from input data using multiple recognition engines,
A selection engine that selects a recognition engine for input data to be estimated using a learning model learned from teacher data in which input data is associated with an identifier of the recognition engine, and outputs the input data to the selected recognition engine When,
Determining a recognition engine in which the recognition score calculated by the recognition engine for the input data is equal to or greater than a recognition threshold, and causing the computer to function as a recognition score determination unit that feeds back the identifier of the recognition engine to the selection engine;
The selection engine causes a computer to function so as to re-learn the learning model based on teacher data in which the input data is associated with the fed back identifier of the recognition engine. The recognition engine is based on a class classification that calculates a recognition score for each class,
The recognition engine causes a computer to calculate a statistical value of any one of a maximum value, a minimum value, an average value, and an addition value of a plurality of scores of a plurality of classes as the recognition score. Item 4. The recognition program according to item 1. The recognition score determination means causes the computer to function so as to feed back only the identifier of the recognition engine whose processing time required for the input data is equal to or less than a predetermined threshold time to the selection engine. The recognition program according to claim 1 or 2. The selection engine is based on a class classification that calculates a selection score for each class,
The said selection engine makes a computer function so that the said input data may be output to the recognition engine from which the said selection score with respect to the input data which should be estimated becomes more than a 1st selection threshold value. The recognition program according to any one of claims. The selection engine further causes the computer to function so as to output the input data to a recognition engine whose selection score is less than a first selection threshold and greater than or equal to a second selection threshold. The recognition program described in 1. The selection engine has a difference between a selection score of one recognition engine that is equal to or higher than the first selection threshold and a recognition score of the other recognition engine that is lower than the first selection threshold is equal to or less than a predetermined difference. 5. The recognition program according to claim 4, further causing a computer to function to output the input data to the other recognition engine. The input data is video data,
The plurality of recognition engines are different from each other.
An object recognition engine based on RGB images;
Motion recognition engine based on optical flow and / or
The recognition program according to any one of claims 1 to 6, wherein the computer functions so as to be one of human joint recognition engines based on skeleton information. In a recognition apparatus that estimates a context from input data using a plurality of recognition engines,
A selection engine that selects a recognition engine for input data to be estimated using a learning model learned from teacher data in which input data is associated with an identifier of the recognition engine, and outputs the input data to the selected recognition engine When,
A recognition score determining means for determining a recognition engine for which the recognition score calculated by the recognition engine for the input data is equal to or greater than a recognition threshold, and feeding back an identifier of the recognition engine to the selection engine;
The recognition apparatus, wherein the selection engine re-learns the learning model based on teacher data in which the input data is associated with the fed back identifier of the recognition engine. In a method for recognizing a device that estimates a context from input data using a plurality of recognition engines,
The device is
First, a recognition engine for input data to be estimated is selected using a learning model learned from teacher data in which input data is associated with an identifier of a recognition engine, and the input data is output to the selected recognition engine. And the steps
A second step of determining a recognition engine having a recognition score calculated by the recognition engine for the input data equal to or greater than a recognition threshold;
And a third step of re-learning the learning model with the teacher data in which the input data is associated with the identifier of the recognition engine determined to be true in the second step. Recognition method.

Images