JP2020101948A - Action recognition system and action recognition method - Google Patents

Abstract

To make it possible to perform highly accurate behavior recognition even if there is only a small amount of labeled data in a target domain and there are data in various source domains.SOLUTION: An action recognition system 1 comprises: an inter-domain similarity calculation unit 13 for calculating, using domain information 12 regarding a target domain as an input, inter-domain similarity between the target domain and a plurality of domains based on first information stored in an inter-domain relationship information DB14; a data conversion unit 15 for executing a predetermined data conversion on input sensor data 11 to be identified using reference information read from second information stored in domain-based sensor data DB16 based on the inter-domain similarity; and an identifying unit 17 for identifying an action type corresponding to the input sensor data 11 using the input sensor data 11 after the data conversion and parameters of the identifier stored in the domain-based identifier DB18.SELECTED DRAWING: Figure 1

Description

The present invention relates to an action recognition system and an action recognition method, and is suitable for being applied to an action recognition system and an action recognition method for automatically recognizing an action type using sensor data.

Conventionally, there has been a demand for automatically recognizing human behavior by analyzing sensor data obtained from a sensor such as a camera for monitoring system, life log, or marketing for each customer behavior. A large number of behavior recognition technologies have been developed to meet such demand, and a typical method thereof is to collect learning data for each behavior to be recognized and to use a neural network or SVM (Support Vector Machine). This method was used to learn the association between sensor data and action labels.

When deploying an action recognition system using the above action recognition technology to various sites, if the action type of the recognition target is the same as that of the existing site where the system has already been deployed, the action recognition model will be applied to the new site. From the viewpoint of system deployment cost, it is desirable to be able to utilize the data and models that have already been collected at the existing site without recollecting the learning data for construction.

However, the statistical properties of the behavior data collected at different sites may differ due to various factors even if the behavior types are the same. For example, even with the same "screw tightening" behavior, if the site is different, there are differences in the screws themselves and tools, differences in the screw mounting positions, or differences in the degree of screw tightening. Work methods may differ. As a result, the behavior data collected by observing the behavior with a sensor may have different properties depending on the site. In other words, the various factors that contribute to the generation of data are different, and the distribution of data generation is also different. It is generally known that when data generated from different generation distributions or an identification model constructed based on the data is used, the identification accuracy is lowered.

In order to solve such a problem, for example, in Patent Document 1, it is possible to perform accurate identification by using a model constructed using data whose generation distribution is the same as that of the acquired data. An intended object recognition method is disclosed. The object recognition device disclosed in Patent Document 1 is detected by an image acquisition unit that acquires an image captured by the imaging unit, a brightness detection unit that detects brightness around the vehicle, and a brightness detection unit. An image recognition unit that changes the content of the process of recognizing an object included in the image acquired by the image acquisition unit according to the brightness, and more specifically, the brightness detected by the illuminance sensor. Depending on the situation, the model for daytime and the model for nighttime are switched and used.

In addition, Patent Document 2, Non-Patent Document 1 and Non-Patent Document 2 disclose a technique called domain matching, which performs accurate identification by performing conversion so that generation distributions of different data are matched and then performing identification. Has been done. Here, the domain is a term indicating an environment related to data generation, and generally, it is considered that the data generation distribution is different when the data collection domain is different. For example, in the above example, one site can be considered as one domain and another site can be considered as another domain. The domain to which the identification data (identification target data) belongs is called a target domain. Generally, it is assumed that only a small amount of labeled data (learning data) is available from the target domain. In the above example, the target domain is a site where a behavior recognition system is being newly developed, and the situation in which labeled data cannot be collected from the site corresponds to this. On the other hand, a domain where a large amount of labeled data exists is called a source domain. In the above example, this corresponds to the existing site where the system has already been deployed. Domain matching is a transformation of the data distribution of the target domain, which has only a small amount of training data, to the data distribution of the source domain, which has a large amount of training data. This is a technology that attempts to perform accurate identification using a model that was originally constructed.

Japanese Patent Laid-Open No. 2018-041209 JP, 2016-058079, A

V. Patel, R. Gopalan, R. Li and R. Chellappa: Visual domain adaptation: A survey of recent advances.IEEE signal processing magazine, Vol. 32, No. 3, pp 53--69, 2015 H. Daume and D. Marcu: Domain adaptation for statistical classifiers. Journal of Artificial Intelligence Research, Vol. 26, pp 101--126, 2006

However, in the object recognition device disclosed in Patent Document 1 described above, the switchable daytime model and nighttime model are models learned from data collected from the same domain by day and night, and The data to be identified is also acquired from the same domain. That is, the object recognition technology disclosed in Patent Document 1 can be applied only when the source domain is single and the source domain and the target domain are the same.

In addition, the domain matching techniques disclosed in Patent Document 2, Non-Patent Document 1 and Non-Patent Document 2 also assume only a single source domain. However, when deploying the behavior recognition system in various fields, the number of source domains will increase as the deployment of the system progresses. When the conventional domain matching technique is applied in such a situation, there is a problem that it cannot be simply applied because it is not known to which source domain the target domain should be matched.

The present invention has been made in consideration of the above points, and the situation in which the behavior recognition system is further developed and the system is being introduced to a new site, that is, the labeled data of the target domain is small. In addition, the present invention aims to propose a behavior recognition system and a behavior recognition method that enable highly accurate behavior recognition even in a situation where data of various source domains exist.

In order to solve such a problem, the present invention provides the following action recognition system for identifying an action type corresponding to the sensor data, using the sensor data collected in the target domain as an identification target. This action recognition system includes a first storage unit that stores first information regarding a relationship between a plurality of domains for a plurality of domains for which sensor data has been collected in the past, and an action corresponding to the past sensor data. A second storage unit that stores second information that associates a type with the domain from which the sensor data is collected, and an action type corresponding to the sensor data using data for each domain of the plurality of domains. A third storage unit that stores third information including a parameter in the discriminator for each domain that has been learned to be discriminated, and the first storage unit that stores information about the target domain as input. Based on the information of 1, the inter-domain similarity calculation unit that calculates the inter-domain similarity between the target domain and the plurality of domains, and the inter-domain similarity calculated by the inter-domain similarity calculation unit. On the basis of the above, reference information is read from the second information stored in the second storage unit, and predetermined reference data conversion is performed on the sensor data to be identified using the reference information. Using the data conversion unit to be executed, the sensor data of the identification target that has been data-converted by the data conversion unit, and the parameter of the discriminator included in the third information stored in the third storage unit, An identification unit that identifies an action type corresponding to the sensor data to be identified.

Further, in order to solve such a problem, the present invention provides the following behavior recognition method for identifying a behavior type corresponding to the sensor data, with the sensor data collected in the target domain as an identification target. This behavior recognition method inputs information on the target domain and inputs the target domain and the plurality of target domains based on first information on a relationship between the plurality of domains for which sensor data has been collected in the past. Based on the similarity between the domains calculated in the inter-domain similarity calculation step for calculating the similarity between the domains and the domain, the action corresponding to the past sensor data The reference information is read out from the second information in which the type and the domain from which the sensor data is collected are associated with each other, and the reference information is used to determine the predetermined data for the sensor data to be identified. A data conversion step of performing conversion, the sensor data of the identification target converted in the data conversion step, and data for each domain of the plurality of domains are used to identify the action type corresponding to the sensor data. An identification step of identifying the action type corresponding to the sensor data of the identification target by using the learned parameter in the discriminator for each domain.

According to the present invention, it is possible to perform highly accurate action recognition even in the situation where there is only a small amount of labeled data in the target domain and data in various source domains may exist.

It is a block diagram which shows the structural example of the action recognition system which concerns on the 1st Embodiment of this invention. It is a block diagram which shows the hardware structural example of the action recognition system shown in FIG. It is a figure which shows an example of the relationship information between domains. It is a figure which shows an example of GUI used for inputting the relationship information between domains. It is a figure which shows an example of the sensor data classified by domain. It is a figure which shows an example of discriminator parameter information. It is a block diagram which shows the structural example of the action recognition system which concerns on the 2nd Embodiment of this invention. It is a figure for demonstrating the structural relationship information between domains when using a hierarchical structure for the structural relationship between domains. It is a figure which shows an example of GUI used for inputting the structure relation information between domains by a hierarchical structure. 14 is a flowchart showing a processing procedure example of a similarity calculation processing when a hierarchical structure is used for the structural relationship between domains in the second embodiment. It is a figure for demonstrating the structural relationship information between domains when using a network structure for the structural relationship between domains. It is a figure which shows an example of GUI used for the input of the structural relationship information between domains by a network structure. It is a block diagram which shows the structural example of the action recognition system which concerns on the 3rd Embodiment of this invention. It is a figure which shows an example of domain word embedding information.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In each drawing, components having the same or similar functions are designated by the same reference numerals, and repeated description is omitted.

(1) First Embodiment (1-1) Functional Configuration of Action Recognition System FIG. 1 is a block diagram showing a configuration example of the action recognition system according to the first embodiment of the present invention. The action recognition system 1 according to the first exemplary embodiment shown in FIG. 1 includes input sensor data 11 of an identification target acquired by a sensor, and domain information 12 that is information of a domain in which the input sensor data 11 is collected. Is input, the processing for identifying the action type corresponding to the input sensor data 11 to be identified is executed, and the identification result 19 is output. In the following description, the input sensor data 11 and the domain information 12 are collectively referred to as identification target data 10.

The input sensor data 11 is acquired by using one or a combination of various sensors such as a camera, a microphone, a distance sensor, an acceleration sensor, a gyro sensor, a myoelectric sensor, a pulse/heartbeat sensor, an illuminance sensor, or a pressure sensor. However, the type and number of sensors are not limited. Further, the action recognition system 1 may temporarily store the measurement value of the sensor in a storage device such as an HDD (Hard Disk Drive) and then input the input sensor data 11 again to perform the subsequent processing. (Offline processing), the measurement value read from the sensor may be directly input as the input sensor data 11 to perform the subsequent processing (real-time processing).

As shown in FIG. 1, the behavior recognition system 1 includes an inter-domain similarity calculation unit 13, an inter-domain relationship information DB 14, a data conversion unit 15, a domain-specific sensor data DB 16, an identification unit 17, and a domain-specific discriminator DB 18. It is configured with.

Explaining the outline of the processing flow by the action recognition system 1, the action recognition system 1 first reads the domain information 12 of the identification target data. Then, the inter-domain similarity calculation unit 13 reads out information about the inter-domain relationship stored in the inter-domain relationship information DB 14, and identifies the domain (target domain) of the identification target data 10 and the inter-domain relationship information. The degree of similarity with each domain (source domain) whose information is stored in the DB 14 is calculated (similarity degree calculation processing).

Next, the data conversion unit 15 reads the input sensor data 11, and based on the similarity calculated by the inter-domain similarity calculation unit 13, determines the existing sensor data used when converting the input sensor data 11. Then, the determined sensor data is read from the domain-specific sensor data DB 16 and the input sensor data 11 is converted (data conversion processing).

Finally, the identification unit 17 reads from the domain-based discriminator DB 18 the parameters of the discriminator constructed using the existing sensor data used when converting the input sensor data 11, and the read parameters of the discriminator. Is used to execute the process of identifying the action type on the converted input sensor data 11 and output the identification result 19 (action identification process).

(1-2) Hardware Configuration FIG. 2 is a block diagram showing a hardware configuration example of the action recognition system shown in FIG. The action recognition system 1 can be realized by, for example, a computer 100 having the hardware configuration shown in FIG.

As shown in FIG. 2, the computer 100 is configured to include an input device 101, a central processing unit 102, a main storage device 103, an auxiliary storage device 104, and an output device 105, and each configuration is via a bus 106. Connected.

In the computer 100, the central processing unit 102 executes the program stored in the main storage device 103. The main storage device 103 is a high-speed and volatile storage device such as a DRAM (Dynamic Random Access Memory), and stores an operating system and application programs. Then, the central processing unit 102 executes the operating system to realize the basic functions of the computer 100. Further, the central processing unit 102 executes the application program, so that the functions of the inter-domain similarity calculation unit 13, the data conversion unit 15, and the identification unit 17 in the action recognition system 1 shown in FIG. 1 are realized.

The auxiliary storage device 104 is a large-capacity and non-volatile storage device such as a magnetic storage device or a flash memory, and stores a program executed by the central processing unit 102 and data used when the program is executed. That is, the program executed by the central processing unit 102 is read from the auxiliary storage device 104, loaded into the main storage device 103, and executed by the central processing unit 102. The inter-domain relationship information DB 14, the domain-based sensor data DB 16, and the domain-based discriminator DB 18 in the behavior recognition system 1 illustrated in FIG. 1 can be stored in the auxiliary storage device 104.

The input device 101 is, for example, a keyboard or a mouse. The computer 100 can receive the input sensor data 11 based on the operation of the input device 101, for example. Here, the input sensor data 11 may be transmitted from a sensor connected to the computer 100 using, for example, a USB (Universal Serial Bus) cable or the like, may be transmitted via a network, or a CD or Alternatively, the data may be written once in a medium such as a DVD and then read by the computer 100 to obtain it. The output device 105 is, for example, a liquid crystal display, a projector, a printer, or the like, and can output the identification result 19 (display on a predetermined screen, for example) as an output unit (not shown). Further, the output device 105 has a predetermined GUI (Graphic User Interface) for input of data registered in each DB (inter-domain relationship information DB 14, domain-specific sensor data DB 16, domain-specific discriminator DB 18) of the action recognition system 1. ) Or displaying a predetermined screen for browsing the data stored in each DB.

The hardware configuration of the computer 100 that implements the behavior recognition system 1 is not limited to the example illustrated in FIG. 2, and other examples include, for example, the inter-domain similarity calculation unit 13, the data conversion unit 15, or the identification unit. A GPU (Graphic Processing Unit) may be used for speeding up the calculation by 17. It is preferable that each of these devices can exchange data via the bus 106.

(1-3) Details of Processing and Data Hereinafter, the processing and data format of each component in the action recognition system 1 according to the present embodiment will be described in more detail.

(1-3-1) Identification Target Data As described above, the identification target data 10 includes the input sensor data 11 and the domain information 12. Of these, the input sensor data 11 is generally a multidimensional vector, and may be the value itself output from the sensor or may be the data after performing a predetermined preprocessing (feature extraction).

Further, as the domain information 12, for example, the name of the domain from which the identification target data 10 is acquired can be used. Specifically, “factory A”, “work site B” and the like, which indicate the data acquisition location, are examples of domain names. As mentioned above, the term "domain" means the environment related to the generation of data, so not only the location where the data was acquired as described above, but also the "person" who performed the action is also one domain. Can be considered. This is because even if the same action is taken, different workers may have different ways of taking the action. In this case, for example, “worker C” or the like can be considered as the domain information 12. In addition to this, any factor that has a certain influence on the generation of data can be considered as one domain.

(1-3-2) Inter-domain Relationship Information and Similarity Calculation Processing FIG. 3 is a diagram showing an example of inter-domain relationship information. The inter-domain relationship information 141 shown in FIG. 3 is an example of data stored in advance in the inter-domain relationship information DB 14 shown in FIG. 1, and holds similarity between domains in a matrix format. More specifically, in the case of FIG. 3, between the domains of “Factory A”, “Factory B”, “Line C”, “Line D”, “Worker E”, and “Worker F”, Is indicated by a numerical value between "0" and "1". It should be noted that in the present example, the “similarity” is used as an example of the index indicating the relationship between domains, but the present embodiment is not limited to this, and other indices such as the degree of divergence, for example. May be used.

Here, a method of registering the inter-domain relationship information 141 will be described. In the action recognition system 1, for example, the inter-domain relationship information 141 can be registered in the inter-domain relationship information DB 14 based on an input operation by a user (including an administrator or the like) on a predetermined GUI.

FIG. 4 is a diagram showing an example of a GUI used for inputting inter-domain relationship information. The new domain registration screen 142 shown in FIG. 4 is a GUI for newly registering a domain in the inter-domain relationship information DB 14. The specific registration method is as follows. First, the user inputs the name of the new domain to be newly registered in the text box 1421, and then, in the input field 1422, the similarity of the new domain to the already registered domain. input. After that, by clicking the registration button 1425, the input content regarding the new domain is added to the inter-domain relationship information 141 and registered in the inter-domain relationship information DB 14. If the user wants to cancel the registration of the new domain on the new domain registration screen 142, clicking the return button 1424 returns to the page before the transition.

Note that when registering the first domain on the new domain registration screen 142, there is no registered domain, so there is no need to enter it in the input field 1422. Instead, enter the domain name in the text box 1421 and enter the register button 1425. Just click. Further, when the matrix display button 1423 is clicked on the new domain registration screen 142, the inter-domain relationship information 141 registered so far in the inter-domain relationship information DB 14 is displayed in a predetermined screen in a matrix format as shown in FIG. Is displayed. By allowing browsing in such a matrix display format, the convenience of the user can be improved.

In order to input the relationship between domains as described above, the user needs to have knowledge enough to define the similarity of each domain. Therefore, as a specific user who inputs data (inter-domain relationship information 141) to the inter-domain relationship information DB 14, for example, an administrator who is in a position of managing the entire factory of the company is suitable. According to such a user, specifically, for example, since “Factory A” and “Factory B” are working using the same tools and parts, the similarity of the work behavior is higher than that of other factory. Even if the same “Factory A”, “Line C” and “Line D” apply the same regulation as work behavior for safety, it is judged that the similarity is higher. Since the “workers E” and “workers F” have similar physiques, it is possible to determine that the degree of similarity is high.

Next, the similarity calculation processing according to the present embodiment will be described. As described above, the similarity calculation process in the present embodiment is a process in which the inter-domain similarity calculation unit 13 calculates the similarity between the domain (target domain) of the identification target data and another domain.

In the present embodiment, it is conceivable that the similarity of each domain is directly stored in the inter-domain relationship information DB 14 like the inter-domain relationship information 141 illustrated in FIG. 3, and in such a case, The inter-domain similarity calculation unit 13 can acquire the similarity of the domain of the identification target data 10 to other domains only by reading the similarity without performing a special calculation process in the similarity calculation process. .. However, in order to directly store the similarity of each domain, all of them need to be manually input as described with reference to FIG. On the other hand, a method of calculating the similarity when only the indirect relationship between the domains is stored will be described in detail in the second and third embodiments described later.

(1-3-3) Sensor Data by Domain and Data Conversion Process FIG. 5 is a diagram showing an example of sensor data by domain. The domain-specific sensor data 161 illustrated in FIG. 5 is an example of data stored in the domain-specific sensor data DB 16 illustrated in FIG. 1. Specifically, the sensor data value (sensor value 1613) acquired in the past is used. ), the domain name (domain 1611) from which the sensor value 1613 is acquired and the label name (action type 1612) of the corresponding action type are stored in association with each other. Here, the sensor value 1613 “x _i ”is generally a multidimensional vector. Further, as the sensor value 1613, a measurement value directly obtained from the sensor may be used as it is, or a value after subjecting the measurement value to a predetermined preprocessing (for example, feature extraction) may be used.

Next, the data conversion processing by the data conversion unit 15 in the present embodiment will be described.

In the data conversion process, first, the data conversion unit 15 calculates, based on the similarity between the domain (target domain) of the identification target data 10 calculated by the inter-domain similarity calculation unit 13 and each existing domain (source domain), Existing data (source domain data) to be referenced when converting the input sensor data 11 is determined, and the determined domain-specific sensor data 161 is read from the domain-specific sensor data DB 16. As a method of determining and reading the data of the source domain, specifically, for example, a method of using only the data of the source domain having the highest degree of similarity, a method of using only the data of the source domain having the degree of similarity equal to or higher than a predetermined threshold value are used. Alternatively, a method may be considered in which the data of the source domain having a high degree of similarity is sequentially read, and the reading process is terminated when the number of data becomes equal to or more than a predetermined threshold value, but the method is not necessarily limited to these methods.

Next, the data conversion unit 15 converts the input sensor data 11 using the read source domain data. When only the data of the source domain having the highest degree of similarity is used in the conversion of the input sensor data 11, the situation is similar to the case where there is only one type of source domain, and thus, for example, Non-Patent Document 1 and Non-Patent Document described above. The input sensor data 11 can be converted by applying any existing domain matching method described in 2 etc.

On the other hand, when a plurality of types of source domain data are used in the conversion of the input sensor data 11, the data used for the data conversion can be determined by performing the following processing, for example. That is, if the source domain to be used is “s={s1, s2,..., sN}” and the similarity between the target domain (domain of the identification target data 10) and the source domain si is “simi”, then use "Si*simi" pieces of data are randomly sampled from each of the source domains to be used. As a result, a total of “Σi=1N{si*simi}” data is used for the entire source domain. When the data to be referred to at the time of conversion is determined in this way, thereafter, by using these data, any existing domain matching method described in Non-Patent Document 1 or Non-Patent Document 2 described above is applied. , The input sensor data 11 can be converted.

Further, as a special example of the data conversion processing, it is possible to use the input sensor data 11 as it is, without performing identity conversion, that is, actually performing data conversion. In general, data collected in different domains have different statistical properties, so it is not preferable to use a discriminator trained with data in other domains without data conversion, but the method is simple. From the viewpoint of ease of interpretation, by performing the identity mapping (identity transformation) for the sake of convenience, the input sensor data 11 can be identified from the identification model substantially learned with the data of the source domain. It is also possible to realize a function such as "model selection" for selecting a model to be used.

(1-3-4) Discriminator Parameter Information and Behavior Discrimination Processing The behavior discrimination processing according to the present embodiment and information regarding the discriminator parameters used in the behavior discrimination processing (discriminator parameter information) will be described.

As described above, the action identifying process according to the present embodiment is executed by the identifying unit 17. More specifically, the identification unit 17 reads out the parameters of the discriminator constructed using the data of the source domain used in the data conversion processing by the data conversion unit 15 from the domain-specific discriminator DB 18, and reads the read parameters. By using the input sensor data 11 after the data conversion processing, the processing for identifying the action type is executed, and the processing result is output as the identification result 19.

In the action identifying process of the present embodiment, when the identifying unit 17 identifies the action type with respect to the input sensor data 11, a neural network (NN) or a support vector machine (SVM: Support Vector Machine) is used. ) Or any random identification method, such as a random forest method. For example, when only one type of source domain data is used as reference data when the data conversion unit 15 converts the input sensor data 11 in the above-described data conversion process, the discriminator of the source domain data may be used in advance. By learning the above, the identification unit 17 can read the learned parameter (identifier parameter) in the identification processing and use it for the identification of the input sensor data 11.

FIG. 6 is a diagram showing an example of the discriminator parameter information. The discriminator parameter information 181 illustrated in FIG. 6 is an example of the data stored in the domain-based discriminator DB 18 illustrated in FIG. 1. As described in the preceding paragraph, the discriminator parameter information 181 indicates the discriminator of the source domain. It can be obtained in advance by learning. In the case of FIG. 6, in the discriminator parameter information 181, discriminator parameters 1813 used in the discrimination processing are defined for each combination of the domain 1811 and the action type 1812. Here, the discriminator parameter 1813 “θ _i ”is generally a vector value.

In the present embodiment, as described above, the parameters of the discriminator for each source domain may be learned in advance and stored in the domain discriminator DB 18, but as another method, After the data conversion unit 15 determines the reference data when converting the input sensor data 11, the discriminator is learned again using the determined reference data, and the learned discriminator is also used for the action discrimination processing. You may In other words, when the data conversion unit 15 performs the action identification process, the above-mentioned another method selects data to be used for learning of the discriminator based on the similarity between the domains, and the learning data is classified by the domain. The discriminator is read from the sensor data DB 16 and the discriminator is learned based on the read learning data. The data conversion unit 15 uses the parameters of the discriminator after learning to discriminate the behavior of the input sensor data 11. Perform processing. The subject of learning of the above-described classifier is not particularly limited, and may be executed by a predetermined processing unit (for example, the data conversion unit 15) included in the action recognition system 1. According to such a method, the discriminator can be learned at any time by using the processing results of the similarity calculation process and the data conversion process, and the parameter information of the discriminator after learning can be obtained. It becomes unnecessary to prepare the discriminator parameter information 181 in the domain-based discriminator DB 18 by learning the parameters of the discriminator.

Further, when outputting the identification result 19 which is the processing result of the action type processing, the identification unit 17 outputs the identified action type, but at this time, the reliability of the identification result is given and output. May be. By adding and outputting the reliability, the user corrects the identification result with low reliability while referring to the input sensor data 11, or discards the identification result as necessary. It becomes possible to deal with the above. Regarding the reliability of the identification result, it is preferable to use a value that can be standardly output by many identification models. Specifically, for example, when a neural network is used as the discriminant model, the softmax function is normally applied in the final layer of the network, and “0≦p _c ≦1 and Σ” for each action class “c”. _{c p} c = ₁ "and becomes such a value p _c can be calculated, the thus calculated value p _c, can be used as a value representing the reliability of the identification results.

Furthermore, in the action recognition system 1 according to the present embodiment, the series of processes as described above (specifically, the similarity calculation process by the inter-domain similarity calculation unit 13, the data conversion process by the data conversion unit 15, and After performing the action identification process (by the identification unit 17), the domain information 12 of the identification target data 10 is stored in the inter-domain relationship information DB 14, and only the identification result 19 with high reliability of the input sensor data 11 is stored. May be automatically stored in the sensor data DB 16 for each domain. By doing so, the data of the source domain can be expanded every time the input sensor data 11 is newly acquired, and therefore, the data of the target domain (identification target data 10) is newly collected after that. In that case, it can be expected to further improve the accuracy of action identification.

As described above, according to the behavior recognition system 1 in the present embodiment, the labeled data (identification target data 10) of the target domain is small, and the data of various source domains (for example, FIG. 5). Even in the situation where the sensor data 161 for each domain shown in 1) exists, it is possible to realize highly accurate action recognition by determining the source domain used for the data conversion process based on the relationship between the domains. You can

(2) Second Embodiment In the behavior recognition system 1 according to the first embodiment described above, the behavior recognition system 1 is stored every time the user tries to additionally register information on a new domain in the inter-domain relationship information DB 14. There was some inconvenience in that it was necessary to manually input the similarity with all existing domains.

Here, it is known that it is possible to assume some "structural relationship" between domains that generate behavior data. A hierarchical structure is an example of the “structural relationship”. Specifically, for example, if there are circumstances such as using common parts and tools for each production line, the work behavior between workers who are working on the same “production line A” in a factory of a certain company. May have a stronger relationship (higher similarity) than the work behavior of a worker working on another “production line B” in the same factory. Furthermore, due to reasons such as having the same rules for work in the same factory, even if the production lines that are working are different, the work behavior between workers who are working on the production line in the same factory. May be more relevant (more similar) than the work behavior of a worker working on the production line of another factory.

Therefore, in the second embodiment, it is possible to easily describe the relationship between the domains by using the structural relationship between the domains as described above, and to calculate the similarity between arbitrary domains. By making the calculation easy, the inconvenience described above in the first embodiment is reduced. The action recognition system 2 according to the second embodiment will be described below in detail.

FIG. 7: is a block diagram which shows the structural example of the action recognition system which concerns on the 2nd Embodiment of this invention. The behavior recognition system 2 shown in FIG. 7 is different from the behavior recognition system 1 shown in FIG. 1 in terms of the functional configuration. The behavior recognition system 2 includes an inter-domain structural relationship information DB 21 instead of the inter-domain relationship information DB 14. An inter-domain similarity calculation unit 23 is provided instead of the similarity calculation unit 13. Although details will be described later, the inter-domain similarity calculation unit 33 performs processing different from that of the first embodiment based on the information indicating the structural relationship between domains stored in the inter-domain structural relationship information DB 21. The similarity calculation processing is executed based on the contents, and the similarity between the domain of the identification target data 10 (target domain) and each domain (source domain) in which information is stored in the inter-domain structural relationship information DB 21 is calculated.

The inter-domain structural relationship information DB 21 stores information (inter-domain structural relationship information) indicating a structural relationship between the source domains. In the following description, “hierarchical structure” and “network structure” will be described as specific examples of structural relationships between domains.

First, a case will be described where the structural relationship between domains according to the “hierarchical structure” is used in the second embodiment.

FIG. 8 is a diagram for explaining inter-domain structural relationship information when a hierarchical structure is used for the structural relationship between domains. 8A shows the inter-domain structural relationship information 211 as an example, and FIG. 8B shows the inter-domain structural relationship information held by the inter-domain structural relationship information 211 of FIG. 8A. The relationships are shown in hierarchical structure 212.

The inter-domain structural relationship information 211 of FIG. 8A is an example of data stored in the inter-domain structural relationship information DB 21, and specifically, for each domain of the source domain (domain name 2111), The combination with the domain (parent domain 2112) is stored. For example, in the case of FIG. 8A, the parent domains of the domains “worker A” and “worker B” are both “line C”, and the parent domain of the domain “line C” is “factory D”. I understand. For the domain “Factory D” located at the top of the hierarchical structure, a parent domain “root” is set for convenience.

Then, as a result of the combination with the parent domain being set for each domain as shown in FIG. 8(A), the inter-domain structural relationship information 211 shows that the inter-domain structural relationship as in the hierarchical structure 212 of FIG. 8(B). It means that the sexuality can be retained. As with the inter-domain structural relationship information 211, the domain information 12 of the identification target data 10 also inputs the domain and the parent domain.

FIG. 9 is a diagram showing an example of a GUI used for inputting inter-domain structural relationship information in a hierarchical structure. The new domain registration screen 213 shown in FIG. 9 is a GUI for newly registering a domain in the inter-domain structural relationship information DB 21. As a specific registration method, first, the user inputs the name of the new domain to be newly registered in the text box 2131, and then selects the parent domain of the new domain from the combo box 2132. After that, by clicking the registration button 2135, the input content regarding the new domain is added to the inter-domain structural relationship information 211 and registered in the inter-domain structural relationship information DB 21. If the user wants to cancel the registration of the new domain on the new domain registration screen 213, clicking the return button 2134 returns to the page before the transition.

The hierarchical structure display button 2133 is a button for displaying the structural relationship held by the inter-domain structural relationship information 211 registered in the inter-domain structural relationship information DB 21 up to now in the hierarchical structure format. When the structure display button 2133 is clicked, the current hierarchical structure between domains is displayed on a predetermined screen, as in the hierarchical structure 212 of FIG. 8B. By enabling browsing in such a display format on the new domain registration screen 213, it is possible to improve the convenience for the user, as in the case of the new domain registration screen 142 described in FIG.

FIG. 10 is a flowchart showing a processing procedure example of the similarity calculation processing when a hierarchical structure is used for the structural relationship between domains in the second embodiment. In the similarity calculation process according to the second embodiment, the inter-domain similarity calculation unit 23 uses the identification target data 10 and the inter-domain structural relation information 211 stored in the inter-domain structural relation information DB 21 to identify the identification target data. The similarity between the 10 domains (target domain) and each of the other domains (source domain) is calculated.

According to FIG. 10, first, in step S101, the inter-domain similarity calculation unit 23 performs the following processing. Specifically, the domain information of the domain (target domain) is read from the domain information 12 of the identification target data 10 and set as “t”. Further, "1" is set to the value "t.d" representing the distance from this target domain. Then, "1/t.d", that is, "1" is set to the value "t.sim" indicating the degree of similarity with the target domain. In addition, all existing source domains are set in the set “NA” of domains for which the degree of similarity with the target domain has not been calculated. The existing source domain here means a domain registered in the inter-domain structural relationship information DB 21. Also, "t" is set to the set "W" of domains for which the degree of similarity is being calculated.

In the next step S102, the inter-domain similarity calculation unit 23 determines whether or not the set “W” of domains for which the similarity is being calculated is an empty set (φ). If the set “W” is an empty set in step S102 (YES in step S102), it means that there is no source domain for which the similarity is to be calculated, and therefore the inter-domain similarity calculation unit 23 uses the similarity calculation process. To finish. On the other hand, if the set "W" is not an empty set in step S102 (NO in step S102), the process proceeds to step S103.

In step S103, the inter-domain similarity calculation unit 23 extracts one element (domain) from the set “W” and sets it as “x”. At this time, “x” is removed from the set “W”.

Next, in step S104, the inter-domain similarity calculation unit 23 extracts the domain “x” extracted in step S103 from the domain information 12 of the identification target data 10 or the inter-domain structural relationship information 211 stored in the inter-domain structural relationship information DB 21. , A set of parent domains “P(x)” and a set of child domains “C(x)” are read. Then, the set “P(x) ∩ NA” of which the similarity with the target domain has not been calculated in the read set “P(x)” is set to “P”. Similarly, the set “C(x) ∩ NA” of which the similarity with the target domain has not been calculated in the read set “C(x)” is set to “C”.

In the next step S105, the inter-domain similarity calculation unit 23 sets the value of "p.d" to "x.d+1" for all the elements included in "P" set in step S104, and sets "p.d". Set the value of ".sim" to "pd". Further, the inter-domain similarity calculation unit 23 also sets the value of “cd” to “x.d+1” and “c.d+1” for all the elements included in “C” set in step S104. Set the value of "sim" to "cd".

Next, in step S106, the inter-domain similarity calculation unit 23 removes the elements belonging to “P” and “C” from the set “NA” and adds the elements of “P” and “C” to the set “W”. After the addition, the process returns to step S102. As described above, in step S102, it is determined whether or not the set “W” of the domains whose similarity is being calculated is an empty set (φ), and it is determined that the set “W” is an empty set. Repeats the processing of steps S103 to S106, and when the set "W" becomes an empty set, the similarity calculation processing ends.

By performing the similarity calculation processing as described above, the inter-domain similarity calculation unit 23 qualitatively follows in order to reach the target domain in the hierarchical structure as shown in FIG. 8B. The similarity between the target domain and each source domain can be determined according to the number of links, that is, based on the hierarchical distance.

Next, a case where the structural relationship between domains according to the “network structure” is used in the second embodiment will be described.

FIG. 11 is a diagram for explaining inter-domain structural relationship information when a network structure is used for the structural relationship between domains. 11A shows the inter-domain structural relationship information 214 as an example, and FIG. 11B shows the inter-domain structural relationship information held by the inter-domain structural relationship information 214 of FIG. 11A. The relationships are shown in network structure 215.

The inter-domain structural relationship information 214 of FIG. 11A is an example of data stored in the inter-domain structural relationship information DB 21, and specifically, for each domain (domain name 2141) of the source domain, The combination with the adjacent domain (neighboring domain 2142) is retained. Here, an arbitrary number of domains can be set in the proximity domain, and there may be no proximity domain. For example, in the case of FIG. 11A, the proximity domain of the domain “Factory D” is one of “Factory F”, but the proximity domain of the domain “Line C” is “Worker A, Worker B, Factory D”. , Line E” are set.

Then, as a result of setting the combination with the adjacent domain 2142 for each domain as shown in FIG. 11A, the inter-domain structural relationship information 214 shows the inter-domain structural relationship as in the network structure 215 of FIG. 11B. It means that the relationship can be maintained. Note that the domain information 12 of the identification target data 10 is also input with the domain and the adjacent domain, as in the inter-domain structural relationship information 214.

FIG. 12 is a diagram illustrating an example of a GUI used for inputting inter-domain structural relationship information according to a network structure. The new domain registration screen 216 shown in FIG. 12 is a GUI for newly registering a domain in the inter-domain structural relationship information DB 21. The specific registration method is as follows. First, the user inputs the name of the new domain to be newly registered in the text box 2161, then selects the adjacent domain of the new domain from the combo box 2162, and clicks the add button 2163. click. If the user inputs the proximity domain by mistake and wants to clear the input, the clear button 2164 is clicked. Then, after adding all the adjacent domains to the new domain, by clicking the register button 2167, the input content regarding the new domain is added to the inter-domain structural relationship information 214 and registered in the inter-domain structural relationship information DB 21. .. If the user wants to cancel the registration of the new domain on the new domain registration screen 216, clicking the return button 2166 returns to the page before the transition.

The network structure display button 2165 is a button for displaying the structural relationship held by the inter-domain structural relationship information 214 registered in the inter-domain structural relationship information DB 21 so far in the network structure format. When the structure display button 2165 is clicked, the current network structure between domains is displayed on a predetermined screen, as in the network structure 215 of FIG. 11(B). By enabling browsing in such a display format on the new domain registration screen 216, the convenience of the user is improved in the same manner as the new domain registration screen 142 described in FIG. 4 and the new domain registration screen 213 described in FIG. 9. Can be improved.

Then, the processing procedure of the similarity calculation processing when the network structure is used for the structural relationship between the domains is almost the same as the processing flow of FIG. 10 described above. Supplementally, in the processing flow shown in FIG. 10, the “proximity domain” may be used instead of the “parent domain” and the “child domain”. By performing such a similarity calculation process, the inter-domain similarity calculation unit 23 qualitatively determines the number of links to reach the target domain on the network structure as shown in FIG. 11B. The similarity between the target domain and each source domain can be calculated according to the number, that is, based on the distance on the network.

As described above, in the behavior recognition system 2 according to the second embodiment, the structural relationship between domains is used (reflected), and the inter-domain structural relationship information 211, 214 is added to the inter-domain structural relationship information 211. Can be simply described, and the inter-domain similarity calculation unit 23 can easily calculate the similarity between arbitrary domains by the similarity calculation process.

Then, in the behavior recognition system 2 according to the second embodiment, after the similarity calculation processing is performed, the data conversion processing by the data conversion unit 15 and the identification unit 17 are performed as in the first embodiment. As a result of performing the action identification process by, the identification result 19 of the action identification is output.

From the above, according to the behavior recognition system 2 according to the second embodiment, it is possible to easily describe the relationship between the domains by utilizing the structural relationship between the domains, and By making it possible to easily calculate the degree of similarity between domains, the inconvenience of input by the user is reduced, and the labeled data of the target domain (identification target data 10) is provided as in the first embodiment. Even if there is only a small amount of data and data of various source domains (for example, the sensor data 161 for each domain shown in FIG. 5) exists, the source domains used for the data conversion process are related to each other. By deciding on the basis of sex, it is possible to realize highly accurate action recognition.

(3) Third Embodiment As described above, according to the action recognition system 2 according to the second embodiment, the structural relationship between domains is utilized to simplify the relationship between domains. Can be described, and the degree of similarity between arbitrary domains can be easily calculated, so that the inconvenience in the first embodiment can be reduced. However, in the second embodiment as described above, although the amount is small, each time the user tries to additionally register information regarding a new domain in the inter-domain relationship information DB 14, the user can establish the relationship between the domains. I had to enter it manually.

Here, with the development of natural language processing technology in recent years, various words can be embedded in a high-dimensional vector space (referred to as a word embedding space), and the relationship between words can be calculated in the high-dimensional space. It has become. At this time, the relationship between the words can be indicated by the distance in the word embedding space, for example.

Therefore, in the third embodiment, the technique of calculating the relationship between words by word embedding as described above is utilized for the calculation of the index indicating the relationship such as the degree of similarity between domains, thereby inputting manually. A highly-relevant source domain is extracted without any intervention, and a highly accurate action identification is realized using the extracted source domain. The action recognition system 3 according to the third embodiment will be described below in detail.

FIG. 13: is a block diagram which shows the structural example of the action recognition system which concerns on the 3rd Embodiment of this invention. The behavior recognition system 3 shown in FIG. 13 differs from the behavior recognition system 1 shown in FIG. 1 in terms of the functional configuration, in that a domain word embedding information DB 31 is provided instead of the inter-domain relationship information DB 14, and the inter-domain similarity information is similar. An inter-domain similarity calculation unit 33 is provided instead of the degree calculation unit 13. Although details will be described later, the inter-domain similarity calculation unit 33 determines the domain (target domain) of the identification target data 10 and the domain word embedding information based on the word embedding vector of each domain name stored in the domain word embedding information DB 31. The degree of similarity with each domain (source domain) whose information is stored in the DB 31 is calculated. Further, the inter-domain similarity calculation unit 33 has a function of calculating a word embedding vector of a domain name.

The domain word embedding information DB 31 stores information indicating a vector (domain word embedding vector) in the word embedding space of each domain name of the source domain.

FIG. 14 is a diagram showing an example of domain word embedding information. The domain word embedding information 311 shown in FIG. 14 is an example of data stored in the domain word embedding information DB 31 shown in FIG. 13, and for each domain of the source domain, the domain name 3111 and the domain name embedding space The combination with the vector (word embedding vector 3112) when projected onto In the case of FIG. 14, the word embedding vector 3112 of each domain is represented by “θ _i (i=1, 2,... )”. Note that the word embedding vector 3112 is not particularly limited in the method of converting the domain name into a vector expression, and an existing method such as word2Vec, GloVe, or fastText may be used, or a work procedure manual in a factory or A method that more easily reflects the context information, such as using a model such as Skip-gram for the related material, may be adopted. The registration processing of the domain word embedding information 311 may be performed by, for example, the inter-domain similarity calculation unit 13 having a function of calculating a word embedding vector of a domain name, or the action recognition system 3 has a similar calculation function. Other processing units (not shown) may be provided for processing.

Then, as the similarity calculation processing in the third embodiment, the inter-domain similarity calculation unit 33 uses the domain word embedding information 311 described above to embed the domain (target domain) of the identification target data 10 and the domain word embedding. The degree of similarity with each domain (source domain) in which information is stored in the information DB 31 is calculated.

The similarity calculation processing in the third embodiment will be described in detail. First, the inter-domain similarity calculation unit 33 reads the domain information 12 of the identification target data 10 and sets the domain name of the domain (target domain) to the domain. The word embedding vector “θ” of the target domain is acquired by projecting onto the word embedding space by the same method as used when constructing the word embedding information DB 31.

Next, the inter-domain similarity calculation unit 13 obtains each word (source domain) obtained by referring to the word embedding vector “θ” of the target domain and the domain word embedding information 311 stored in the domain word embedding information DB 31. For the word embedding vector “θ _i ”of, the distance (for example, Euclidean distance or cosine distance) in the word embedding space is calculated.

Then, the inter-domain similarity calculation unit 13 calculates the similarity between the target domain and each source domain based on the distance in the word embedding space obtained by the above calculation. Specifically, for example, the reciprocal of the distance in the word embedding space can be represented as the similarity, and in this case, the closer the distance in the word embedding space is between the domains, the higher the similarity is calculated.

As described above, in the action recognition system 3 according to the third exemplary embodiment, by using the technique of calculating the relationship between words by word embedding, the domain name in the word embedding space is calculated for each domain of the source domain. The word embedding vector can be calculated and stored in the domain word embedding information DB 31 as the word embedding vector of the source domain. Further, in the behavior recognition system 3, in the similarity calculation process, the distance between the target domain and the source domain is calculated using the word embedding vector of each domain, and the distance is used as an index indicating the similarity relationship between the domains. By doing so, a highly relevant source domain can be extracted. And these processes do not require manual input.

Then, in the action recognition system 3 according to the third exemplary embodiment, after the above-described similarity calculation processing is performed, the data conversion unit 15 is performed as in the first exemplary embodiment and the second exemplary embodiment. As a result of the data conversion processing by the above and the action identification processing by the identification unit 17, the identification result 19 of the action identification is output.

From the above, according to the action recognition system 3 according to the third embodiment, it is possible to automatically describe the relationship between domains by using the technique of calculating the relationship between words by embedding words. Also, by making it possible to easily calculate the degree of similarity between arbitrary domains, it is possible to appropriately determine the degree of similarity between domains without requiring user input and without depending on the user's judgment. .. Thus, the convenience of the user can be improved and the processing speed of the entire system can be improved. Furthermore, according to the action recognition system 3 according to the third exemplary embodiment, similarly to the first exemplary embodiment, the labeled data (identification target data 10) of the target domain is small and various sources are included. Even in the situation where domain data (for example, sensor data 161 for each domain shown in FIG. 5) exists, the accuracy can be improved by determining the source domain used for the data conversion process based on the relationship between the domains. It is possible to realize high behavior recognition.

Although the present invention has been described in detail above with reference to the accompanying drawings, the present invention is not limited to such specific configurations, and various modifications and equivalents within the spirit of the appended claims The configuration is included. For example, the above-described embodiments have been described in detail for the purpose of explaining the present invention in an easy-to-understand manner, and are not necessarily limited to those having all the configurations described. Further, a part of the configuration of one embodiment can be replaced with the configuration of another embodiment, and the configuration of one embodiment can be added with the configuration of another embodiment. .. Further, it is possible to add, delete, and replace other configurations with respect to a part of the configurations of the respective embodiments.

Further, each of the above-mentioned configurations, functions, processing units, processing means, etc. may be realized in hardware by designing a part or all of them with, for example, an integrated circuit. Further, each of the above-described configurations, functions, and the like may be realized by hardware when the processor executes a program that realizes each function. Information such as programs, tables, and files that realize each function is stored in a storage device (memory), a storage device such as a hard disk or SSD (Solid State Drive), or non-temporary such as an IC card, an SD card, or a DVD. It can be stored in a recording medium.

Further, in the drawings, control lines and information lines are shown as being considered necessary for explanation, and not all control lines and information lines necessary for mounting on a product are shown. In practice, it may be considered that almost all configurations are connected to each other.

1,2,3 Action recognition system 10 Identification target data 11 Input sensor data 12 Domain information 13,23,33 Inter-domain similarity calculation unit 14 Inter-domain relationship information DB
15 Data converter 16 Sensor data DB by domain
17 Discriminator 18 Domain discriminator DB
19 Identification result 21 Inter-domain structural relationship information DB
31 Domain word embedded information DB
100 computer 101 input device 102 central processing unit 103 main storage device 104 auxiliary storage device 105 output device 106 bus 141 inter-domain relationship information 142 new domain registration screen 161 domain-specific sensor data 181 discriminator parameter information 211 inter-domain structural relationship information 212 Hierarchical structure 213 New domain registration screen 214 Inter-domain structural relationship information 215 Network structure 216 New domain registration screen 311 Domain word embedding information

Claims (15)

An action recognition system for identifying an action type corresponding to the sensor data, using the sensor data collected in the target domain as an identification target,
A first storage unit that stores first information regarding a relationship between a plurality of domains for a plurality of domains for which sensor data has been collected in the past;
A second storage unit that stores second information in which the action type corresponding to the past sensor data and the domain in which the sensor data is collected are associated with each other;
A third storage unit that stores third information including a parameter in a discriminator for each domain learned so as to identify an action type corresponding to sensor data using data of each domain of the plurality of domains;
Inter-domain similarity calculating the similarity between the target domain and the plurality of domains based on the first information stored in the first storage unit, using the information about the target domain as input A calculator,
Based on the similarity between the domains calculated by the inter-domain similarity calculation unit, reference information is read from the second information stored in the second storage unit, and the reference information is read. Using a data conversion unit that performs a predetermined data conversion on the sensor data of the identification target,
By using the sensor data of the identification target that has been data-converted by the data conversion unit and the parameter of the discriminator included in the third information stored in the third storage unit, the sensor data of the identification target is obtained. An identification unit that identifies the corresponding action type,
An action recognition system comprising: The behavior recognition system according to claim 1, wherein the first information stored in the first storage unit stores the degree of similarity between domains in the plurality of domains. The relationship between the plurality of domains is expressed based on a predetermined structure,
The first information stored in the first storage unit holds configuration information of the predetermined structure that expresses a relationship between the domains for the plurality of domains,
The inter-domain similarity calculation unit calculates a similarity between the domains by using a distance between the target domain and the plurality of domains, which is calculated based on the information about the target domain and the first information. It calculates, The action recognition system of Claim 1 characterized by the above-mentioned. The predetermined structure is a hierarchical structure,
The first information stored in the first storage unit holds a combination of its own domain and a parent domain for each of the plurality of domains,
The inter-domain similarity calculation unit may calculate the similarity between the domains using a hierarchical distance of the hierarchical structure calculated based on the information about the target domain and the first information. The action recognition system according to claim 3, which is characterized in that. The predetermined structure is a network structure,
The first information stored in the first storage unit holds a combination of its own domain and a neighboring domain for each of the plurality of domains,
The inter-domain similarity calculation unit may calculate the similarity between the domains using a distance on the network of the network structure calculated based on the information about the target domain and the first information. The action recognition system according to claim 3, which is characterized in that. The relationship between the plurality of domains is expressed as a relationship in a word embedding space in which each domain name is embedded,
The first information stored in the first storage unit holds a vector when the domain name of each domain is embedded in the word embedding space for the plurality of domains,
The inter-domain similarity calculation unit is calculated based on the vector when the domain name of the target domain is embedded in the word embedding space and the vector for the plurality of domains held in the first information. The action recognition system according to claim 1, wherein the similarity between the domains is calculated using a distance in the word embedding space. The data conversion unit performs identity mapping on the sensor data to be identified to identify an appropriate domain from among the learned discriminators for each domain based on the similarity between the domains. The action recognition system according to claim 1, having a function of selecting a vessel. The data conversion unit selects a matching source domain from among the plurality of domains in which information is stored in the first storage unit based on the similarity between the domains, and corresponds to the source domain. 2. The domain information matching is performed by reading the second information as the reference information and using the reference information to match the sensor data of the identification target with the source domain as a matching destination. Behavior recognition system. The similarity between the domains calculated by the inter-domain similarity calculation unit from the second information stored in the second storage unit before the action class is identified by the classifier. The learning data of the discriminator based on the degree, and a discriminator learning unit that learns the discriminator based on the learning data is newly provided in place of the third storage unit,
The identification unit uses the sensor data of the identification target that has been data-converted by the data conversion unit and the parameter of the identification device that has been learned by the identifier learning unit, and corresponds to the sensor data of the identification target. The action recognition system according to claim 1, wherein the action type to be performed is identified. The input screen for registering the first information, and an input/output unit for providing an output screen for displaying an identification result of the action type identified by the identification unit are further included. Action recognition system described in. The action recognition system according to claim 10, wherein the input/output unit adds the reliability of the identification result to the identification result and displays the identification result on the output screen. A behavior recognition method for identifying a behavior type corresponding to the sensor data, using the sensor data collected in the target domain as an identification target,
Between the target domain and the plurality of domains, based on the first information regarding the relationship between the plurality of domains for the plurality of domains for which sensor data was collected in the past, using the information about the target domain as an input. Inter-domain similarity calculation step of calculating the similarity of,
Second information that associates the action type corresponding to the past sensor data with the domain in which the sensor data is collected, based on the similarity between the domains calculated in the inter-domain similarity calculation step. Of the above, a data conversion step of reading reference information, and using the reference information, performing predetermined data conversion on the sensor data to be identified,
Parameters in the discriminator for each domain learned to identify the action type corresponding to the sensor data using the sensor data of the identification target that has been data-converted in the data conversion step and the data for each domain of the plurality of domains And an identification step of identifying the action type corresponding to the sensor data of the identification target,
An action recognition method comprising: The relationship between the plurality of domains is expressed based on a predetermined structure,
The first information holds configuration information of the predetermined structure that expresses a relationship between the domains for the plurality of domains,
In the inter-domain similarity calculation step, the similarity between the domains is calculated using the distance between the target domain and the plurality of domains, which is calculated based on the information about the target domain and the first information. The action recognition method according to claim 12, wherein the action recognition method is performed. The relationship between the plurality of domains is expressed as a relationship in a word embedding space in which each domain name is embedded,
The first information holds a vector when the domain name of each domain is embedded in the word embedding space for the plurality of domains,
In the inter-domain similarity calculation step, it is calculated based on a vector when the domain name of the target domain is embedded in the word embedding space and the vector for the plurality of domains held in the first information. The behavior recognition method according to claim 12, wherein the similarity between the domains is calculated using a distance in the word embedding space. Prior to the identifying step, the learning data of the discriminator is selected from the second information based on the similarity between the domains calculated in the interdomain similarity calculating step, and the learning data is selected. Further comprising a discriminator learning step of learning the discriminator based on the data,
In the identification step, the sensor data of the identification target, which has been data-converted in the data conversion step, and the parameter of the discriminator after learning in the discriminator learning step are used to correspond to the sensor data of the discrimination target. The action recognition method according to claim 12, wherein the action type to be performed is identified.

Images