JP7259322B2 - Information processing device, learning model generation program, and learning model generation method - Google Patents

Description

The present invention relates to an information processing device, a learning model generation program, and a learning model generation method.

In recent years, the use of artificial intelligence (AI) has become active. Algorithms have also been developed to appropriately handle various tasks.

Human movements in work are complicated, and it is difficult for an observer to distinguish the classification of work and action states (in other words, "labeling"). Therefore, labeling that considers human movements and high-quality training data are created.

JP 2018-72029 A JP 2012-3494 A JP 2016-149136 A

T. Mori et al, "Online Recognition and Segmentation for Time-Series Motion with HMM and Conceptual Relation of Actions", In IROS2005., August 2005

However, when the action or motion is tagged by an observer (in other words, "a person who tags") is a binary value indicating whether the action is the relevant action or not, the feature value of the acquired data Even if there is a difference between the training data and the tagging result, the difference may not be reflected in the training data.

One aspect aims at enabling learning for behavior analysis with less teacher data.

The information processing apparatus includes an acquisition unit that acquires sensor data, and weighting information indicating whether the motion of an observation target object is closer before or after the transition to the sensor data acquired by the acquisition unit. and a weighting unit that records by receiving a stepwise input about the state of .

In one aspect, learning for behavior analysis is possible with less training data.

FIG. 11 is a diagram for explaining acquisition processing of teacher data in a related example; It is a block diagram which shows typically the hardware structural example of the computer in an example of embodiment. 3 is a block diagram schematically showing a functional configuration example of the computer shown in FIG. 2; FIG. 3 is a diagram illustrating details of weighting processing in the computer shown in FIG. 2; FIG. 3 is a diagram for explaining a first example of weighting processing in the computer shown in FIG. 2; FIG. 3 is a diagram for explaining a second example of weighting processing in the computer shown in FIG. 2; FIG. FIG. 3 is a diagram for explaining processing of reflecting likelihood of annotation to weighting information in the computer shown in FIG. 2 ; FIG. 3 is a diagram for explaining filtering processing of annotation certainty with respect to weighting information in the computer shown in FIG. 2 ; FIG. 3 is a diagram for explaining learning change processing according to the certainty of annotations for weighting information in the computer shown in FIG. 2 ; 3 is a flowchart for explaining weighting processing in the computer shown in FIG. 2;

An embodiment will be described below with reference to the drawings. However, the embodiments shown below are merely examples, and are not intended to exclude the application of various modifications and techniques not explicitly described in the embodiments. In other words, the present embodiment can be modified in various ways without departing from the spirit of the embodiment.

Also, each drawing does not mean that it has only the constituent elements shown in the drawing, but can include other functions and the like.

In the following figures, the same reference numerals denote the same parts, so the description thereof will be omitted.

[A] Related Example FIG. 1 is a diagram for explaining a teacher data acquisition process in a related example.

In the example shown in FIG. 1, the sensor state characterizes from action A to action B, for example, when action A and action B are tagged by an observer (see A1).

However, if the tagging is performed after the movement B is definitely confirmed (see symbol A2), the expression of the feature amount is buried, and the accuracy as teacher data is insufficient.

Then, as shown in FIG. 1, when dataset #1 and dataset #2 exist, the actual feature amount There is a risk that there will be a gap between the timing of the fluctuation of the tag and the timing of tagging.

[B] Example of Embodiment [B-1] Example of System Configuration FIG. 2 is a block diagram schematically showing an example of the hardware configuration of the computer 1 in one example of the embodiment.

The computer 1 includes a Central Processing Unit (CPU) 11, a memory 12, a display control section 13, a storage device 14, an input interface (I/F) 15, a read/write processing section 16 and a communication I/F 17.

The memory 12 is an example of a storage unit, and is illustratively a storage device including Read Only Memory (ROM) and Random Access Memory (RAM). A program such as a Basic Input/Output System (BIOS) may be written in the ROM of the memory 12 . The software programs in the memory 12 may be read and executed by the CPU 11 as appropriate. Also, the RAM of the memory 12 may be used as a primary recording memory or a working memory.

The display control unit 13 is connected to the display device 130 and controls the display device 130 . The display device 130 is a liquid crystal display, an Organic Light-Emitting Diode (OLED) display, a cathode ray tube (CRT), an electronic paper display, or the like, and displays various information for the operator or the like. The display device 130 may be combined with an input device, such as a touch panel.

The storage device 14 is illustratively a device that stores data in a readable and writable manner, and may be, for example, a Hard Disk Drive (HDD), Solid State Drive (SSD), or Storage Class Memory (SCM).

The input I/F 15 is connected to input devices such as the mouse 151, keyboard 152, and sensor group 153, and controls the input devices such as the mouse 151, keyboard 152, and sensor group 153. FIG. The mouse 151 and keyboard 152 are examples of input devices, and the operator performs various input operations via these input devices. The sensor group 153 acquires information about the motion of an observation object such as a person, an animal, a robot, or the like.

The read/write processing unit 16 is configured such that a recording medium 160 can be attached. The read/write processing unit 16 is configured to be able to read information recorded on the recording medium 160 while the recording medium 160 is attached. In this example, the recording medium 160 has portability. For example, the recording medium 160 is a flexible disk, optical disk, magnetic disk, magneto-optical disk, or semiconductor memory.

Communication I/F 17 is an interface for enabling communication with an external device.

The CPU 11 is a processing device that performs various controls and calculations, and implements various functions by executing an operating system (OS) and programs stored in the memory 12 .

A device for controlling the operation of the entire computer 1 is not limited to the CPU 11, and may be, for example, any one of MPU, DSP, ASIC, PLD, and FPGA. Also, the device for controlling the operation of the entire WMS 1 may be a combination of two or more of CPU, MPU, DSP, ASIC, PLD and FPGA. Note that MPU is an abbreviation for Micro Processing Unit, DSP is an abbreviation for Digital Signal Processor, and ASIC is an abbreviation for Application Specific Integrated Circuit. PLD is an abbreviation for Programmable Logic Device, and FPGA is an abbreviation for Field Programmable Gate Array.

FIG. 3 is a block diagram schematically showing a functional configuration example of the computer 1 shown in FIG.

Calculator 1 functions as a feature amount calculator 111, a tag information weighting unit 112, a feature amount output unit 113, and a learning model generator 114, as shown in FIG.

Programs for realizing the functions of the feature amount calculation unit 111, the tag information weighting unit 112, the feature amount output unit 113, and the learning model generation unit 114 are provided in a form recorded in the above-described recording medium 160, for example. be done. Then, the computer reads the program from the recording medium 160 through the read/write processing unit 16, transfers it to an internal storage device or an external storage device, and stores it for use. Alternatively, the program may be recorded in a storage device (recording medium) such as a magnetic disk, an optical disk, or a magneto-optical disk, and provided to the computer from the storage device via a communication path.

When realizing the functions of the feature amount calculation unit 111, the tag information weighting unit 112, the feature amount output unit 113, and the learning model generation unit 114, programs stored in the internal storage device are executed by the microprocessor of the computer. . At this time, the computer may read and execute the program recorded on the recording medium 160 . Note that in this embodiment, the internal storage device is the memory 12 and the microprocessor is the CPU 11 .

As shown in FIG. 3 , sensor group 153 functions as sensor input section 1530 . The sensor input unit 1530 acquires information about the motion of an object to be observed, such as a person, an animal, or a robot, and inputs it to the computer 1 via an interface (I/F) as teacher data.

As shown in FIG. 3, the input device such as mouse 151 and keyboard 152 functions as tag information input section 1520 . The tag information input unit 1520 accepts input of tag information (also referred to as “weighting information”) from the observer of the observation object, and inputs the accepted tag information to the computer 1 via the I/F.

The feature amount calculation unit 111 calculates feature amounts from the teacher data input from the sensor input unit 1530 . That is, the feature amount calculation unit 111 functions as an acquisition unit that acquires teacher data.

The tag information weighting section 112 weights the feature amount calculated by the feature amount calculating section 111 based on the tag information input from the tag information input section 1520 . That is, the tag information weighting unit 112 functions as a weighting unit that records weighting information indicating which of the motions of the observed object is closer to before or after the transition in the acquired teacher data.

The feature quantity output section 113 outputs the feature quantity weighted by the tag information weighting section 112 .

The learning model generation unit 114 generates a learning model based on the feature quantity output by the feature quantity output unit 113 .

FIG. 4 is a diagram for explaining the details of weighting processing in computer 1 shown in FIG.

As shown in FIG. 4, feature amounts are acquired as teacher data in time series based on raw data about acceleration and atmospheric pressure from the sensor group 153 (see symbol B1).

The acquired teacher data are calculated as feature quantities #1 and #2 by filter feature quantity calculation (see symbol B2). Calculation of the filter feature amount is performed, for example, by calculating an average value or maximum value in a certain interval. Three or more features may be calculated depending on the filter design.

The observer observes the behavior of the subject (in other words, the “object to be observed”) and tags the behavior (see symbol B3). In the example shown, action A is tagged J1 and action B is tagged J2. When transitioning from J1 to J2, the observer may use slide bar 131 to analogically tag whether the action is from J1 or from J2.

That is, the tag information weighting unit 112 shown in FIG. 3 functions as a weighting unit that acquires weighting information by receiving stepwise input regarding the state before and after the transition using the slide bar 131 displayed on the display device 130. .

In the example indicated by reference numeral B3 in FIG. 4, the observer moves the slide bar 131 left and right while hesitating from J1 to J2, and finally inputs that it has moved to J2.

Transition feature quantities #1 and #2 are generated based on the feature quantities #1 and #2 calculated in code B2 and the tag information acquired in code B3 (see code B4). By analogically weighting the transition state for each feature amount in this way, the feature amount from J1 to J2 is recorded as a tag, and the feature amount can be strictly reflected in the teacher data.

FIG. 5 is a diagram for explaining a first example of weighting processing in computer 1 shown in FIG. FIG. 6 is a diagram for explaining a second example of weighting processing in computer 1 shown in FIG.

As shown in FIGS. 5 and 6, tagging using the slide bar 131 is performed in an analog tagging region (see symbol C1 in FIGS. 5 and 6) by the observer, so that It is possible to reflect the feature quantity that is used as training data.

であり、正解ラベルが

であるとき、下記の数式１で表わされる。 The cost function for learning is

and the correct label is

is expressed by Equation 1 below.

ただし、

は、本実施形態の一例において計算できるサンプルの重み付けである。

は、学習対象の特徴量重みである。

は、ロス関数であり、例えはhinge-loss関数

である。ここで、

である。

however,

is the sample weighting that can be calculated in one example of this embodiment.

is the feature weight to be learned.

is the loss function, for example the hinge-loss function

is. here,

is.

であり、各々の時刻サンプルに対して全て等化となり、サンプルへのアノテーションの確からしさがモデル化できない。 where all the data

, and all are equalized for each time sample, and the likelihood of annotation to the sample cannot be modeled.

FIG. 7 is a diagram for explaining the process of reflecting the probability of annotation to the weighting information in the computer 1 shown in FIG.

をスライドバー１３１の入力値として使用する。 To model the likelihood of annotations to samples, as shown in Figure 7,

is used as the input value for the slide bar 131 .

This makes it possible to reflect the likelihood of annotations to samples in tagging.

FIG. 8 is a diagram for explaining the filtering processing of annotation certainty for weighting information in computer 1 shown in FIG.

において、前後のサンプルの変動が大きい区間を無視する。すなわち、

は、以下の数式２によって表わされる。

In the example shown in Figure 8, the sample weighting

, we ignore sections where the fluctuations between the preceding and succeeding samples are large. i.e.

is represented by Equation 2 below.

That is, the tag information weighting unit 112 shown in FIG. 3 functions as an example of a weighting unit that filters input weighting information based on likelihood.

As a result, it is possible to exclude uncertain feature amounts that cause the observer to hesitate in making a decision, so that learning is possible only for certain information.

FIG. 9 is a diagram for explaining learning change processing according to the certainty of annotations for weighting information in computer 1 shown in FIG.

In the example shown in FIG. 9, by learning easy-to-determine feature amounts in the early stage of learning and learning difficult feature amounts in the latter stage of learning, learning including more complex feature amounts can be accelerated.

を小さい

に設定（別言すれば、「特徴量を判断しやすく設定」）して、後半には大きい

に設定（別言すれば、「特徴量を判断しにくく設定」）して学習を繰り返す。 That is, initially as shown in FIG.

the small

(in other words, "set features so that they are easy to judge"), and in the latter half

(in other words, “set to make it difficult to judge the feature quantity”) and repeat the learning.

The first learning is represented by Equation 3 below.

The second learning is represented by Equation 4 below.

In other words, the tag information weighting unit 112 shown in FIG. 3 functions as an example of a weighting unit that changes learning based on the likelihood of inputting weighting information.

[B-2] Operation Example The weighting process in the computer 1 shown in FIG. 2 will be described according to the flowchart (steps S1 to S10) shown in FIG.

The feature amount calculation unit 111 receives input of information about the action A indicated by the tag information J1 of the observed object from the sensor input unit 1530, and calculates a feature amount (step S1).

The tag information weighting unit 112 adds weights "J1:1.0, J2:0.0" indicating tagging as the tag information J1 to the feature amount calculated by the feature amount calculation unit 111 (step S2).

The tag information weighting unit 112 determines whether the action A indicated by the tag information J1 has ended based on the feature amount calculated by the feature amount calculation unit 111 and the input from the tag information input unit 1520 (step S3 ).

If the operation A indicated by the tag information J1 has not ended (see No route in step S3), the process returns to step S2.

On the other hand, if the action A indicated by the tag information J1 has been completed (see Yes route of step S3), the process proceeds to step S4. Then, the tag information weighting section 112 receives input of information indicating that the slide bar 131 has been slid from the J1 side to the J2 side from the tag information input section 1520 (step S4).

The tag information weighting unit 112 determines whether the operation B indicated by the tag information J2 has started based on the feature amount calculated by the feature amount calculation unit 111 and the input from the tag information input unit 1520 (step S5 ).

If the operation B indicated by the tag information J2 has not started (see No route in step S5), the process proceeds to step S6. The tag information weighting unit 112 receives from the tag information input unit 1520 input of information indicating that the slide bar 131 has stopped sliding or information indicating that the slide bar 131 has been slid from the J2 side to the J1 side. Accept (step S6). Then, the process returns to step S5.

On the other hand, if the operation B indicated by the tag information J2 has started (see Yes route in step S5), the process proceeds to step S7. Then, the tag information weighting unit 112 receives input of information indicating that the slide bar 131 has been slid from the J1 side to the J2 side from the tag information input unit 1520 (step S7).

The tag information weighting unit 112 determines whether the start of the action B indicated by the tag information J2 has been confirmed based on the feature amount calculated by the feature amount calculation unit 111 and the input from the tag information input unit 1520 ( step S8).

If the start of the action B indicated by the tag information J2 has not been determined (see No route in step S8), the process returns to step S7.

On the other hand, when the start of the action B indicated by the tag information J2 has been confirmed (see Yes route in step S8), the tag information weighting unit 112 receives from the tag information input unit 1520 that the slide bar 131 has been confirmed on the J2 side. Input of information indicating that is accepted (step S9).

The tag information weighting unit 112 adds weights "J1:0.0, J2:1.0" indicating tagging as the tag information J2 to the feature amount calculated by the feature amount calculation unit 111 (step S10). The weighting process then ends.

[B-3] Effects According to the computer 1 in the example of the embodiment described above, for example, the following effects can be obtained.

The feature amount calculation unit 111 acquires teacher data. The tag information weighting unit 112 assigns weighting information to the acquired teacher data, which indicates whether the motion of the observation object is closer before or after the transition, instead of tagging the acquired teacher data with a binary value such as motion A or motion B. Record.

As a result, learning for behavior analysis can be performed with a small amount of teacher data.

The tag information weighting unit 112 acquires weighting information by accepting stepwise input regarding the state before and after the transition using the slide bar 131 displayed on the display device 130 .

This allows fine-grained analog tagging of motion transitions.

The tag information weighting unit 112 filters input weighting information based on likelihood.

As a result, it is possible to exclude uncertain feature amounts that cause the observer to hesitate in making a decision, so that learning is possible only for certain information.

The tag information weighting unit 112 changes the learning based on the likelihood of inputting the weighting information.

As a result, it is possible to accelerate learning including more complex feature values by implementing curriculum learning in which easy-to-determine feature values are learned in the early stage of learning, and difficult feature values are learned in the latter stage of learning.

[C] Others The technology disclosed herein is not limited to the above-described embodiments, and various modifications can be made without departing from the spirit of the embodiments. Each configuration and each process of the present embodiment can be selected as necessary, or may be combined as appropriate.

In the above-described example of the embodiment, the observer inputs analog weighting using the slide bar 131, but the present invention is not limited to this. The analog weighting input by the observer may be performed, for example, by inputting numerical values into input boxes displayed on the display device 130 . Further, the input of analog weighting by the observer may be performed by multistage hardware switches such as dials and knobs connected to the input I/F 15 .

[D] Supplementary Notes Regarding the above embodiment, the following supplementary notes are disclosed.

(Appendix 1)
an acquisition unit that acquires teacher data;
a weighting unit that records weighting information indicating whether the motion of the observation target is closer to before or after the transition for the teacher data acquired by the acquisition unit;
An information processing device.

(Appendix 2)
The weighting unit acquires the weighting information by accepting stepwise input regarding the state before and after the transition using a slide bar displayed on a display device.
The information processing device according to appendix 1.

(Appendix 3)
The weighting unit filters the input of the weighting information based on likelihood.
The information processing device according to appendix 1 or 2.

(Appendix 4)
The weighting unit changes the learning based on the likelihood of inputting the weighting information.
The information processing device according to appendix 1 or 2.

(Appendix 5)
to the computer,
get teacher data,
recording weighting information indicating whether the movement of the observed object is closer to before or after the transition in the acquired teacher data;
A learning model generation program that executes processing.

(Appendix 6)
Acquiring the weighting information by accepting stepwise input about the state before and after the transition using a slide bar displayed on the display device;
The learning model generation program according to appendix 5, causing the computer to execute processing.

(Appendix 7)
filtering the input of the weighting information by probability;
7. The learning model generation program according to appendix 5 or 6, causing the computer to execute processing.

(Appendix 8)
changing the learning based on the likelihood of inputting the weighting information;
7. The learning model generation program according to appendix 5 or 6, causing the computer to execute processing.

(Appendix 9)
get teacher data,
recording weighting information indicating whether the movement of the observed object is closer to before or after the transition in the acquired teacher data;
Learning model generation method.

(Appendix 10)
Acquiring the weighting information by accepting stepwise input about the state before and after the transition using a slide bar displayed on the display device;
The learning model generation method according to appendix 9.

(Appendix 11)
filtering the input of the weighting information by probability;
The learning model generation method according to appendix 9 or 10.

(Appendix 12)
changing the learning based on the likelihood of inputting the weighting information;
11. The learning model generation method according to appendix 9 or 10, wherein the processing is executed by the computer.

1: computer 11: CPU
111: feature amount calculation unit 112: tag information weighting unit 113: feature amount output unit 114: learning model generation unit 12: memory 13: display control unit 130: display device 131: slide bar 14: storage device 15: input I/F
151: Mouse 152: Keyboard 153: Sensor group 1520: Tag information input unit 1530: Sensor input unit 16: Read/write processing unit 160: Recording medium 17: Communication I/F

Claims (6)

an acquisition unit that acquires sensor data;
For the sensor data acquired by the acquisition unit, weighting information indicating whether the motion of the observation target is closer to before or after the transition is recorded by accepting stepwise input regarding the state before and after the transition. a weighting unit;
An information processing device. The weighting unit receives the stepwise input by a slide bar displayed on a display device ;
The information processing device according to claim 1 . The weighting unit filters the input of the weighting information based on likelihood.
The information processing apparatus according to claim 1 or 2. The weighting unit changes the learning based on the likelihood of inputting the weighting information.
The information processing apparatus according to claim 1 or 2. to the computer,
get sensor data,
For the acquired sensor data, weighting information indicating whether the motion of the observation target is closer before or after the transition is recorded by receiving stepwise input regarding the state before and after the transition.
A learning model generation program that executes processing. get sensor data,
For the acquired sensor data, weighting information indicating whether the motion of the observation target is closer before or after the transition is recorded by receiving stepwise input regarding the state before and after the transition.
Learning model generation method.

Images