JP2012238072A - Action evaluation device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an action estimation device capable of improving accuracy of an evaluation of a movement or an action that a subject has performed, by a simple and inexpensive configuration including only an acceleration sensor as detection means for the subject.SOLUTION: The action evaluation device includes the acceleration sensor and a memory for storing outputs of the acceleration sensor. A first neural network has preliminarily learned how to recognize a movement or an action that a subject has performed and, after the learning, recognizes the movement or the action that the subject has performed, in real time on the basis of an output of the acceleration sensor. A second neural network, based on a command, has read the output of the acceleration sensor corresponding to the movement or the action recognized through the first neural network and has learned them as examples and, after the learning, outputs an approximation degree representing the degree to which the movement or the action recognized through the first neural network is approximate to the examples.

Description

    The present invention relates to a behavior evaluation apparatus, and more particularly, to a behavior evaluation apparatus that recognizes and evaluates the motion or behavior of a subject using an acceleration sensor.

  Here, the “subject” typically refers to a human but may be an animal.

  The “motion” refers to the movement of the subject. “Behavior” refers to movements and actions that include conscious things. In this specification, for the sake of simplicity, an operation and an action are collectively referred to as an “operation” as appropriate.

  The behavior evaluation apparatus of the present invention can typically be used by being incorporated into a pedometer / activity meter, a mobile phone, a mobile computer, or the like.

  Conventionally, for example, Patent Document 1 (Japanese Patent Application Laid-Open No. 10-113343) discloses a measurement unit that observes a change in state associated with a motion or action of a subject, a feature amount extraction unit that extracts a feature amount in the observation result, Recognition for recognizing the motion or action of the subject from the storage means for storing the feature quantity regarding the action or action to be recognized by the recognition device, the feature quantity extracted from the observation result, and the stored feature quantity A system comprising the means is disclosed. Furthermore, this document describes standard movements (such as walking movements before being damaged) and measurement actions (currently being damaged) in order to be used for rehabilitation in the event of a foot injury due to an accident. It is described that the difference in the walking motion of the subject is visually shown as a difference in the frequency spectrum of the observed waveform. In addition, it is best to extract and register the features of the standard motion before the accident of the subjects in the standard motion feature database, but if there is not, the average motion of many subjects It is stated that the feature amount obtained from the above may be used as a standard operation.

JP-A-10-113343

  However, it is usually not possible for a subject to register a standard motion in the standard motion feature database assuming that an accident will occur. Moreover, when using the feature quantity calculated | required from the average motion of many test subjects as standard motion, there exists a problem that the precision of evaluation falls.

  Accordingly, an object of the present invention is to provide an action evaluation apparatus that can increase the accuracy of evaluation of actions or actions performed by a subject with a simple and inexpensive configuration that includes only an acceleration sensor as a detection means for the subject. It is in.

In order to solve the above problems, the behavior evaluation apparatus of the present invention provides:
An acceleration sensor to be attached to the subject;
A memory for storing the output of the acceleration sensor;
From the output of the acceleration sensor, comprising a neural network that has completed learning to recognize a model action or action,
A behavior evaluation device that recognizes the motion or behavior of the subject through the neural network from the output of the acceleration sensor attached to the subject and evaluates the degree of the recognized motion or behavior close to the example In
The neural network is
Learning that recognizes the movement or action performed by the subject based on the correspondence between the plurality of types of movement or action performed by the subject in advance and the output of the acceleration sensor attached to the subject. After learning, based on the output of the acceleration sensor, a first neural network that recognizes the movement or action of the subject in real time;
Based on the instruction, the output of the acceleration sensor corresponding to the motion or action recognized through the first neural network is read from the memory and learned as a model, and after the learning, the first neural network And a second neural network that outputs a degree of approximation representing the degree of movement or action recognized through the model.

  In the behavior evaluation apparatus according to the present invention, the first neural network has a correspondence relationship between a plurality of types of actions or behaviors performed by the subject in advance and an output of the acceleration sensor attached to the subject. Based on this, learning for recognizing the motion or action performed by the subject is completed, and after the learning, the motion or behavior performed by the subject is recognized in real time based on the output of the acceleration sensor. Based on the instruction, the second neural network reads the output of the acceleration sensor corresponding to the motion or action recognized through the first neural network from the memory and learns it as a model. Therefore, the user (refers to a person who operates this behavior evaluation apparatus. Typically, it is a subject but may be another person. The same shall apply hereinafter). For example, the second neural network can learn the movement when the subject is in good condition, for example, in sports or dance, as the example. After the learning, the second neural network outputs a degree of approximation representing the degree to which the motion or action recognized through the first neural network is close to the example. Based on this degree of approximation, the user can know the degree to which the action or action performed by the subject is close to the example.

  In this behavior evaluation apparatus, when the user inputs the instruction, the second neural network can be made to learn an appropriate model at a desired timing. Therefore, according to this behavior evaluation apparatus, it is possible to improve the accuracy of evaluation of the action or behavior performed by the subject as compared with the conventional case. Moreover, since the said behavior evaluation apparatus is provided with only the acceleration sensor as a detection means with respect to a test subject, it can be comprised simply and cheaply.

In the behavior evaluation device of one embodiment,
An instruction switch for inputting the above instructions;
When the instruction is input by the instruction switch, the output of the acceleration sensor corresponding to the action or action recognized through the first neural network immediately before the current time in the memory is searched and searched. And a control unit that reads out the output of the acceleration sensor from the memory and inputs the output to the second neural network for learning.

  In the behavior evaluation apparatus according to the embodiment, the user inputs the instruction using the instruction switch. When the instruction is input by the instruction switch, the control unit outputs the output of the acceleration sensor corresponding to the operation or action recognized through the first neural network in the memory, which is the latest from the current time. Searching is performed, and the output of the searched acceleration sensor is read from the memory and input to the second neural network for learning. Accordingly, the user can cause the second neural network to learn an appropriate model at a desired timing.

  In an action evaluation apparatus according to an embodiment, the output layer of the second neural network includes only a single element.

  In the behavior evaluation apparatus of this embodiment, since the output layer of the second neural network includes only a single element, the degree of approximation obtained from the output of the single element is determined by the subject. The degree to which the action or action is close to the above example can be clearly expressed by one numerical value.

  In the behavior evaluation apparatus according to an embodiment, the approximation degree is a numerical value.

  In the behavior evaluation apparatus according to this embodiment, since the degree of approximation is a numerical value, the user can clearly know the degree to which a certain action or behavior performed by the subject is close to the example.

  In the behavior evaluation apparatus according to one embodiment, the operation or behavior includes a golf swing.

  In the behavior evaluation apparatus according to the embodiment, the operation or behavior includes a golf swing. Therefore, the user inputs the above instruction to the second neural network, for example, the swing when the subject makes a nice shot and the swing when the golf coach takes a model shot as the above model. You can learn. As a result, at the behavioral evaluation stage, the user can see how close the subject's swing is to the swing when the subject is in good condition (when taking a nice shot), or the golf coach takes a model shot. You can know how close you are to the swing when you do.

  As is clear from the above, according to the behavior evaluation apparatus of the present invention, the accuracy of evaluation of the action or behavior performed by the subject can be improved with a simple and inexpensive configuration including only the acceleration sensor as the detection means for the subject. Can do.

FIG. 1A is a diagram showing the appearance of the behavior evaluation apparatus according to the embodiment of the present invention as viewed from the front. FIG. 1B is a diagram showing the behavior evaluation apparatus as viewed from the right side. FIG. 1C is a diagram showing the behavior evaluation apparatus as seen from the right side in a disassembled state. FIG. 1D is a diagram illustrating components included in the behavior evaluation apparatus. FIG. 1E is a diagram showing an aspect in which a person as a subject wears the behavior evaluation apparatus. It is a figure which shows the block configuration of the arithmetic unit contained in the said behavior evaluation apparatus. It is a figure which illustrates the structure of the 1st neural network contained in the said arithmetic unit. It is a figure which illustrates the structure of the 2nd neural network contained in the said arithmetic unit. It is a figure which shows a mode that the person as a test subject is performing "swing A". It is a figure which shows the output of the acceleration sensor contained in the said behavior evaluation apparatus when a test subject is performing the exercise | movement of FIG. It is a figure which illustrates the correspondence between each neuron of the output layer of the first neural network and the operating state of the person as the subject. FIG. 8A is a diagram illustrating the output of a neuron when the output layer of the second neural network is composed of a single element. FIG. 8B is a diagram illustrating the output of each neuron when the output layer of the second neural network includes two output elements. FIG. 9A is a diagram showing a schematic processing flow in the behavior evaluation apparatus. FIG. 9B is a diagram showing a processing flow of the “recording of model” stage in the behavior evaluation apparatus. FIG. 10A is a diagram showing a processing flow in the learning stage of the first neural network. FIG. 10B is a diagram showing a processing flow in the recognition stage of the first neural network. FIG. 11A is a diagram schematically illustrating the recognition operation of the first neural network. FIG. 11B is a diagram for schematically explaining the evaluation operation of the second neural network. FIG. 12A is a diagram showing a processing flow in the learning stage of the second neural network. FIG. 12B is a diagram showing a processing flow in the evaluation stage of the second neural network.

  Hereinafter, an action evaluation apparatus according to an embodiment of the present invention will be described in detail with reference to the drawings.

  As shown in FIG. 1A, a behavior evaluation apparatus 1 according to an embodiment of the present invention includes a display unit including a main body casing 7 and an LCD (liquid crystal display element) provided on a front surface portion 7A of the main body casing 7. 3, a normally open button switch 11 that functions as an instruction switch, and a slide-type power switch 42.

  As shown in FIG. 1B (corresponding to the behavioral evaluation device 1 viewed from the right side), the behavioral evaluation device 1 is placed on the rear surface portion 7B of the main body casing 7 (a person in this example). A clip 8 is provided for mounting on 90. Specifically, the clip 8 is attached to the rear surface portion 7B via a hinge 9, and is urged by a spring (not shown) so that the tip 8a of the clip 8 approaches the rear surface portion 7B. Thereby, as shown in FIG. 1E, the behavior evaluation apparatus 1 can be easily attached to the side of the waist of the subject 90 (for example, the clip 8 is hooked on a belt (not shown) worn by the subject 90). Just do it.)

  As can be seen from FIGS. 1C and 1D, the casing 7 contains an electronic circuit board 10 and a rechargeable battery 6 for supplying power to the electronic circuit board 10. ing. Power is supplied from the battery 6 to the electronic circuit board 10 according to whether the power switch 42 is turned on (ON) or off (OFF), or is not supplied.

  On the electronic circuit board 10, the wireless communication device 4, the acceleration sensor 2, and the arithmetic device 5 are mounted.

  The wireless communication device 4 is a known device that performs wireless communication with a mobile phone, a personal computer, or the like that exists in the vicinity of the behavior evaluation device 1 according to the Bluetooth standard. The wireless communication device 4 can transmit an evaluation result (described later) by the behavior evaluation device 1 to the mobile phone, personal computer, or the like.

  The acceleration sensor 2 is a well-known semiconductor type capable of detecting accelerations in the X axis, Y axis, and Z axis directions in the range of ± 2 G in this example. The acceleration sensor 2 measures and outputs accelerations in the X, Y, and Z directions with a period of 10 milliseconds. Values output from the acceleration sensor 2 are sequentially stored in a memory 54 (see FIG. 2) built in the control unit 50 described below, and held for at least 5 seconds.

  As shown in FIG. 2, the arithmetic device 5 includes a control unit 50 composed of a CPU (central processing unit) that controls the operation of the entire arithmetic device 5 and an input processing unit 51 that receives and processes the output of the acceleration sensor 2. A first neural network 52, a second neural network 55, an output processing unit 53 that receives and processes the outputs of the first neural network 52 and the second neural network 55, and temporarily stores data. And a possible memory 54.

  The input processing unit 51 performs signal processing on the output of the acceleration sensor 2 under the control of the control unit 50, and generates a plurality of types of statistical data as feature data representing the movement of the person 90. Specifically, the input processing unit 51 generates the following statistical data.

  First, the current time (corresponding to the time when the operating state of the person 90 is recognized) is set to t1, and the time 5 seconds before is set to t0. The input processing unit 51 calculates, as statistical data, average values of outputs in the X, Y, and Z directions (10-millisecond cycle values) of the acceleration sensor 2 stored in the memory 54 for 5 seconds from time t0 to t1. To do. The calculated average values are referred to as feature data X1, Y1, and Z1, respectively.

  Further, the input processing unit 51 calculates the standard deviation of the output in the X, Y, and Z directions of the acceleration sensor 2 for 5 seconds from time t0 to t1 as statistical data. The calculated standard deviations are referred to as feature data X2, Y2, and Z2, respectively.

  Further, the input processing unit 51 searches for the maximum value and the minimum value of the output in the X, Y, and Z directions of the acceleration sensor 2 for 5 seconds from the time t0 to the time t1, and calculates the maximum value and the minimum value. Each difference is calculated as statistical data. The difference between the calculated maximum value and minimum value is referred to as feature data X3, Y3, and Z3, respectively.

  In addition, the input processing unit 51 counts the number of times the output of the acceleration sensor 2 in the X, Y, and Z directions becomes 0 in 5 seconds from time t0 to time t1 (hereinafter referred to as “the number of times of zero crossing”). Find as data. The obtained zero crossing numbers are X4, Y4, and Z4, respectively.

  In this way, the input processing unit 51 generates feature data X1 to X4, Y1 to Y4, and Z1 to Z4 that are 12 types of statistical data. The generation of the feature data X1 to X4, Y1 to Y4, and Z1 to Z4 by the input processing unit 51 is performed at a cycle of 100 milliseconds. With respect to one axial direction, for example, the X-axis direction, four types of feature data X1 are input by the input processing unit 51 from one type of data (output value of the acceleration sensor 2 over 5 seconds) stored in the memory 54. ~ X4 is generated every 100 milliseconds.

In this example, as shown in FIG. 3, the first neural network 52 includes an input layer 52 i including ₁₂ input elements i ₁ , i ₂ ,..., I ₁₂ , and 7 elements m ₁ , m ₂ , ..., an intermediate layer 52m comprising _{m 7,} 7 pieces of output elements _{n 1,} n 2, ..., is composed of three layers and the output layer 52n including _{n 7.} The first neural network 52, by the learning method called inverse error propagation method (Back Propagation algorithm) In this example, the input element i _1, i 2 of the input layer 52i, _..., from the input that is input to the i _12, .., N ₇ is configured to perform learning so that a correct output can be obtained at the output elements n ₁ , n ₂ ,.

Specifically, first, an input element _{i 1,} i 2, _..., respectively weighting factor inputs to _{i 12 w j1, w j2,} ..., w j12 a (j = 1, 2, ..., is. 7) Multiplication is performed to obtain a linear sum u _j of products of these inputs and weighting factors as follows.

Assume that the output of the output elements n ₁ , n ₂ ,..., N ₇ is a sigmoid function f (u _j ) of its linear sum u _j as follows.
f (u _j ) = 1 / (1 + exp (−u _j ))

Then, the input element _{i 1,} i 2, _..., output element _n 1 when a certain set of input data is input to the _{i 12,} n 2, ..., an output of _{n 7,} and teacher data representing the correct answer Find the error between.

Output element _{n 1,} n 2, ..., as the error between the output of the _{n 7} and the teacher data is eliminated, the weight coefficient _{w j1,} w j2, _..., slide into change _{w j12.} The output of the intermediate layer that caused the error is also adjusted sequentially.

When learning is performed in this manner, only one output element among the output elements n ₁ , n ₂ ,..., N ₇ of the output layer 52n reacts to a certain input data set (the value is 1). The remaining output element values become substantially zero. The first neural network 52 is configured to perform learning by such an inverse error propagation method.

In this example, as shown in FIG. 4, the second neural network 55 includes an input layer 55 i including ₁₂ input elements i ₁ , i ₂ ,..., I ₁₂ , and 7 elements m ₁ , m ₂ , .., M ₇ includes an intermediate layer 55 m and an output layer 55 n including _one output element n ₁ . As in the first neural network 52, the second neural network 52 uses, in this example, an input inputted to the input elements i ₁ , i ₂ ,..., I ₁₂ of the input layer 55i by the reverse error propagation method. , the correct output to the output device n ₁ of the output layer 55n is configured to perform the learning, as obtained.

  2 receives the outputs of the output layer 52n of the first neural network 52 and the output layer 55n of the second neural network 55 under the control of the control unit 50, and recognizes / A process of outputting the evaluation result as an image is performed. This recognition / evaluation result is displayed on the display unit 3 of the behavior evaluation apparatus 1.

  The above-described input processing unit 51, first neural network 52, second neural network 55, and output processing unit 53 are configured by software. Since this behavior evaluation apparatus 1 includes only the acceleration sensor 2 as a detection unit for the subject 90, it can be configured simply and inexpensively.

  As shown in FIG. 9A, the behavior evaluation apparatus 1 is roughly divided by the control by the control unit 50, and the process of “recording a model” stage shown in step S101 and the “behavior evaluation” shown in step S102. Perform stage processing.

  FIG. 9B shows more specifically the process (step S101) in the “recording of model” stage. In this example, when the user continues to press the button switch 11 for 3 seconds or more during the operation period in which the power switch 42 is turned on, the processing of this “exemplary recording” stage is started.

  First, in step S201, the control unit 50 displays, for example, “Do you want to learn recognition?” On the display unit 3, and gives a learning instruction to the first neural network 52 (this is referred to as “first learning instruction”). Determine whether or not there is). In this example, the first learning instruction is input by the user pressing the button switch 11 once (less than 3 seconds). If there is a first learning instruction within a predetermined period (for example, 5 seconds) (YES in step S201), the process proceeds to step S202, and learning processing of the first neural network 52 is performed. When the learning of the first neural network 52 is completed in advance and there is no first learning instruction in step S201 (NO in step S201), or after learning in step S202, the process proceeds to step S203. In step S203, the recognition process by the first neural network 52 is started.

  During the period when the recognition by the first neural network 52 is performed in real time, the control unit 50 displays, for example, “Do you want to learn the evaluation?” It is determined whether or not there is a learning instruction (referred to as a “second learning instruction”) for the neural network 55. In this example, the second learning instruction is input when the user presses the button switch 11 once (less than 3 seconds), similarly to the first learning instruction. If there is a second learning instruction within a predetermined period (for example, 5 seconds) (YES in step S204), the process proceeds to step S205 to perform learning processing of the second neural network 55. When the learning of the second neural network 55 is completed in advance and there is no second learning instruction in step S205 (NO in step S204), or after the learning in step S205, the process of “recording a model” is ended. To do.

Next, assuming the case where the subject 90 performs a plurality of types of actions, the learning stage processing (step S202 in FIG. 9B) of the first neural network 52 in the behavior evaluation apparatus 1 will be described with reference to FIG. This will be specifically described with reference to (A). In this example, the plurality of types of movements are “pause”, “walking A”, “walking B”, “running A”, “running B”, “ It is assumed that seven types of motions “swing A” and “swing B” are indicated (the number of types of motion corresponds to the number of output elements n ₁ , n ₂ ,..., N ₇ of the first neural network 52). Yes.) Here, “pause” refers to a state in which the subject 90 is stationary. “Walk A” refers to the state where the subject 90 is walking slowly, and “Walk B” refers to the state where the subject 90 is walking fast. “Run A” indicates a state where the subject 90 is running slowly, and “Run B” indicates a state where the subject 90 is running fast. “Swing A” refers to a state in which the subject 90 is swinging a golf club having a long shaft (for example, No. 1 wood) in a flat manner (the swing plane has a small inclination angle with respect to the horizontal plane). Indicates a state in which a golf club having a short shaft (for example, a 9 iron) is swinging upright (the inclination angle of the swing plane with respect to the horizontal plane is large). The swing category may include “swing C”, “swing D”,... In addition to “swing A” and “swing B”.

  As shown in step S1 in FIG. 10A, first, the subject 90 wears the behavior evaluation device 1 and performs seven types of operations that the behavior evaluation device 1 (first neural network 52) wants to learn. Do it sequentially. As a result, the acceleration sensor 2 of the behavior evaluation apparatus 1 obtains the acceleration data corresponding to the above-described seven types of operations by actually measuring each. As described above, the output of the acceleration sensor 2 in the X, Y, and Z directions (the value of a 10 millisecond period) is stored in the memory 54. At the same time, the input processing unit 51 generates the 12 types of feature data X1 to X4, Y1 to Y4, and Z1 to Z4 described above. For example, when the subject 90 is performing “swing A” as indicated by an arrow 91 in FIG. 5, the average values X1, Y1, and Z1 of outputs in the X, Y, and Z directions are the data 12 in FIG. , 13, 14 are obtained.

  Next, as shown in step S2 in FIG. 10A, a characteristic part is extracted in each of the seven types of operations. In the example of FIG. 6, data of period 15 representing characteristic data is extracted when the subject 90 is performing “swing A”. In this case, the accuracy of learning can be improved, resulting in prevention of erroneous recognition, which is meaningful. In the example of FIG. 6, only the average values X1, Y1, and Z1 are shown, but the standard deviations X2, Y2, and Z2, the differences X3, Y3, and Z3 between the maximum value and the minimum value, or the number of zero crossings X4 , Y4, and Z4, if there is a period representing characteristic data, it is desirable to extract data for that period.

Next, as shown in step S3 in FIG. 10A, teacher data representing a correct answer (action) is created. As the teacher data, 12 types of feature data X1 to X4, Y1 to Y4, and Z1 to Z4 generated by the input processing unit 51 when the subject 90 performs a certain operation are input to the first neural network 52. It shall correspond to the case. In the example in which the subject 90 performs “swing A”, the teacher data is 0, 0, 0, for the outputs of the output elements n ₁ , n ₂ , n ₃ , n ₄ , n ₅ , n ₆ , n ₇ . 0, 0, 1, 0.

Next, as shown in step S4 in FIG. 10A, the first neural network 52 is trained by the inverse error propagation method using the teacher data. That is, feature data X1 to X4, Y1 to Y4, and Z1 to Z4 are inputted as input data sets to the input elements i ₁ , i ₂ ,..., I ₁₂ of the first neural network 52 shown in FIG. The error between the output of the output elements n ₁ , n ₂ ,..., N ₇ and the teacher data representing the correct answer is obtained. Then, feedback is performed so that an error between the output of the output elements n ₁ , n ₂ ,..., N ₇ and the teacher data is eliminated, and the weight coefficients w _j1 , w _{j2 ,.} Change and adjust. The output of the intermediate layer that caused the error is also adjusted sequentially. As a result, as shown in steps S5 and S6 in FIG. 10A, weight coefficients w _j1 , w _j2 ,..., W _j12 for determining the operation state of the subject 90 are obtained, and the first neural network is obtained. These weighting factors are incorporated into 52. This is equivalent to establishing a calculation formula (algorithm) for obtaining an output data set from an input data set on the software constituting the first neural network 52.

When learning is performed in this manner, only one output element among the output elements n ₁ , n ₂ ,..., N ₇ of the output layer 52n reacts to an input data set representing a certain motion. (The value becomes 1), and the remaining output elements do not react (the value becomes substantially 0). In this example, as shown in FIG. 7, when n ₁ reacts, “pause”, when n ₂ reacts, “walk A”, when n ₃ reacts, “walk B”, n ₄ reacts “Run A”, “Run B” when n ₅ reacts, “Swing A” when n ₆ reacts, and “Swing B” when n ₇ reacts. Become.

  In this way, in the learning stage of the first neural network 52, between the plural types (seven types in this example) of movements or actions performed by the subject 90 and the output of the acceleration sensor 2 attached to the subject 90. Based on the corresponding relationship, learning for recognizing the action or action performed by the subject 90 from the output of the acceleration sensor 2 is completed.

  Next, the process in the recognition stage of the first neural network 52 in the behavior evaluation apparatus 1 (step S203 in FIG. 9B) will be specifically described with reference to FIG. It is assumed that the subject 90 wears the behavior evaluation device 1 and performs a certain operation among the seven types of operations in FIG.

  As shown in step S <b> 11 in FIG. 10B, first, the acceleration sensor 2 of the behavior evaluation apparatus 1 actually acquires and acquires acceleration data corresponding to the seven types of operations. As described above, the output of the acceleration sensor 2 in the X, Y, and Z directions (the value of a 10 millisecond period) is stored in the memory 54. At the same time, as shown in step S12, the input processing unit 51 generates feature data X1 to X4, Y1 to Y4, and Z1 to Z4, which are the 12 types of statistical data described above.

Next, as shown in step S13, the input elements i ₁ , i ₂ ,..., I ₁₂ of the first neural network 52 include the 12 types of feature data X1 to X4, Y1 to Y4, and Z1 to Z4. An input data set is entered. Then, as shown in step S14, data representing the result is output to the output layer 52n (output elements n ₁ , n ₂ ,..., N ₇ ) of the first neural network 52. As shown in step S15, from the result, the output processing unit 53 recognizes what the subject 90 is currently performing.

  Then, as shown in step S <b> 16, the recognition result by the output processing unit 53 is displayed on the display unit 3.

  The behavior evaluation apparatus 1 is configured so that the learning process of the first neural network 52 (step S202 in FIG. 9B) and the second neural network 55 are within the operation period in which the power switch 42 is turned on. Except for the period in which the learning stage process (step S205 in FIG. 9B) is performed, the processes of steps S11 to S16 are repeated at a cycle of 100 ms. Thereby, recognition of the operating state of the subject 90 is always performed substantially in real time.

  For example, as schematically shown in FIG. 11A, as the time t elapses, the operation state of the subject 90 at that time is sequentially recognized from the measurement data 21 of the acceleration sensor 2 stored in the memory 54 in real time. (The recognition result is indicated by reference numeral 22). In this example, “pause”, “pause”, “walk A”, “pause”, “swing A”, “walk A”, “walk A”, and “pause” are recognized in this order. The memory 54 sequentially stores the recognition result 22 of the operation state in association with the output of the acceleration sensor 2.

  The learning process of the first neural network 52 (step S202 in FIG. 9B) and the learning process of the second neural network 55 (step S205 in FIG. 9B) are performed. During the period, for example, “learning” is displayed on the display unit 3 to clearly indicate that the operation state of the subject 90 is not recognized or the behavior is not evaluated.

  Next, the learning stage processing (step S205 in FIG. 9B) of the second neural network 55 in the behavior evaluation apparatus 1 will be specifically described.

  First, the subject 90 wears the behavior evaluation device 1 and performs an operation desired to be evaluated by the behavior evaluation device 1 (the second neural network 55). In this example, it is assumed that the subject 90 as a user performs “swing A”, that is, swings No. 1 wood as a golf club and hits a golf ball.

  If the subject 90 hits a nice shot that is satisfactory to the subject 90, the subject 90, as a user in this example, presses the button switch 11 once (less than 3 seconds) within 30 seconds after the swing. That is, the above-described second learning instruction is input (step S204 in FIG. 9B). Thereby, the process of the learning stage of the second neural network 55 shown in FIG.

  When the button switch 11 is pressed, as shown in step S21 in FIG. 12A, the control unit 50 is recognized in the memory 54 through the first neural network 52 that is the latest from the current time t1. Search for a motion or action. In this example, the control unit 50 is set to search for a swing among a plurality of types of actions or actions that the first neural network 52 has learned. The length of the search target period is set as the latest one minute retroactive from the current time t1.

  When the control unit 50 searches the memory 54 and finds a certain swing in the recognition result 22 of the operation state (YES in step S22), the control unit 50 uses the swing when the subject 90 makes a nice shot. If it is determined that there is, the corresponding output (measurement data) of the acceleration sensor 2 is read from the memory 54 as shown in step S23. In the example of FIG. 11A, the output 21F of the acceleration sensor 2 corresponding to “swing A” is read.

Next, as shown in step S24 in FIG. 12A, the control unit 50 inputs the read output 21F of the acceleration sensor 2 to the input layer 55i of the second neural network 55 for learning. Specifically, twelve types of feature data X1 to X4, Y1 to Y4, and Z1 to Z4 generated by the input processing unit 51 as the measurement data 21F are input to the input elements i ₁ and i of the second neural network 55. ₂ ,..., I ₁₂ are input as an input data set. At the same time, the control unit 50 gives a numerical value “1” as teacher data to the output layer 55 n (output element n 1) of the second neural network 55.

This causes the second neural network 55 to perform learning by the reverse error propagation method, as in the first neural network 52. That is, feature data X1 to X4, Y1 to Y4, and Z1 to Z4 are inputted as input data sets to the input elements i ₁ , i ₂ ,..., I ₁₂ of the second neural network 55 shown in FIG. obtaining an error between the output of the output element n _1, and the teacher data representing the correct answer time. Then, feedback is performed so that the error between the output of the output element n ₁ and the teacher data is eliminated, and the weight coefficients w _j1 , w _j2 ,..., W _j12 are changed and adjusted. The output of the intermediate layer that caused the error is also adjusted sequentially. Thereby, as shown in steps S25 and S26 in FIG. 12A, the weighting factors w _j1 , w _j2 ,..., W _j12 for evaluating the “swing A” of the subject 90 are obtained, and the second These weight coefficients are incorporated into the neural network 55 of FIG.

This way Yuku performs learning on the input data set representing the "swing A" when the subject 90 has a nice shot results in a value of 1 output element n ₁ of the output layer 55n, Miss "swing B" swing and another when the shot, "swing C", (close to 0) ... input value of the output element n ₁ is smaller than the data set representing the way becomes.

  In this way, at the learning stage of the second neural network 55, the correspondence between “swing A” when the subject 90 makes a nice shot and the output of the acceleration sensor 2 attached to the subject 90. Based on the output of the acceleration sensor 2, learning for evaluating the swing performed by the subject 90 is completed.

  In the above example, the output of the acceleration sensor 2 corresponding to the swing when the subject 90 makes a nice shot is used as the input data set for learning of the second neural network 55. Of course, the present invention is not limited to this. For example, the output of the acceleration sensor 2 corresponding to a swing when the golf coach takes a model shot may be used as an input data set for learning of the second neural network 55.

  In any case, in this behavior evaluation apparatus 1, when the user presses the button switch 11, the second neural network 55 can learn an appropriate model at a desired timing.

  Next, processing in the “behavior evaluation” stage in the behavior evaluation apparatus 1 (step S102 in FIG. 9A) will be specifically described.

  In this “behavior evaluation” stage, the first neural network 52 always recognizes the operating state of the subject 90 in real time. For example, as schematically shown in FIG. 11B, as the time t elapses, the operation state of the subject 90 at that time is sequentially recognized from the measurement data 23 of the acceleration sensor 2 stored in the memory 54 in real time. (The recognition result is indicated by reference numeral 24). In this example, “pause”, “pause”, “walk A”, “pause”, “swing A”, “walk A”, “walk A”, and “pause” are recognized in this order. The memory 54 sequentially stores the recognition result 24 of the operation state in association with the output of the acceleration sensor 2.

  As shown in step S31 in FIG. 12B, the control unit 50 constantly searches in real time whether or not a certain swing is included in the recognition result 24 of the operation state in the memory 54. If the swing is included in the recognition result 24 of the operation state (YES in step S32), the control unit 50 reads the output of the acceleration sensor 2 corresponding to the swing from the memory 54 as shown in step S33. In the example of FIG. 11A, the output (measurement data) 23F of the acceleration sensor 2 corresponding to “swing A” is read.

Next, as shown in step S34 in FIG. 12B, the control unit 50 inputs the read output (measurement data) 23F of the acceleration sensor 2 to the second neural network 55 for behavior evaluation. Input to layer 55i. Specifically, the 12 types of feature data X1 to X4, Y1 to Y4, and Z1 to Z4 generated by the input processing unit 51 as the measurement data 23F are input to the input elements i ₁ and i _{2 of} the second neural network 55. , _..., it is input to the _{i 12} as the input data set. Then, as shown in step S35, data representing the result is output to the output layer 55n (output element n ₁ ) of the second neural network 55. That is, the degree of approximation (numerical value from 0 to 1) representing the degree close to the model (learned in step S205 in FIG. 9B) is output. For example, as illustrated in FIG. 8A, a numerical value of 0.80 is output to the output element n ₁ as the degree of approximation. In this example, the output layer 55n of the second neural network 55 includes only a single output element n _1, that one swing subject 90 was clearly represent the degree close to model a single number Can do.

  Then, as shown in step S <b> 36 in FIG. 12B, the output processing unit 53 displays the swing evaluation result on the display unit 3. In this example, the output processing unit 53 multiplies the numerical value of the degree of approximation by 100 to calculate a score P of 100 points. In the case of P = 80, display is made such that “the previous swing is 80 points”.

  As described above, according to the behavior evaluation apparatus 1, the score display based on the degree of approximation indicates that the user is close to the model of the swing that the subject 90 has made, that is, when the subject 90 is in good condition ( It is possible to clearly know how close the swing is (when a nice shot is taken) or how close the swing is when the golf coach takes a model shot.

  The behavior evaluation apparatus 1 repeats the processes of steps S31 to S36 in a cycle of 100 ms during the period when the “behavior evaluation” process is performed. Thereby, the evaluation of the swing performed by the subject 90 can be performed substantially in real time at all times.

  In this example, when the power switch 42 is turned off, or when the user keeps pressing the button switch 11 for 3 seconds or more during the operation period in which the power switch 42 is turned on, this “behavior evaluation” stage is performed. This processing is finished.

  As is clear from the above, in the behavior evaluation apparatus 1 of this embodiment, when the user presses the button switch 11 and inputs the second learning instruction, the second neural network 55 is appropriately handed at a desired timing. You can learn books. Therefore, according to this behavior evaluation apparatus 1, it is possible to increase the accuracy of evaluation of the action or behavior performed by the subject 90 in the “behavior evaluation” stage as compared with the conventional case.

  In this embodiment, the user presses the button switch 11 to input the first and second learning instructions. However, the present invention is not limited to this. The wireless communication device 4 may input first and second learning instructions via a wireless communication device 4 from a mobile phone or a personal computer that exists in the vicinity of the behavior evaluation device 1.

  In this embodiment, in the learning stage of the first neural network 52 (step S202 in FIG. 9B), the subject 90 wears the behavior evaluation device 1, measures the acceleration data, and first Input data to be learned by the neural network 52 is obtained. However, since the first neural network 52 only recognizes a rough operation state, it is not always necessary to use actually measured data from the subject 90 as input data. In the learning of the first neural network 52, input data prepared for learning is input in advance from a personal computer or the like existing in the vicinity of the behavior evaluation apparatus 1 via the wireless communication apparatus 4, and a learning algorithm is executed. You may go by.

  In the above-described embodiment, the first neural network 52 includes three layers and includes 12 input elements and 7 output elements. The second neural network 55 has three layers and includes 12 input elements and 1 output element. However, the present invention is not limited to this, and the first neural network 52 and the second neural network 55 may be multi-layered and may include many input elements and output elements.

In particular, the number of output elements of the output layer 55n of the second neural network 55 may be the number of models. For example, when there is model data for “Swing A” and model data for “Swing B”, two output layers 55n of the second neural network 55 are output as illustrated in FIG. 8B. It is assumed to include elements n ₁ and n ₂ . In this case, in the learning stage of the second neural network 55, when inputting the input data set of role model corresponding to "swing A" gives the value "1" to n ₁ as teacher data, numeric n ₂ " 0 ”is given. On the other hand, when a model input data set corresponding to “Swing B” is input, a numerical value “0” is given to n ₁ as teacher data, and a numerical value “1” is given to n ₂ . In this way, learning is completed. Then, in the “behavior evaluation” stage, for example, when the subject 90 performs “swing A” close to a model, as illustrated in FIG. 8B, for example, the output element n ₁ has a numerical value “0.90”. Is output, and the numerical value “0.10” is output to the output element n ₂ . This output result indicates that the swing performed by the subject 90 is “swing A” because the value of n ₁ is the largest among n ₁ and n ₂ . In addition, the degree of approximation of “swing A” by the subject 90 with respect to the model “swing A” is 0.90 (90 points out of 100).

  Thus, when the output layer 55n of the second neural network 55 includes a plurality of output elements, a plurality of types of swings can be evaluated.

  Further, in the above-described embodiment, the feature data X1 to 12 which are the above-described 12 types of statistical data are input by the input processing unit 51 from the X-, Y- and Z-direction outputs (values of 10 milliseconds) of the acceleration sensor 2. X4, Y1 to Y4, and Z1 to Z4 were produced. However, the present invention is not limited to this. For example, instead of or in addition to the standard deviations X2, Y2, and Z2, the “distribution” of the output in the X, Y, and Z directions of the acceleration sensor 2 for 5 seconds from time t0 to t1 is generated as feature data. It may be used. Alternatively, instead of or in addition to the differences X3, Y3, and Z3 between the maximum value and the minimum value, the “maximum” of the output in the X, Y, and Z directions of the acceleration sensor 2 during 5 seconds from time t0 to t1 The “value” and “minimum value” may be extracted and used as feature data. In short, the present invention can be implemented by using at least two types of feature data among the feature data mentioned in this embodiment. Of course, all of the feature data mentioned in this embodiment may be used.

  In this embodiment, the behavior evaluation apparatus 1 can be configured to be integrated into an electronic device such as the mobile phone 101, a pedometer / activity meter, and a mobile computer. Thereby, a user's convenience can be improved.

  On the other hand, the acceleration sensor 2 may be configured separately from the main body casing 7 so that only the acceleration sensor 2 (and the associated wireless communication device) is attached to the subject 90. In this case, the acceleration sensor 2 communicates with the main body via a wireless communication device.

  Moreover, in this embodiment, although the behavior evaluation apparatus 1 described the example which evaluates a golf swing, naturally, it is not restricted to this. For example, in dance, it is possible to evaluate how close the movement of the subject 90 is to the model by having the behavior evaluation apparatus 1 input data using the movement of the instructor as a model.

  Note that the subject 90 targeted by the behavior evaluation apparatus 1 typically refers to a human as described above, but also includes animals. This behavioral evaluation apparatus 1 can be preferably applied also when a user prepares tricks for an animal as the subject 90.

DESCRIPTION OF SYMBOLS 1 Behavior evaluation apparatus 2 Acceleration sensor 3 Display part 4 Wireless communication apparatus 5 Arithmetic apparatus 6 Battery 7 Main body casing 41 Instruction switch 50 Control part 51 Input processing part 52 1st neural network 53 Output processing part 54 Memory 55 2nd neural network

Claims (5)

An acceleration sensor to be attached to the subject;
A memory for storing the output of the acceleration sensor;
From the output of the acceleration sensor, comprising a neural network that has completed learning to recognize a model action or action,
A behavior evaluation device that recognizes the motion or behavior of the subject through the neural network from the output of the acceleration sensor attached to the subject and evaluates the degree of the recognized motion or behavior close to the example In
The neural network is
Learning that recognizes the movement or action performed by the subject based on the correspondence between the plurality of types of movement or action performed by the subject in advance and the output of the acceleration sensor attached to the subject. After learning, based on the output of the acceleration sensor, a first neural network that recognizes the movement or action of the subject in real time;
Based on the instruction, the output of the acceleration sensor corresponding to the motion or action recognized through the first neural network is read from the memory and learned as a model, and after the learning, the first neural network And a second neural network that outputs a degree of approximation representing the degree of movement or action recognized through the model. The behavior evaluation apparatus according to claim 1,
An instruction switch for inputting the above instructions;
When the instruction is input by the instruction switch, the output of the acceleration sensor corresponding to the action or action recognized through the first neural network immediately before the current time in the memory is searched and searched. A behavior evaluation apparatus comprising: a control unit that reads out the output of the acceleration sensor from the memory and inputs the output to the second neural network for learning. The behavior evaluation apparatus according to claim 1 or 2,
The behavior evaluation apparatus according to claim 1, wherein the output layer of the second neural network includes only a single element. In the behavior evaluation device according to any one of claims 1 to 3,
The behavior evaluation apparatus characterized in that the degree of approximation is a numerical value. In the behavior evaluation device according to any one of claims 1 to 4,
The behavior evaluation apparatus characterized in that the operation or behavior includes a golf swing.

Images