JP6530350B2 - Sequential posture identification device, sequential posture identification method, and sequential posture identification program - Google Patents

Description

  The present invention relates to a sequential posture identification device, a sequential posture discrimination method, and a sequential posture discrimination program.

  BACKGROUND In recent years, the use of information processing terminals so-called wearable devices (hereinafter, wearable devices) that can be worn on a human body is spreading. The wearable device can also be used to monitor the health condition and lifestyle of the user continuously and over the long term, since the user can wear and carry it on a daily basis.

Satoshi Akahori, Atsushi Kishimoto, Koji Oguri, "Action Estimation Using a Single 3-Axis Acceleration Sensor", IEICE Technical Report, MBE, ME and Bio Cybernetics, The Institute of Electronics, Information and Communication Engineers, 2005 Dec. 2, 105, 456, p. 49-52

  The present inventors detect the posture and exercise state of the user with the wearable device, measure the biological signal of the user, and associate the posture and exercise state with the biological signal, thereby linking the user to health care and disease prevention. We considered that. Then, the present inventors have filed patent applications for inventions that sequentially perform posture identification using wearable devices and acquire and evaluate autonomic nerve function information (Japanese Patent Application No. 2015-028850).

  By the way, when a user's posture and an exercise state are identified using a wearable device, it has been found that there is a posture and an exercise state which are difficult to identify due to wrinkles and the like unique to the user.

  The embodiment of the disclosure is made in view of the above, and an object of the embodiment is to provide a technique for realizing an improvement in accuracy of sequential posture identification using information measured by a user.

  The disclosed posture recognition apparatus, method, and program extract a first feature corresponding to a first predetermined period and a second feature corresponding to a second predetermined period from acceleration information of a user, and A machine learning based on the feature amounts of 1 creates a discrimination model for identifying the posture and motion state of the user, and based on the discrimination model and the second feature amount, a plurality of corresponding to the posture and motion state of the user The posture of the user for a second predetermined period from a classification that is easily distinguishable to a classification that is difficult to distinguish among a plurality of divisions that are divisions and that include subdivisions corresponding to predetermined dates and times for the same posture and exercise state And by sequentially selecting the sections corresponding to the exercise state, the posture and the exercise state of the user in the second predetermined period are identified.

  The disclosed sequential posture identification device, sequential posture identification method, and sequential posture identification program have an effect of realizing improvement in accuracy of sequential posture identification using information measured by the user.

FIG. 1 is a schematic diagram showing an example of the configuration of the sequential posture identification system according to the first embodiment. FIG. 2 is a flowchart showing an example of the flow of the sequential posture identification process according to the first embodiment. FIG. 3 is a flowchart showing an example of the flow of stepwise determination in the sequential posture identification process of the first embodiment. FIG. 4 is a diagram for explaining the flow of stepwise determination in the sequential posture identification process in the first embodiment. FIG. 5 is a diagram for describing a method of detecting the number of oscillations among feature amounts used in the posture recognition process. FIG. 6 is a graph showing an example of the acceleration and the synthetic acceleration when the sensor is rotated. FIG. 7 is a schematic diagram showing an example of the flow of learning processing in the sequential posture identification processing by the sequential posture identification device according to the first embodiment. FIG. 8 is a schematic view showing an example of the flow of identification processing (including attitude identification processing) in the sequential attitude identification processing by the sequential attitude identification device according to the first embodiment. FIG. 9 is a diagram for explaining the determination in the sequential posture identification process according to the modification. FIG. 10 is a diagram for explaining the improvement in accuracy when the sequential posture identification method according to the embodiment is used. FIG. 11 is a diagram illustrating that information processing by the sequential posture identification program according to the disclosed technology is specifically realized using a computer.

  Hereinafter, embodiments of the disclosed sequential posture identification device, sequential posture identification method, and sequential posture identification program will be described in detail based on the drawings. The present invention is not limited by this embodiment. Moreover, each embodiment can be combined suitably.

First Embodiment
The sequential posture identification apparatus according to the first embodiment creates a discrimination model for identifying the posture and motion state of the user by performing sequential machine learning based on acceleration information measured from the user wearing the wearable device. . Then, based on the identification model, the sequential posture identification device identifies that the posture and the motion state of the user correspond to a plurality of different segments. The sequential posture identifying apparatus identifies the posture and the motion state of the user by selecting the plurality of identified segments in a stepwise manner based on a predetermined order.

  The sequential posture identification apparatus according to the first embodiment, in particular, is provided with two identification units for identifying the stationary state in order to improve the accuracy of discrimination of the stationary state. And one specific part distinguishes two states of "the recumbent position" and "other than a recumbent position". And another specific part distinguishes two states of "standing" and "sitting" which are subdivisions of "other than a recumbent". As described above, the sequential posture identifying apparatus according to the first embodiment selects the identification results of the posture and the motion state in a plurality of stages. That is, the posture identifying apparatus sequentially performs determination on the upper class in the upper stage, and performs determination in the lower stage on the lower class. For this reason, by adjusting settings such as the number of stages, the number of subdivisions, and the type of division, it is possible to improve the identification accuracy for the difficult to distinguish.

  In addition, the sequential posture identification device of the first embodiment improves identification accuracy by specifying the type of stationary state in multiple stages. The feature quantities used for identification differ greatly between the stationary state and the operating state. When trying to distinguish between the stationary state and the operating state by one discrimination model, it tends to be difficult to distinguish the posture in the stationary state, which has a relatively small difference from each other, behind the contrast between the stationary state and the operating state . In addition, it is difficult to further classify and identify postures with relatively small differences. Therefore, the sequential posture identifying apparatus according to the first embodiment is configured to first identify the posture and the motion state with high contrast, and separately identify the posture and the motion state with low contrast. For example, even in the same resting state, the contrast between "recumbent position" and "standing position" or "sitting position" is larger than the contrast between "standing position" and "sitting position". Therefore, the posture recognition apparatus sequentially executes the identification of the “throw position” and the “non-throw position” and the identification of the “standing position” and the “sitting position”.

  In the following first embodiment, “standing position” and “sitting position” will be described as an example of a division that is difficult to identify. In addition, "other than recumbent position" is set as the upper class of "standing position" and "sitting position". Note that "standing position", "sitting position" and "throw position" are all types of stationary states. Here, “division” is the same as “label” used in machine learning, and refers to an output generated for an input based on machine learning.

(One Example of Configuration of Sequential Posture Identification System 1 According to First Embodiment)
FIG. 1 is a schematic diagram showing an example of the configuration of the sequential posture identification system 1 according to the first embodiment. The sequential posture identification system 1 shown in FIG. 1 includes a wearable device 10 and a sequential posture identification device 20. The wearable device 10 and the posture identification device 20 are connected communicably via a network.

  The type of network connecting the wearable device 10 and the posture identification device 20 sequentially is not particularly limited, and may be a wired network or a wireless network. However, in order not to disturb the action of the user who wears the wearable device 10, for example, use a smart phone or the like connected by Bluetooth (registered trademark) or use Wi-Fi (registered trademark) or the like. Is preferred.

(An example of the configuration of the wearable device 10)
The wearable device 10 is an electronic device that a user can wear and carry. In the example of FIG. 1, the wearable device 10 has a shirt shape that can be attached and detached by the user. However, the shape of the wearable device 10 is not limited to a shirt, and may be any shape as long as it can be attached to the user, such as a belt shape. Alternatively, a wearable computer including a processor and a memory may be used as the wearable device 10, and the measured information may be appropriately stored in the wearable device 10. In this case, each unit (acceleration information measurement unit 101, biological signal information measurement unit 102, transmission / reception unit 103, input unit 104) described below may be configured as a processor function. Further, the information used for the processing of each part and the information generated by the processing of each part may be stored in the memory.

  The wearable device 10 includes an acceleration information measurement unit 101, a biological signal information measurement unit 102, a transmission / reception unit 103, and an input unit 104.

  The acceleration information measurement unit 101 is a sensing device that detects and measures the motion of the user's body. The acceleration information measurement unit 101 measures acceleration information of the user's body. For example, the acceleration information measurement unit 101 is configured by an acceleration sensor, for example, a three-axis acceleration sensor disposed near the trunk of the user. The acceleration information measurement unit 101 measures the acceleration of the movement of the user's body along the three axes of the longitudinal axis, the lateral axis, and the vertical axis. Hereinafter, when the front, rear, left, right, up and down are referred to, the direction in which the user's body faces is taken as a reference when the wearable device 10 is worn and stood upright.

  The biological signal information measurement unit 102 measures biological signal information that can be acquired from the user's body. The biological signal information measurement unit 102 is, for example, a sensing device that measures cardiac potential. The biological signal information measurement unit 102 measures, for example, the user's cardiac potential and information on the heartbeat. Specifically, the biological signal information measurement unit 102 measures cardiac potential by single lead at constant intervals. Further, the biological signal information measurement unit 102 measures, for example, a heartbeat interval, that is, an RR interval. Other than this, pulse waves such as photoelectric pulse waves, body impedances such as bioelectric resistance, body slight vibrations or body pressure fluctuations, and arterial pressure such as cuff pressure such as a sphygmomanometer may be measured as a biosignal. Besides this, bioelectric potential, myoelectric potential, electroencephalogram, evoked potential and the like can also be used.

  The transmission and reception unit 103 transmits the acceleration information and the biological signal information measured by the acceleration information measurement unit 101 and the biological signal information measurement unit 102 to the outside of the wearable device 10. The transmitting and receiving unit 103 also receives a signal transmitted from the outside of the wearable device 10. The transmission / reception unit 103 transmits the information to the outside each time the acceleration information measurement unit 101 and the biological signal information measurement unit 102 acquire the information. For example, the transmission and reception unit 103 transmits information by the wireless communication function. Specifically, the transmission / reception unit 103 sequentially transmits the acceleration information and the biological signal information to the posture identification device 20.

  The input unit 104 receives information that a user or the like inputs to the wearable device 10. For example, the input unit 104 receives a label (described later) input by the user or the like. The received label is sent to the transmission / reception unit 103 and sequentially sent to the posture identification device 20. The posture identification apparatus 20 sequentially associates the label with the acceleration information and the biological signal information to be received thereafter, based on a predetermined processing procedure. The details of the label will be described later. Also, the label may be input not from the input unit 104 but from the input unit 213 (described later) included in the posture identification device 20 sequentially. The wearable device 10 may not include the input unit 104. The input unit 104 can also be used for the user or the like to select a learning mode or an identification mode to be described later.

(Example of Configuration of Sequential Pose Identification Device 20)
The posture recognition apparatus 20 includes a transmitting / receiving unit 201, a feature extraction unit 202, a first creating unit 203, a second creating unit 204, a third creating unit 205, a fourth creating unit 206, a first specifying unit A second identification unit 208, a third identification unit 209, a fourth identification unit 210, an identification unit 211, a generation unit 212, and an input unit 213 are provided. The functions of the units included in the posture recognition apparatus 20 may be realized by a processor (CPU: Central Processing Unit or the like).

  The transmission / reception unit 201 receives acceleration information and biological signal information transmitted from the transmission / reception unit 103 of the wearable device 10. The transmitting and receiving unit 201 receives the acceleration information and the biological signal information sequentially transmitted by the transmitting and receiving unit 103, and sends the acceleration information and the biological signal information to the feature amount extracting unit 202.

  The feature amount extraction unit 202 extracts a feature amount used for posture identification from at least one of the acceleration information and the biological signal information. The extracted feature amount is sent to the first creating unit 203, the second creating unit 204, the third creating unit 205, and the fourth creating unit 206, and acquires baseline information of the posture and motion of the user. Used for learning processing. The extracted feature amount is also sent to the first specifying unit 207, the second specifying unit 208, the third specifying unit 209, and the fourth specifying unit 210, and is used for identification processing for identifying the posture and motion of the user. Used. Details of the feature quantity extraction process performed by the feature quantity extraction unit 202 will be described later. Note that the feature amount extraction unit 202 is not an independent configuration unit, and the first creation unit 203, the second creation unit 204, the third creation unit 205, the fourth creation unit 206, the first identification unit 207, The processing of the feature quantity extraction unit 202 may be incorporated in each of the second identification unit 208, the third identification unit 209, and the fourth identification unit 210.

  The first creating unit 203, the second creating unit 204, the third creating unit 205, and the fourth creating unit 206 execute learning processing in which machine learning is sequentially performed using feature amounts. The first creation unit 203 creates a motion / still discrimination model for identifying the motion state and the still state of the user. The second creating unit 204 creates an exercise identification model for identifying the type of exercise when the user is in an exercise state. The third creating unit 205 creates a first stationary identification model for identifying whether the type of the stationary state is “recumbent position” or “other than decedent position” when the user is in a stationary state. The fourth creating unit 206 creates a second stationary identification model for identifying whether the user is "standing" or "sitting" when the posture of the user is "other than lying". The motion / stationary discrimination model, the motion discrimination model, the first stationary discrimination model, and the second stationary discrimination model created by the learning process become baseline information of the posture and motion of the user. The learning process is a process of creating such a discrimination model, that is, a process of acquiring baseline information. Details of the learning process will be described later.

  Note that, for example, the first stationary identification model created by the third creating unit 205 is one of "Supposing", "Supposing (backing)", and "Sideover (lateral)" as "recumbent". Identify some or all. The first stationary identification model is created so that each type of decubitus position can also be identified. That is, the first stationary identification model can identify "suspicious position", "supine position", "lateral position", and "others (standing position and sitting position)".

  Further, the second stationary identification model created by the fourth creating unit 206 is created to be identified as “standing position”, including postures such as “curly” and “body curl”. Furthermore, the second stationary identification model is created to identify "sit in a chair" and "sit on the floor" as "sit".

  The first specifying unit 207, the second specifying unit 208, the third specifying unit 209, and the fourth specifying unit 210 are respectively a first creating unit 203, a second creating unit 204, and a third creating unit 205. The posture and the motion of the user are specified using the identification model generated by the fourth generation unit 206 and the feature amount extracted by the feature amount extraction unit 202.

  The first identification unit 207 identifies whether the user is in an exercise state or in a rest state, using the motion / still discrimination model created by the first creation unit 203. The second identification unit 208 uses the exercise identification model created by the second creation unit 204 to identify the type of exercise that the user is exercising in the exercise state. The third specifying unit 209 uses the first stationary identification model created by the third creating unit 205 to identify whether the posture of the user is “recumbent” or “other than decubitus”. The fourth specifying unit 210 uses the second stationary identification model created by the fourth creating unit 206 to identify whether the user's posture is “standing” or “sitting”. Details of the posture specifying process by the first specifying unit 207, the second specifying unit 208, the third specifying unit 209, and the fourth specifying unit 210 will be described later.

  The identification unit 211 changes the result of the posture identification process by the first identification unit 207, the second identification unit 208, the third identification unit 209, and the fourth identification unit 210 from the easily distinguishable classification to the difficult classification. And sequentially select to identify the posture and motion of the user. As a result of the identification processing of the identification unit 211, identification results of the posture and motion of the user can be obtained. For example, the identification result that the user is "exercising" and "walking" can be obtained. Details of the identification process will also be described later.

  The generation unit 212 generates information in which the identification result of the identification unit 211 and the biological signal information received from the wearable device 10 are associated in time series. The generation unit 212 associates the posture and the motion of the user identified based on at least one of the acceleration information and the biological signal information acquired in a predetermined period with the biological signal information acquired in the same period. Generate correspondence information. For example, the generation unit 212 generates correspondence information by correlating the posture and exercise of the user between time T1 and T5 with the heart rate of the user between time T1 and T5. Details of the information generated by the generation unit 212 will also be described later.

  The input unit 213 sequentially receives information input from the outside of the posture identification device 20. The input unit 213 may be, for example, an input device such as a keyboard or a touch pad. Similar to the input unit 104 of the wearable device 10, the input unit 213 can be used by a user or the like to input a label to be described later. Also, the input unit 213 can be used by the user or the like to select the learning mode or the identification mode.

(An example of the general flow of sequential pose identification processing)
FIG. 2 is a flowchart showing an example of the flow of the sequential posture identification process according to the first embodiment. When the process starts, the posture identification device 20 sequentially receives a mode selection (step S201). Specifically, the selection of the learning mode or the identification mode is accepted. Here, the learning mode is an operation mode in which the posture identification apparatus 20 sequentially executes a learning process. Further, the identification mode is an operation mode in which the posture identification apparatus 20 sequentially performs identification processing (including posture identification processing). In FIG. 2, for convenience of explanation, either the learning mode or the identification mode is selected, but a mode in which the learning process and the identification process are performed in parallel may be further provided and executed. Further, the device that executes the learning process and the device that executes the identification process may be configured separately.

  Next, the posture identification apparatus 20 sequentially determines whether the learning mode is selected (step S202). When the learning mode is selected (Yes at step S202), the posture identification apparatus 20 sequentially receives a label input by the user or the like. The label is information identifying the "posture" and "exercise" of the user. For example, the label includes information of "standing position", "sitting position", "rest position" as "posture", and "exercise state" or "rest state" as "exercise". In addition, as "exercise", labels such as "walking", "jumping", "stepping", "walking" and the like can be created. Here, "walking" simply refers to walking as routine walking, and "walking" as walking actively as exercise. The sequential posture identifying apparatus 20 can distinguish between daily “walking” and “walking” as exercise by determining exercise intensity also for exercise similar to the exercise state of the body. In this manner, the sequential posture identification device 20 can distinguish and identify motions having similar intensities but different intensities. At the time of learning processing, in order to memorize in the posture identification device 20 what kind of "posture, exercise" information is to be learned, the feature amount and the label including the two pieces of "posture, exercise" are corresponded To process.

  Then, the posture identification device 20 sequentially extracts the feature amount from at least one of the acceleration information and the biological signal information received from the wearable device 10. The received label and the extracted feature amount are input to the first creating unit 203, the second creating unit 204, the third creating unit 205, and the fourth creating unit 206 (step S203). Each creation unit 203, 204, 205, 206 determines whether the label indicates the "exercise state" (step S204). If it is determined that the label indicates the "exercise state" (Yes at step S204), the first creating unit 203 performs machine learning based on the feature amount, and creates or updates the motion / still discrimination model (step S205). In addition, the second creating unit 204 performs machine learning based on the feature amount, and creates or updates a motion identification model (step S205). If it is determined that the label indicates “exercise state” (Yes at Step S204), the third creating unit 205 and the fourth creating unit 206 do not execute machine learning.

  When it is determined that the label indicates the "rest state" (No at step S204), the first creating unit 203 performs machine learning based on the feature amount, and creates or updates the motion / still identification model (step S206). Also, the second creating unit 204 does not execute machine learning. The third creating unit 205 and the fourth creating unit 206 perform machine learning based on the feature amount, and create or update the first stationary identification model and the second stationary identification model, respectively (step S206). The process in the learning mode ends here.

  On the other hand, it is assumed that the learning mode is not selected in step S202, that is, it is determined that the identification mode is selected (No in step S202). In this case, the posture identification apparatus 20 sequentially extracts the feature amount from at least one of the acceleration information and the biological signal information received from the wearable device 10 (step S207). Then, the posture identification apparatus 20 sequentially sends the feature amount to the first specifying unit 207, the second specifying unit 208, the third specifying unit 209, and the fourth specifying unit 210. The first specifying unit 207 specifies whether the user is in an exercise state or a rest state based on the feature amount and the motion / still discrimination model that has already been created (step S208). The second identification unit 208 identifies what kind of motion state it is based on the feature amount and the motion identification model that has already been created (step S209). The third specifying unit 209 specifies whether the user is lying or not on the basis of the feature amount and the first stationary identification model that has already been created (step S210). The fourth identifying unit 210 identifies the standing position or the sitting position based on the feature amount and the second stationary identification model that has already been created (step S211).

  The identification results of the first identification unit 207, the second identification unit 208, the third identification unit 209, and the fourth identification unit 210 are sent to the identification unit 211, and the identification unit 211 determines the posture of the user based on the identification results. , Identify motion (step S212). For example, the identification unit 211 identifies that the user's posture and motion at that time are “standing position and motion state”. Then, the posture identification apparatus 20 sequentially generates information in which the identification result and the biological signal information corresponding to the identification result are associated in time series (step S213). This completes the identification process. The stepwise identification process in the identification unit 211 will be further described below.

(Selection of specific result in identification process)
FIG. 3 is a flowchart showing an example of the flow of stepwise determination in the sequential posture identification process of the first embodiment. The process of FIG. 3 is performed in step S212 of FIG.

  As described above, the identification unit 211 of the posture recognition apparatus 20 sequentially uses the identification results of the user based on the identification results of the first identification unit 207, the second identification unit 208, the third identification unit 209, and the fourth identification unit 210. Identify posture and motion status.

  The identification unit 211 first receives the identification results of the first identification unit 207, the second identification unit 208, the third identification unit 209, and the fourth identification unit 210 (step S31). The identification unit 211 determines whether the identification result of the first identification unit 207 is "static" (step S32). If the identification result is not “rest” (No at step S32), the identification unit 211 selects the identification result of the second identification unit 208, and uses it as the identification result of the posture and motion state (step S33). On the other hand, when the identification result of the first identification unit 207 is "rest" (Yes at step S32), the identification unit 211 determines whether the identification result of the third identification unit 209 is "recumbent" (step S32). Step S34). When the identification result of the third identification unit 209 is “placement position” (Yes in step S34), the identification unit 211 selects the identification result of the third identification unit 209, and identifies the posture and motion state identification results. (Step S35). On the other hand, when the identification result of the third identification unit 209 is "other than lying position" (No at step S34), the identification unit 211 selects the identification result of the fourth identification unit 210, and determines the posture and movement state. It is set as an identification result (step S36). The processing by the identification unit 211 ends here.

  The process of the identification unit 211 shown in FIG. 3 will be further described with reference to FIG. FIG. 4 is a diagram for explaining the flow of stepwise determination in the sequential posture identification process in the first embodiment.

  As shown in FIG. 4, the first identification unit 207 identifies, based on the motion / static discrimination model, which one of “motion” and “static” the input feature amount corresponds to. Further, the second specifying unit 208 specifies which one of “jump”, “walking”, and “traveling” the input feature amount corresponds to, based on the exercise identification model. The third identifying unit 209 identifies, based on the first stationary identification model, which one of the “placement position” and the “non-placement position” the input feature amount corresponds to. Further, the fourth determination unit 210 specifies which of the “standing position” and the “sitting position” the input feature amount corresponds to, based on the second stationary identification model. Each identification unit independently executes identification processing, and the identification result as the execution result is transmitted to the identification unit 211.

  When the identification result of the first identification unit 207 is "exercise" ((1) in FIG. 4), the identification unit 211 does not matter regardless of the identification results of the third identification unit 209 and the fourth identification unit 210. The identification result of the second identification unit 208 is selected, and is output as the identification result of the posture and motion state ((2) in FIG. 4). At this time, the identification result of the identification unit 211 is determined according to which one of “jump”, “walk” and “travel” the identification result of the second identification unit 208 is.

  On the other hand, when the identification result of the first identification unit 207 is "rest" ((3) in FIG. 4), the identification unit 211 refers to the identification result of the third identification unit 209 ((4) in FIG. )). If the identification result of the third identification unit 209 is "recumbent", the identification unit 211 selects "recumbent" regardless of the identification result of the fourth identification unit 210, and identifies the posture and the movement state. It outputs as a result ((5) of FIG. 4). On the other hand, if the identification result of the third identification unit 209 is “other than the lying position”, the identification unit 211 refers to the identification result of the fourth identification unit 210 ((7) in FIG. 4). Then, if the identification result of the fourth identification unit 210 is "sitting", the identification unit 211 selects "sitting" and outputs it as the discrimination result of the posture and the movement state ((8) in FIG. 4). On the other hand, if the identification result of the fourth identification unit 210 is "standing", the identifying unit 211 selects "standing" and outputs it as a result of discrimination of posture and motion state ((9) in FIG. 4). ).

  As described above, in the sequential posture identification process according to the first embodiment, the posture and the movement state are specified in each of the four specifying units, and the identification unit 211 selects the specification results in a predetermined order in a stepwise manner. The final sequential attitude identification result is output. For this reason, the sequential posture identification device 10 according to the first embodiment determines in the upper part the classification that is easy to determine, for example, "exercise" and "rest". Then, the posture identification apparatus 10 sequentially determines, for example, “standing” and “sitting” in the lower part of the classification that is difficult to determine. For this reason, while the posture identification device 10 completes the easy-to-judge identification first to realize quick processing, it is possible that the hard-to-judge identification executes a detailed identification process by providing a dedicated identification unit. it can. Therefore, the posture recognition apparatus 10 can improve the accuracy of the posture recognition process sequentially.

  Next, each of feature amount extraction processing, learning processing, and identification processing (including posture identification processing) included in the sequential posture identification processing will be further described. The method of the feature quantity extraction process in the first embodiment is not particularly limited. An example of the feature amount extraction process will be described below.

(One example of feature quantity extraction processing)
In the sequential posture identification device 20 according to the first embodiment, the feature extraction unit 202 receives, from the wearable device 10, the acceleration information and the biosignal information measured in a predetermined period, and the plurality of different overlapping predetermined periods are received. Divide into periods of length. Then, the feature amount extraction unit 202 calculates a feature amount for each period. The plurality of feature quantities are used in the learning process and the posture specifying process. Note that the feature quantities used in the first creation unit 203 and the extraction cycles (T1 to T4 below) are the same as the feature quantities used in the first identification unit 207 and their extraction cycles. Also, the feature amount used by the second creation unit 204 and the extraction cycle thereof are the same as the feature amount used by the second identification unit 208 and the extraction cycle thereof. Also, the feature amount used in the third creating unit 205 and the extraction cycle thereof are the same as the feature amount used in the third specifying unit 209 and the extraction cycle thereof. In addition, the feature quantity used in the fourth creation unit 206 and the extraction cycle thereof are the same as the feature quantity used in the fourth identification unit 210 and the extraction cycle thereof. However, the feature amount used by the first creating unit 203 and its extraction cycle, the feature amount used by the second creating unit 204 and its extraction cycle, and the feature amount used by the third creating unit 205 and its extraction cycle; The feature quantities used by the fourth creating unit 206 and their extraction cycles may be different from each other.

(First feature amount)
As the first feature quantity, the feature quantity extraction unit 202 calculates basic statistics in a time-series set of acceleration information of each axis in T1 seconds. For example, at least one of the maximum value, the minimum value, the average value, and the variance value is calculated. For example, assuming that T1 = 0.4 seconds, ten pieces of acceleration information are measured for each axis in 0.4 seconds. In this case, the feature quantity extraction unit 202 extracts basic statistics of ten pieces of information on each axis as feature quantities on each axis.

(Second feature)
As the second feature quantity, the feature quantity extraction unit 202 calculates a basic statistic in a time-series set of acceleration information of each axis in T2 seconds (where T1 <T2). For example, at least one of the maximum value, the minimum value, the average value, and the variance value is calculated. For example, T2 = 2 seconds.

(Third feature amount)
As the third feature amount, the feature amount extraction unit 202 extracts the number of oscillations of each axis in T3 seconds (where T2 <T3), the average value and the dispersion value of the overall number of oscillations, or the heartbeat extracted from biological signal information At least one of an average value and a variance value of the intervals is calculated. The shaking number is a feature amount corresponding to the frequency of the user's body. The method of detecting the number of swings will be described in detail below.

(4th feature amount)
As the fourth feature amount, the feature amount extraction unit 202 calculates the number of oscillations of each axis in T4 seconds (where T3 <T4), the basic statistic of the overall number of oscillations, or the heartbeat interval extracted from biological signal information Calculate at least one of the basic statistics. Basic statistics include, for example, maximum value, minimum value, mean value and variance value.

  In addition, the feature value calculated from the heartbeat interval is not limited to the average value or the variance value of the heartbeat interval. For example, a basic statistic of heart rate may be calculated as a feature. In addition, as the first, second, third and fourth feature quantities, it is not necessary to use all of the above-mentioned feature quantities, and depending on the posture and motion to be identified, only some of the feature quantities use.

(Detection method of swing number)
Next, a method of detecting the number of swings will be described. Here, the “number of swings” is the number of steps having the same meaning as the number of vibrations of the body, such as the number of steps and the number of jumps in the vertical direction of the body. By using the number of swings as a feature amount for posture identification, it is possible to create a discrimination model in consideration of the degree of tilt of the user's body and vibration, that is, vibration, and to realize detailed posture and motion identification.

  When using the number of swings as the feature amount, the acceleration information measurement unit 101 includes at least an acceleration sensor such as a 3-axis acceleration sensor in order to measure the swing of the body of the user wearing the wearable device 10. For example, in the case of using a three-axis acceleration sensor, it is possible to measure the acceleration received by the acceleration sensor in three axes (see FIG. 5) of X, Y, and Z axes.

  FIG. 5 is a diagram for describing a method of detecting the number of oscillations among feature amounts used in the posture recognition process. When the user changes the posture or moves and the acceleration sensor is tilted, the gravitational acceleration detected by the acceleration sensor changes. Thus, the inclination of the acceleration sensor, that is, the inclination of the user's body can be detected. Further, by changing the synthetic acceleration of the acceleration in each of the X, Y, Z axes detected by the acceleration sensor, it is possible to detect the acceleration sensor, that is, the degree of shaking of the user's body. For example, FIG. 5 is a graph showing an example of measurement values of three-axis acceleration and synthetic acceleration detected by the acceleration sensor while the user is walking.

  The swing number is detected using the measurement value obtained by the acceleration sensor as described above. An example of the method of swing number detection will be described below.

First, the synthetic acceleration _Ar is calculated based on the following equation (1).

According to the synthetic acceleration A _r as defined in formula (1), can shake equal detection for each axis direction, it does not change substantially synthesis acceleration when there is a movement such as rotation of the sensor. FIG. 6 is a graph showing an example of the acceleration and the synthetic acceleration when the sensor is rotated. In the example of FIG. 6, the rotational movement in the Z-axis direction occurs, but the combined acceleration substantially follows the measurement value of the Z-axis.

Next, when the combined acceleration _{A r} is below a predetermined lower threshold value theta _btm, records the resultant acceleration _{A btm} at that time. Then, the resultant acceleration _{A btm} below the lower threshold theta _btm is recorded a predetermined upper limit time in _{t w,} when the combined acceleration _{A r} exceeds a predetermined upper limit threshold value theta _top, records the resultant acceleration _{A top} at that time . By this processing, a period in which the value of the synthetic acceleration changes significantly within a short time is detected.

The difference between the recorded _{A btm} and _{A top} then is equal to or greater than a predetermined shake detection threshold theta _{# 038.} If the difference is larger than the shake detection threshold θ _amp , the number of shakes is counted as 1. Then, reset A _btm and A _top . Also, if the number of oscillations is not counted within the predetermined upper limit period t _w , A _btm and A _top are reset.

  By the above processing, when strong acceleration is applied in a constant direction for a long time, it is not counted as the number of swings, and it is possible to accurately detect body shake in which the direction of acceleration changes rapidly.

  As a method of detecting the number of swings, (1) rotation or long-term change is not detected as a swing, and (2) a certain amount of acceleration is not detected as a swing unless it changes, and (3) which axial direction However, any method can be used without particular limitation as long as it can be detected by the same algorithm and (4) a piezoelectric sensor, that is, a sensor which can not detect angular velocity or the like. As described above, the number of oscillations is calculated regardless of the operation of the synthetic acceleration regardless of the direction in the front-rear direction or the left-right direction. The feature quantity extraction unit 202 according to the first embodiment divides T3 into several small frames and calculates a variance value or an average value based on the number of oscillations in each small frame.

  Further, “the number of swings of each axis” is a number obtained by counting only the axis that most affected the swing when detecting the number of swings. For example, among the components constituting the composite acceleration when the number of swings is counted by the above-described method, the axis where the largest acceleration is measured is detected. Then, the number of vibrations on this axis is counted as 1. For example, when the largest acceleration among the accelerations of the respective axes when the number of swings 1 is counted is detected from the Z axis, it is counted as the number of swings 1 of the Z axis. The shake of the Z axis is detected, for example, when the user is walking or when the user jumps. For this reason, movement such as "walking" can be detected based on the number of swings of the Z axis.

  In addition, since the sequential posture identification device 20 according to the first embodiment employs a machine learning model, weighting for each feature amount is automatically performed.

(One example of the flow of learning process)
FIG. 7 is a schematic diagram showing an example of the flow of learning processing in the sequential posture identification processing by the sequential posture identification device 20 according to the first embodiment. An example of a specific learning process will be described with reference to FIG.

  When performing machine learning first and subsequent machine learning in the posture recognition device 20 sequentially, the user instructs the posture recognition device 20 to start execution of the learning process sequentially by selecting the learning mode, etc. Step S300). Then, the user registers the posture. That is, the user inputs a label. That is, the user identifies the machine learning operation (“name of posture A”), the “posture” of the operation, and which of the “exercise state” and the “rest state” the operation corresponds to Information is sequentially input to the posture identification apparatus 20 ("preparation for registration of posture A", step S301). For example, in the example of FIG. 7, "sit" is input as "the name of posture A". Since the sitting state is the "rest state", the information indicating the "resting" motion state is also input. In the example of FIG. 7, this identifies the posture of "sit". In order to identify more detailed states, the classifications of postures and motion states indicated by the labels are adjusted to correspond. For example, in response to the name “sit”, information “posture: sitting”, “exercise state: stationary” may be input. In addition, in response to the name "jumping", "posture: standing position", "exercise state: up and down movement" or the like may be used. The input may be performed from the input unit 104 of the wearable device 10 or sequentially from the input unit 213 of the posture identification device 20. The labels are input for each of the operations for performing machine learning, and sequentially acquired and stored in the posture identification device 20 (step S302).

  After inputting the label, the user sequentially instructs the posture identification device 20 to start machine learning of the operation corresponding to the input label ("Position A start", step S303). For example, the start button and the end button are sequentially provided in the input unit 213 of the posture identification device 20, and the user presses the start button at the start of machine learning. Then, the user starts an operation corresponding to the input label. By the user's instruction input, the learning mode is sequentially activated in the posture identification device 20 (step S304). Then, the feature quantity extraction unit 202 of the posture identification device 20 sequentially executes feature quantity extraction processing (step S305). The feature amount extraction process is repeated for each predetermined period (T1 to T4) for each feature amount as described above.

  When the feature amount is extracted, the feature amount is sent to the first creating unit 203, the second creating unit 204, the third creating unit 205, and the fourth creating unit 206. Then, based on the input label (e.g., "sit: stop"), the first creating unit 203 sets data of "exercise state" or "rest state" specified by the label (e.g. : If it is "stopped" ("rest state"), machine learning of the input feature amount is performed (step S306). In the example of FIG. 7, machine learning is performed every time T1 seconds in which the feature value is extracted. Note that the feature amount extracted in the extraction cycle shorter than T1 seconds is to be slide input. The execution cycle of machine learning may be set according to the measurement cycle of the acceleration information measurement unit 101 and the biological signal information measurement unit 102.

  Similarly, the second creating unit 204 executes the process based on the input label (for example, "sit: stop"). In the example of FIG. 7, since the input label is “sit: stopped”, ie, “at rest”, the second creating unit 204, which is a component creating the identification model of the operating state, It waits without performing (step S307).

  Similarly, the third creating unit 205 executes a process based on the input label (for example, “sit: stop”). In the example of FIG. 7, since the input label is "sit: stopped", that is, "rest state", the third preparation unit 205 corresponds to "other than the recumbent position" in the "rest state". Machine learning of the input feature amount is performed as the feature amount to be input (step S308).

  Similarly, the fourth preparation unit 206 executes processing based on the input label (for example, “sit: stop”). In the example of FIG. 7, the input label is “sitting: stopped”, that is, “resting state”, and corresponds to “sitting” out of “other than lying”. Therefore, the fourth creating unit 206 performs machine learning of the input feature amount as the feature amount corresponding to the "sitting position" among the "rest state" and the "other than lying position" (step S309).

  When the operation specified by the label is completed on the user side, the posture identification device 20 is sequentially instructed to finish machine learning of the operation ("attitude A end", step S310). For example, the user presses the end button of the input unit 213. In response to the user's instruction, the posture identification apparatus 20 sequentially stops the learning mode (step S311). Then, the user starts preparation for registration of posture B to be machine-learned next (step S312). The above is the flow of an example of the learning process in the posture recognition process.

  The label used in the learning process indicates the posture and motion state of the motion to be identified. For example, "walk", "jump", "sit", "stand up", "becoming prone", "becoming supple", etc. And an indication of whether it is in motion or at rest.

(An example of the flow of identification processing)
FIG. 8 is a schematic diagram showing an example of the flow of identification processing (including attitude identification processing) in the sequential attitude identification processing by the sequential attitude identification device 20 according to the first embodiment. Next, with reference to FIG. 8, a specific example of the posture specifying process and the identifying process will be described. Note that the sequential posture identification apparatus 20 according to the first embodiment executes the learning process of FIG. 7 in parallel even while the posture identification process and the identification process shown in FIG. It can be updated.

  When the identification process is started, the user sequentially inputs an instruction to start the identification process to the posture identification device 20 (step S401). For example, the user makes an input to select the identification mode. In response to the user's input, the posture identification apparatus 20 sequentially activates the identification mode (step S402). When the identification mode is activated, the feature quantity extracted by the feature quantity extraction unit 202 is input to the first specifying unit 207, the second specifying unit 208, the third specifying unit 209, and the fourth specifying unit 210. Then, unlike in the learning mode, the user does not perform designation input of a label, and immediately starts an arbitrary operation ("start of posture A," step S403). For example, the user initiates an action of "sitting". The wearable device 10 worn by the user measures the acceleration information and the biological signal information of the user and sequentially transmits them to the posture identification device 20. Then, the posture identification apparatus 20 sequentially extracts the feature amount based on at least one of the received acceleration information and biological signal information (step S404). As described above, the feature amount extraction process is performed every predetermined period for each feature amount. The extracted feature amount is input to the first specifying unit 207, the second specifying unit 208, the third specifying unit 209, and the fourth specifying unit 210.

  The first specifying unit 207 specifies whether the current feature amount corresponds to the motion state or the still state based on the input feature amount and the motion / still identification model created by the first creating unit 203. Do. In the example of FIG. 8, the first specifying unit 207 specifies that the state corresponding to the input feature amount is the “rest state” (that is, “stops”) (step S405).

  The second identifying unit 208 identifies that the current feature corresponds to “walking” in the motion state based on the input feature and the motion identification model created by the second creating unit 204. (Step S406).

  The third identifying unit 209 determines that the current feature amount corresponds to “other than the recumbent position” in the stationary state based on the input feature amount and the first stationary identification model created by the third creating unit 205. It identifies (step S407).

  The fourth identifying unit 210 is based on the input feature amount and the second stationary identification model created by the fourth creating unit 206, and the current feature amount is “stationary, out of“ stationary state, other than recumbent position ”. "Is identified as" (step S408). In the example of FIG. 8, the posture specifying process is repeatedly executed every T1 seconds.

  When the posture specifying process by the first specifying unit 207, the second specifying unit 208, the third specifying unit 209, and the fourth specifying unit 210 is completed, each specifying result is sent to the determining unit 211. Then, the identification unit 211 integrates the four identification results and outputs a final identification result. Specifically, the identification unit 211 first refers to the identification result by the first identification unit 207 (first stage, step S409). Then, since the identification result by the first identification unit 207 is "stopped", that is, the "rest state", the identification unit 211 identifies the identification result of the third identification unit 209 that executes the identification by the first stationary identification model. "Other than recumbent position" is adopted (second step, step S409). Furthermore, the identification unit 211 refers to the identification result of the fourth identification unit 210 that identifies the subdivision of “other than the decile”. Then, the identification unit 211 adopts the identification result “sitting” of the fourth identification unit 210 (third step, step S409). Thereby, the identification unit 211 identifies the posture and motion state of the user as "sitting position".

  As described above, in the identification process, the identification unit 211 checks which of the “motion state” and the “rest state” the feature amount corresponds to as the first step (the identification result of the first identification unit 207). Then, when the result of the first stage is the “exercise state”, the identification unit 211 selects, as the second stage, the identification result (the identification result of the second identification unit 208) that identifies the type of the exercise state. On the other hand, when the result of the first step is “rest state”, the identification unit 211 specifies one of the classification “rest position” and “other than the rest position” in the rest state (third The identification result of the identification unit 209) is selected as the second stage. Furthermore, when “other than deceiving position” for which the subdivision is set is selected in the second step, the identification unit 211 determines that the subdivision “sitting” or “standing position” (“fourth identification portion”) The specific result of 210) is selected as the third step. On the other hand, when “recumbent position” in which the subdivision is not set is selected in the second stage, the identification unit 211 ends the process in the second stage and selects “recumbent position” as an identification result. As described above, the identification unit 211 proceeds to the selection process step by step until the section for which the lower level section is not set, and outputs the lowest section reached in the selection process as an identification result.

  When the user shifts to another operation, the user does not perform input to the posture identification apparatus 20 one by one in particular, and shifts to a different operation ("end of attitude A, start of attitude B", step S410). The acceleration information and the biological signal information sequentially transmitted to the posture identification device 20 change according to the change in the user's operation, and the feature amount changes. As a result, the posture change is reflected in the result of the identification process (including the posture specifying process) sequentially performed by the posture recognition apparatus 20 every T1.

  As described above, the sequential posture identification device 20 according to the first embodiment identifies the posture and the movement state of the user. Then, the generation unit 212 of the posture identification device 20 sequentially generates information that can be used to determine the user's health condition by correlating the identified posture and motion state with the fluctuation of the biological signal information.

(Modification)
In the above-described sequential posture identification processing, the first specification unit 207 specifies the motion state and the stationary state, the second specification unit 208 specifies the type of motion state, and the third specification unit 209 indicates the decubitus position. The fourth identification unit 210 identifies the standing position and the sitting position. Not limited to this, the types of postures and motion states specified by each specifying unit can be further subdivided and set.

  For example, when creating the identification model, the posture label based on the acceleration information measured at a specific time is machine-trained in distinction from the same posture label based on the acceleration information measured at another time. For example, machine learning is performed by distinguishing “sitting” based on acceleration information measured at 8 o'clock in the morning and “sitting” based on acceleration information measured at 9 o'clock at night. Even if the posture is the same, it is considered that the acceleration information measured in the posture changes depending on the date and time of measurement. For example, in the morning, there is a difference in measured acceleration information between the “standing position” of the time when the user is actively active and the “standing position” when the user is tired and vague during the night it is conceivable that. Therefore, in order to realize a more accurate sequential posture identification process, it is considered useful to execute the process in consideration of the change between the same posture caused by the date and time.

  In addition, it is thought that human movement and movement state also fluctuate due to other factors such as season and climate. Therefore, in addition to creating a posture label according to which time of day, it is the same posture or movement state according to which season, what kind of climate, how much temperature, etc. It is also considered useful to create different labels. In addition, the posture identification device 20 may be configured to execute machine learning based on a label corresponding to the health condition of the user. For example, the sequential posture identification device 20 performs machine learning based on labels such as “Stand before recovery,” “Standing during recovery,” “Stand after recovery,” etc. of the injured user. It may be configured.

  Therefore, as a modification, a plurality of labels are prepared for each of the "standing position" and the "sitting position" specified by the fourth specifying unit 210. FIG. 9 is a diagram for explaining the determination in the sequential posture identification process according to the modification. As shown in FIG. 9, in the modification, the fourth specifying unit 210 is configured to specify standing positions corresponding to different time zones as “standing position A”, “standing position B”, “standing position C”, and the like. Configure Furthermore, the fourth specifying unit 210 is configured to specify a sitting position corresponding to a different time zone as “Siting position A”, “Siting position B”, “Siting position C” or the like. In this case, the fourth creating unit 206 creates an identification model corresponding to each label identified by the fourth identifying unit 210.

  In the example of FIG. 9, when acceleration information of an unknown posture is input ((1) in FIG. 9), the identification unit 211 first identifies whether the identification result of the first identification unit 207 is the stationary state or the motion state. Do. If the identification result of the first identification unit 207 is “exercise state” ((2) in FIG. 9, “operation”), the identification unit 211 determines that the identification result of the second identification unit 208 ((3 in FIG. , "Action A" or "action B"). On the other hand, if the identification result of the first identification unit 207 is “rest state” ((4) in FIG. 9, “rest”), the identification unit 211 identifies the identification result of the third identification portion 209 (FIG. 9). (5) (6), adopt "the recumbent position" or "others". If the identification result of the third identifying unit 209 is "recumbent" ((5) in FIG. 9), the identification unit 211 adopts "recumbent" because there is no lower rank. When the identification result of the third identification unit 209 is "other" ((6) in FIG. 9), the identification unit 211 adopts the identification result of the fourth identification unit 210 that further identifies the lower level category ((6)) (7) and (8) of FIG. The fourth identification unit 210 identifies all the acceleration information identified as "standing position A", "standing position B", and "standing position C" as "standing position", and "seating A", "seating B", "seating C" All the acceleration information identified as "is identified as" sitting ". The identification unit 211 adopts the identification result of the fourth identification unit 210 and outputs it as an identification result.

  In this way, the posture identification device sequentially performs machine learning by performing different types of data at different points in time for each of the labels (for example, "standing position" and "sitting position") that are difficult to identify, and performs identification processing By doing this, it is possible to comprehensively detect even postures and exercise states that are difficult to identify. That is, in order to identify "standing position A", "standing position B" and "standing position C" as the same "standing position" sequentially, the posture recognition device classifies them into the same upper class while considering subtle changes in the posture. It is possible to comprehensively detect possible postures and motion states.

  In this case, although one label is attached to the attitude of the predetermined time zone, the present invention is not limited to this, and when a plurality of same attitudes are detected at predetermined intervals, different labels are always attached. You may make it distinguish.

  In the above example, an example of creating a plurality of labels for standing and sitting positions has been described. However, the present invention is not limited to this, and different labels may be created for each posture and exercise state every time or season. .

(Effects of the first embodiment)
As described above, the sequential posture identification apparatus according to the first embodiment includes the extraction unit, the creation unit, and the identification unit. The extraction unit extracts a first feature amount corresponding to a first predetermined period and a second feature amount corresponding to a second predetermined period from the acceleration information of the user. The creation unit creates a discrimination model for identifying the posture and motion state of the user by machine learning based on the first feature amount. The identification unit is a plurality of sections corresponding to the posture and motion state of the user based on the identification model and the second feature amount, and includes a plurality of sub-divisions corresponding to predetermined dates and times for the same posture and motion state. Of the categories, from the easy to distinguish category to the difficult category to distinguish, by sequentially selecting the category corresponding to the posture and exercise state of the user in the second predetermined period, the user of the second predetermined period Identify posture and motion status. For this reason, the posture identification apparatus can set the category specified by the identification unit flexibly to identify the posture and motion state of the user. Also, the sequential posture identification device identifies the posture and motion state of the user from a plurality of segments corresponding to the posture and motion state of the user including the segments corresponding to the posture and motion state of a predetermined time. For this reason, it is possible to distinguish between the posture and motion state corresponding to a predetermined time and the posture and motion state corresponding to another time to improve identification accuracy. In addition, the posture identification apparatus sequentially selects sections from easy-to-identify sections to difficult-to-identify sections. For this reason, the posture identification apparatus can realize quick identification without increasing the processing time for easily identifiable segments. For this reason, it is possible to quickly obtain an identification result for a segment that is easy to identify.

  Further, in the sequential posture identification apparatus according to the first embodiment, the recognition unit selects the classification from the upper classification to the lower classification in order among the plurality of classifications, thereby determining the posture of the user in the second predetermined period and Identify exercise status. For this reason, the posture identification apparatus can perform identification rapidly, starting from a segment that is easier to identify.

  In addition, the sequential posture identification apparatus according to the first embodiment determines, based on the identification model and the second feature value, one category to which the posture and the motion state of the user in the second predetermined period correspond to a predetermined time. The system further includes four or more identification units that respectively identify from among a plurality of sections including sections corresponding to the posture and the movement state of. And an identification part is a division corresponding to the posture of the user of a 2nd predetermined period, and an exercise state from the division which is easy to distinguish among the division which four or more specific parts specified, to a division which is hard to distinguish. Are sequentially selected in at least three stages, thereby identifying the posture and motion state of the user in the second predetermined period. In this manner, the sequential posture identification device identifies the posture and the motion state in each of the four or more identification units, and selects the identification results in stages. Therefore, it is possible to flexibly set the posture and motion state to be specified in each specifying unit, the order of selection, and the like to improve the identification speed and the efficiency.

  Further, in the sequential posture identification apparatus according to the first embodiment, the identification unit selects the section identified by the four or more identification units in at least three stages. Therefore, the posture recognition apparatus can perform accurate posture recognition with high accuracy while appropriately setting the number of stages to adjust the processing load.

  Further, in the sequential posture identifying apparatus according to the first embodiment, the identifying unit is a first identifying unit that identifies any one of the lying position and the lying position, and the standing position and the sitting position among those other than the lying position. And a second identification unit that identifies any one of the categories. For this reason, the posture recognition apparatus can sequentially perform the posture recognition by selecting the specification results in stages based on the specification results of the plurality of specifying units. Also, by identifying standing positions and sitting positions, which are difficult to distinguish, with the easily identifiable distinction positions other than the easily positioned positions and the lying positions, the sequential posture identifying apparatus identifies with high accuracy in the second identification unit. The processing accuracy in each identifying unit can be adjusted, such as using a model and using a identification model with lower accuracy in the first identifying unit. Therefore, it is possible to improve the identification accuracy as a whole.

  In addition, the second identification unit may select one of a plurality of standing sections respectively corresponding to a plurality of predetermined dates and times and a plurality of segments of a plurality of sitting positions respectively corresponding to a plurality of predetermined dates and times during a second predetermined period. The user's posture and motion state are identified as the corresponding 1 category. For this reason, even if the posture identification device is the same "standing position" or "sitting position", it reflects changes in the posture and motion state depending on time, season, temperature, etc. "Standing" and "sitting" can be identified. For this reason, the posture recognition apparatus can sequentially realize posture recognition by improving identification accuracy even in postures and motion states that are difficult to distinguish.

  In addition, the posture identification apparatus does not process the identification results of the identification unit in parallel, but uses them in combination in multiple stages. For this reason, the posture identification device is configured to specify in advance a posture or a motion state which is difficult to identify because the contrast is small by a dedicated specifying unit, and it is possible to improve the identification accuracy.

  One of the reasons why it is difficult to distinguish between “standing” and “sitting” when trying to classify stationary state into multiple categories is the difference between “standing” and “sitting”, but other attitudes It is considered to be small compared to the difference between “H” and “Sit” or “Sit”. Therefore, if it is attempted to sequentially distinguish a plurality of labels, for example, "standing position", "sitating position" and "throwing position" based on machine learning, it becomes difficult to distinguish between "standing position" and "siting position". Become. That is, the difference between the "standing position" and the "sitting position", which is relatively smaller than the difference between the "standing position" or the "sitting position", is relatively hard to be identified. In the above embodiment, identification accuracy is improved by specifying combinations of sections having relatively small differences relatively to each other in a specific unit different from the other sections.

(Other modifications)
As described above, in the first embodiment, four creation units for creating a discrimination model for identifying the posture and exercise state of the user are provided, and the identification unit for specifying the posture and exercise state of the user is Provided. However, the number of creation units and identification units is not limited to four, and five or more creation units and identification units may be provided to finely identify the posture and motion state. Further, the number of stages of the identification process is not necessarily limited to three, and can be set to any number of stages according to the number of specific units.

  In the first embodiment, the identification unit that distinguishes exercise from rest, the identification unit that identifies the type of exercise, the identification unit that identifies the recumbent position and the position other than the repose position among the rest, and the other than the recumbent position A specific section to identify standing position and sitting position was provided. Not only this but the specific part for mutually distinguishing arbitrary two or more especially difficult divisions can be provided.

  In addition, in the case where posture identification is sequentially performed for the purpose of nursing care or rehabilitation, the user may be able to set classification of a posture or exercise state to which attention is particularly focused.

  Moreover, the selection order at the time of selecting the specific result of a specific part in steps is not necessarily limited. In the first embodiment, the selection order is set such that, in the first stage, sections that are easy to identify with each other are selected in the second stage. However, it is also possible to set the selection order based on criteria other than the difficulty of identification.

(Example)
FIG. 10 is a diagram for explaining the improvement in accuracy when the sequential posture identification method according to the embodiment is used. The effect of the sequential posture identification method according to the present embodiment will be described with reference to FIG.

  FIG. 10 shows experimental results of sequential posture identification performed for seven users “u01”, “u02”, “u03”, “u04”, “u05”, “u06”, and “u07”.

  In FIG. 10, data (1) on the left side is provided with three identification parts, and in each of (A) identification of exercise state or rest state, (B) identification of exercise type, (C) rest type We show experiment result when we perform identification. In this case, if "exercise state" is specified in (A) as the posture recognition processing result sequentially, the specific result of (B) is adopted as the identification result, and "static state" is specified in (A). (C) was adopted.

  In FIG. 10, central data (2) is provided with four identification parts as in the first embodiment, and in each of (A) identification of exercise state or rest state, (B) exercise type An experiment result at the time of performing specification (C) specification of recumbency or others and (D) specification of standing position or sitting position is shown. In this case, if "exercise state" is specified in (A) as the posture recognition processing result sequentially, the specification result of (B) is adopted as the identification result. In addition, the identification result of (C) is adopted if it is identified as "static state" in (A), and the identification result of (D) is adopted if it is identified as other than decubitus in (C).

  In FIG. 10, data (3) on the right side shows experimental results in the case where labels of different time zones for “standing” and “sitting” are distinguished as different labels in addition to the case of the center. That is, it is an experimental result corresponding to the above-mentioned modification.

  In FIG. 10, the positive discrimination rate indicates the ratio of correct discrimination results to the number of executions of the sequential posture discrimination processing. The misclassification rate indicates the rate at which an erroneous discrimination result is obtained with respect to the number of executions of the sequential attitude discrimination processing. Further, the recall rate indicates the rate at which the answer that should be obtained actually is obtained among the answers that should be obtained as a result. For example, of the postures to be identified as "sitting", the ratio of the postures actually determined to be "sitting" is shown. Moreover, a precision rate shows the ratio of what was correct among the obtained results. For example, of the postures identified as "sitting", the ratio of the posture that was actually "sitting" is shown. The F value is the harmonic mean of precision and recall. It can be said that the higher the F value, the better the balance between precision and recall, and higher discrimination accuracy.

  In the example of FIG. 10, in each case of the above (1) (2) (3), the positive discrimination rate, the erroneous discrimination rate, the recall rate, the accuracy rate, and the F value for “standing” and “sitting” for each user Calculated.

  When "standing position" is identified, the F value average of (1) is "0.779". On the other hand, the F value average of (2) is greatly improved to "0.840". Furthermore, the F value average of (3) is further improved to "0.857".

  Moreover, when "Sitling" is identified, the F value average of (1) is "0.697". On the other hand, the F value average of (2) is also improved to "0.776". Furthermore, the F value average of (3) is further improved to "0.814".

  As described above, the creating unit and the identifying unit for identifying the difficult-to-identify sections (labels) are provided in the creating unit and the identifying unit of the sequential attitude identifying device, and the posture and motion state are identified in multiple stages. It has been confirmed that the identification accuracy can be improved. Moreover, it was confirmed that it is possible to further improve the identification accuracy by dividing the label set as the subdivision into a plurality of labels according to the time zone in which the acceleration information is measured.

(program)
FIG. 11 is a diagram illustrating that information processing by the sequential posture identification program according to the disclosed technology is specifically realized using a computer. As illustrated in FIG. 11, the computer 1000 includes, for example, a memory 1010, a CPU 1020, a hard disk drive 1080, and a network interface 1070. The components of the computer 1000 are connected by a bus 1100.

  The memory 1010 includes a read only memory (ROM) 1011 and a random access memory (RAM) 1012 as illustrated in FIG. The ROM 1011 stores, for example, a boot program such as a BIOS (Basic Input Output System).

  Here, as illustrated in FIG. 11, the hard disk drive 1080 stores, for example, an OS (Operating System) 1081, an application program 1082, a program module 1083, and program data 1084. That is, the sequential posture identification program according to the embodiment of the disclosure is stored, for example, in the hard disk drive 1080 as a program module 1083 in which an instruction to be executed by a computer is described. For example, the feature amount extraction unit 202, the first generation unit 203, the second generation unit 204, the third generation unit 205, the fourth generation unit 206, the first identification unit 207, the second identification unit 208, The hard disk drive 1080 stores a program module 1083 in which procedures for performing information processing similar to those of the third specifying unit 209, the fourth specifying unit 210, the identifying unit 211, and the generating unit 212 are described.

  Also, the identification model created by the first creation unit 203, the second creation unit 204, the third creation unit 205, and the fourth creation unit 206, the identification result by the identification unit 211, and the information generated by the generation unit 212. As described above, data used for information processing by the posture identification program is stored as program data 1084 in the hard disk drive 1080, for example. Then, the CPU 1020 reads the program module 1083 and program data 1084 stored in the hard disk drive 1080 into the RAM 1012 as necessary, and executes various procedures.

  The program module 1083 and the program data 1084 related to the sequential posture identification program are not limited to the case of being stored in the hard disk drive 1080. For example, the program module 1083 and the program data 1084 may be stored in a removable storage medium. In this case, the CPU 1020 reads data via a removable storage medium such as a disk drive. Similarly, the program module 1083 and program data 1084 related to the sequential attitude identification program are stored in another computer connected via a network (LAN (Local Area Network), WAN (Wide Area Network), etc.) It is also good. In this case, the CPU 1020 reads various data by accessing another computer via the network interface 1070.

(Others)
The sequential posture identification program described in the present embodiment can be distributed via a network such as the Internet. The sequential orientation identification program can also be executed by being recorded on a computer readable recording medium such as a hard disk, a flexible disk (FD), a CD-ROM, an MO, a DVD, etc. and reading from the recording medium by a computer. .

  Of the processes described in the present embodiment, all or part of the process described as being automatically performed may be manually performed, or the process described as being performed manually. All or part of them can be automatically performed by a known method. In addition to the above, the processing procedures, control procedures, specific names, and information including various data and parameters shown in the above documents and drawings can be arbitrarily changed unless otherwise specified.

  The above embodiments and the modifications thereof are included in the invention described in the claims and the equivalents thereof as well as included in the technology disclosed in the present application.

DESCRIPTION OF SYMBOLS 1 sequential posture identification system 10 wearable device 101 acceleration information measurement unit 102 biological signal information measurement unit 103 transmission / reception unit 104 input unit 20 sequential posture identification apparatus 201 transmission / reception unit 202 feature amount extraction unit 203 first creation unit 204 second creation unit 205 third creation unit 206 fourth creation unit 207 first identification unit 208 second identification unit 209 third identification unit 210 fourth identification unit 211 identification unit 212 generation unit 213 input unit

Claims (5)

An extraction unit for extracting a first feature corresponding to a first predetermined period and a second feature corresponding to a second predetermined period from the acceleration information of the user;
A creation unit that creates a discrimination model for identifying the posture and motion state of the user by machine learning based on the first feature amount;
Among a plurality of divisions including a subdivision corresponding to a predetermined date and time, one division to which the posture and motion state of the user of the second predetermined period correspond based on the identification model and the second feature amount. And 4 or more specific parts each specified from
Wherein based on the identification model and the second feature quantity, a plurality of segments corresponding to the posture and motion state of the user, the same posture and the motion state of the plurality including a subdivision corresponding to the predetermined time Of the categories specified by the four or more specific parts of the categories, from the easy to identify category to the hard to identify category, the order corresponding to the posture and motion state of the user in the second predetermined period is ordered An identification unit that identifies the posture and motion state of the user during the second predetermined period by selecting
Equipped with
The creation unit is
A first creation unit for creating a motion / static discrimination model for identifying a user's motion state and stationary state;
A second creation unit that creates an exercise identification model for identifying the type of exercise state when the user is in an exercise state;
A third creation unit that creates a first stationary identification model that identifies whether the type of the stationary state is recumbent or non-recumbent when the user is in a stationary state;
A fourth creation unit that creates a second stationary identification model for identifying whether the user is standing or sitting when the user's posture is other than lying down;
Have
The identification unit is
A first identification unit that identifies whether the user is in an exercise state or a rest state using the exercise / still identification model;
A second identification unit that identifies what kind of exercise the user is doing when he or she is in motion using the exercise identification model;
A third identification unit that identifies whether the posture of the user is in a recumbent position or a non-recumbent position using the first stationary identification model;
A fourth identification unit that identifies whether the user's posture is standing or sitting using the second stationary identification model;
Have
The identification unit
When the identification result of the first identification unit is an exercise state, the identification result of the second identification unit is selected and output as an identification result, while the identification result of the first identification portion is stationary With reference to the identification result of the third identification unit, if the identification result of the third identification portion is recumbent, it outputs the recumbency as an identification result, and the identification result of the third identification portion is reversion if not, the sequential orientation identification device for an output to said Rukoto a standing or sitting position indicated by the specific results of the fourth specific portion as an identification result.   The identification unit identifies the posture and the movement state of the user in the second predetermined period by sequentially selecting a category from the upper category to the lower category among the plurality of categories. The sequential posture identification device according to 1. The fourth identification unit is
Any of a plurality of standing subdivisions respectively corresponding to a plurality of predetermined dates and times and a plurality of sitting subdivisions respectively corresponding to a plurality of predetermined dates and times, the posture and the movement state of the user of the second predetermined period The sequential posture identification apparatus according to claim 1 , wherein the sequential posture identification device is identified as a corresponding one of the categories. An extraction step of extracting a first feature corresponding to a first predetermined period and a second feature corresponding to a second predetermined period from the acceleration information of the user;
Creating a discrimination model for identifying the posture and motion state of the user by machine learning based on the first feature amount;
Among a plurality of divisions including a subdivision corresponding to a predetermined date and time, one division to which the posture and motion state of the user of the second predetermined period correspond based on the identification model and the second feature amount. And four or more identification steps, each identifying
Wherein based on the identification model and the second feature quantity, a plurality of segments corresponding to the posture and motion state of the user, the same posture and the motion state of the plurality including a subdivision corresponding to the predetermined time Of the classifications identified by the four or more identification steps, the classifications that correspond to the user's posture and exercise state in the second predetermined period from the classifications that are easy to distinguish to the classifications that are difficult to distinguish Identifying, by sequentially selecting, the posture and motion state of the user during the second predetermined period;
On your computer ,
The preparation process is
A first creating step of creating a motion / static discrimination model for identifying a user's motion state and stationary state;
A second creating step of creating an exercise identification model for identifying the type of exercise state when the user is in an exercise state;
A third creating step of creating a first stationary identification model that identifies whether the type of the stationary state is recumbent or non-recumbent when the user is in a stationary state;
A fourth creating step of creating a second stationary identification model for identifying whether the user is standing or sitting when the user's posture is other than lying down;
Including
The specific step is
A first identification step of identifying whether the user is in an exercise state or a rest state using the exercise / stationary discrimination model;
A second identification step of identifying the type of exercise performed by the user using the exercise identification model when the user is in an exercise state;
A third identification step of identifying whether the posture of the user is in a recumbent position or a non-recumbent position using the first stationary identification model;
A fourth identification step of identifying whether the posture of the user is standing or sitting using the second stationary identification model;
Including
In the identification step,
When the identification result of the first identification step is an exercise state, the identification result of the second identification step is selected and output as an identification result, while the identification result of the first identification step is stationary With reference to the identification result of the third identification step, if the identification result of the third identification step is a recumbent position, the decubitus position is output as an identification result, and the identification result of the third identification step is a reversion position. if not, the sequential orientation identification method, wherein also be output from the standing or sitting position indicated by the specific results of the fourth identification step as an identification result. An extraction procedure for extracting a first feature corresponding to a first predetermined period and a second feature corresponding to a second predetermined period from the acceleration information of the user;
A creation procedure for creating a discrimination model for identifying the posture and motion state of the user by machine learning based on the first feature amount;
Among a plurality of divisions including a subdivision corresponding to a predetermined date and time, one division to which the posture and motion state of the user of the second predetermined period correspond based on the identification model and the second feature amount. And 4 or more specific procedures to identify from
Wherein based on the identification model and the second feature quantity, a plurality of segments corresponding to the posture and motion state of the user, the same posture and the motion state of the plurality including a subdivision corresponding to the predetermined time Of the classifications identified by the four or more identification procedures, the classifications that correspond to the user's posture and exercise state in the second predetermined period from the classifications that are easy to distinguish to the classifications that are difficult to distinguish An identification procedure for identifying the posture and motion state of the user during the second predetermined period by selecting in order;
On your computer ,
The creation procedure is
A first creation procedure for creating a motion / stationary discrimination model for identifying a user's motion state and stationary state;
A second creation procedure for creating an exercise identification model for identifying the type of exercise state when the user is in an exercise state;
A third creation procedure for creating a first stationary identification model that identifies whether the type of the stationary state is recumbent or non-recumbent when the user is in a stationary state;
A fourth creation procedure for creating a second stationary identification model for identifying whether the user is standing or sitting when the user's posture is other than lying down;
Including
The specific procedure is
A first identification procedure for identifying whether the user is in an exercise state or a rest state using the exercise / still discrimination model;
A second identification procedure for identifying the type of exercise performed by the user using the exercise identification model when the user is in an exercise state;
A third identification procedure for identifying whether the posture of the user is the recumbent position or the non-recumbent position using the first stationary identification model;
A fourth identification procedure for identifying whether the posture of the user is standing or sitting using the second stationary identification model;
Including
The identification procedure is
When the identification result of the first identification procedure is an exercise state, the identification result of the second identification procedure is selected and output as an identification result, while the identification result of the first identification procedure is stationary Then, referring to the specific result of the third specific procedure, if the specific result of the third specific procedure is a recumbent position, the recumbency is output as an identification result, and the specific result of the third specific procedure is a negative position if not, the sequential orientation identification program to output to said Rukoto a standing or sitting position indicated by the specific result of the fourth determination procedure as identification result.

Images