KR101733746B1

KR101733746B1 - User context based motion counting method, sensor device and wearable device performing the same

Info

Publication number: KR101733746B1
Application number: KR1020150148565A
Authority: KR
Inventors: 마이클 보글; 권성호; 이종성
Original assignee: 주식회사 스탠딩에그
Priority date: 2015-10-26
Filing date: 2015-10-26
Publication date: 2017-05-08

Abstract

A user context based motion counting method, a sensor device for performing the motion counting method, and a wearable device are provided. Wherein the user context based motion counting method comprises: extracting at least one feature from sensor data; identifying a user context based on the extracted feature; determining a motion count based on the identified user context; Determining an algorithm based on the sensor data and using the determined motion counting algorithm to count a user's movement based on the sensor data.

Description

USER CONTEXT BASED MOTION COUNTING METHOD, AND SENSOR DEVICE AND WEARABLE DEVICE PERFORMING THE SAME Field of the Invention < RTI ID = 0.0 >

The present invention relates to a method of counting a user's movement, and more particularly, to a user context-based motion counting method, a sensor device and a wearable device performing the motion counting method.

A wearable device represents a computer system that is provided in a form that can be worn on the wearer's body. The wearable device can measure the movement of the user by using various sensors and provide the measurement result to the user. The wearable device can manually input a user's motion operation and count the number of times the input motion motion is repeated. The wearable device monitors the movement of the user at a specific period of time and can provide additional information such as the amount of calories consumed and the activity time.

Korean Patent Laid-Open No. 10-2009-0039124, Apr. 22, 2009

SUMMARY OF THE INVENTION It is an object of the present invention to provide a user context-based motion counting method that can more accurately count user's movements.

It is another object of the present invention to provide a user context-based motion counting method that can count user's movements more intuitively and conveniently.

Another object of the present invention is to provide a sensor device and a wearable device for performing the user context based motion counting method.

The technical objects of the present invention are not limited to the above-mentioned problems, and other matters not mentioned can be clearly understood by those skilled in the art from the following description.

According to an aspect of the present invention, there is provided a method of counting motion based on a user context, the method comprising: extracting at least one feature from sensor data; determining a user context based on the extracted feature; Determining a motion counting algorithm based on the identified user context, and counting a user's movement based on the sensor data using the determined motion counting algorithm.

In some embodiments of the present invention, the extracted feature may comprise at least one statistical value of the sensor data.

In some embodiments of the present invention, identifying the user context may identify the user context based on the extracted feature using a K-Nearest Neighborhood (KNN) algorithm.

In some embodiments of the present invention, the step of determining the motion counting algorithm comprises determining an autocorrelation parameter of a motion counting algorithm based on the identified user context, Parameters, and determine a second autocorrelation parameter for a second user context.

Also, the autocorrelation parameter may correspond to a repetition period of a peak included in the sensor data.

In some embodiments of the present invention, the step of determining the motion counting algorithm comprises determining a first motion counting algorithm for a first user context based on the identified user context, and determining a second motion counting algorithm for a second user context, The counting algorithm can be determined.

In some embodiments of the present invention, the motion counting algorithm comprises filtering for sensor data in a predetermined frequency range of the sensor data, and wherein the step of determining the motion counting algorithm comprises, based on the identified user context Determine the filtering setting, determine a first filtering setting for the first user context, and determine a second filtering setting for the second user context.

In some embodiments of the present invention, the method may further comprise obtaining the sensor data from an acceleration sensor or a gyro sensor.

According to another aspect of the present invention, there is provided a sensor device capable of performing any one of the above-described user context based motion counting methods.

According to another aspect of the present invention, there is provided a wearable apparatus capable of performing any one of the above-described user context-based motion counting methods.

Other specific details of the invention are included in the detailed description and drawings.

According to the user context based motion counting method of the present invention, a user context is identified and a more suitable motion counting algorithm is determined for each identified user context, so that the motion of the user can be counted more accurately.

In addition, since direct input or selection of the user is not required separately for user context identification, it is more intuitive and user-friendly.

The effects of the present invention are not limited to the above-mentioned effects, and other effects not mentioned can be clearly understood by those skilled in the art from the following description.

FIG. 1 is a flowchart schematically illustrating a user context-based motion counting method according to an embodiment of the present invention.
2 is a diagram schematically illustrating identification of a user context using a KNN algorithm.
3 is a diagram schematically illustrating determination of autocorrelation parameters based on a user context.
4 is a diagram schematically illustrating determining filtering settings for sensor data based on a user context.
5 is a block diagram schematically illustrating a wearable device for performing a user context-based motion counting method according to an embodiment of the present invention.
FIG. 6 is a view schematically showing a state in which the wearable device of FIG. 5 is worn by the wearer's body.
7 is a block diagram schematically illustrating a wearable device for performing a user context-based motion counting method according to another embodiment of the present invention.
FIG. 8 is a block diagram schematically illustrating a sensor hub for performing a user context-based motion counting method according to an embodiment of the present invention.
9 is a block diagram schematically illustrating a sensor hub for performing a user context-based motion counting method according to another embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention, and the manner of achieving them, will be apparent from and elucidated with reference to the embodiments described hereinafter in conjunction with the accompanying drawings. However, it is to be understood that the present invention is not limited to the disclosed embodiments, but may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. It is to be understood by those of ordinary skill in the art that the present invention is not limited to the above embodiments, but may be modified in various ways. Like reference numerals refer to like elements throughout the specification.

Although the first, second, etc. are used to describe various elements, components and / or sections, it is needless to say that these elements, components and / or sections are not limited by these terms. These terms are only used to distinguish one element, element or section from another element, element or section. Therefore, it goes without saying that the first element, the first element or the first section mentioned below may be the second element, the second element or the second section within the technical spirit of the present invention.

Unless defined otherwise, all terms (including technical and scientific terms) used herein may be used in a sense commonly understood by one of ordinary skill in the art to which this invention belongs. In addition, commonly used predefined terms are not ideally or excessively interpreted unless explicitly defined otherwise.

The terminology used herein is for the purpose of illustrating embodiments and is not intended to be limiting of the present invention. In the present specification, the singular form includes plural forms unless otherwise specified in the specification. The terms " comprises "and / or" comprising "used in the specification do not exclude the presence or addition of one or more other elements in addition to the stated element.

FIG. 1 is a flowchart schematically illustrating a user context-based motion counting method according to an embodiment of the present invention.

Referring to FIG. 1, in step S10, sensor data is obtained from an acceleration sensor, a gyro sensor, or at least one sensor to which they are coupled. The sensor data may include acceleration data in three axial directions (e.g., X, Y, and Z axes) or acceleration data in three axial directions (e.g., a pitch axis, a yaw axis, But is not limited to, angular velocity data. Depending on the embodiment, processing such as predetermined sampling or quantization of the sensor data may be provided.

Then, in step S20, at least one feature is extracted from the sensor data. The feature indicates the aspect or component in which the sensor data is differentiated from the other sensor data. The feature can represent the whole part of the sensor data or represent part of the sensor data. For example, the feature may include at least one statistical value of the sensor data. For example, the statistical value may include a maximum value, a minimum value, an intermediate value, an average value, an IQR (Interquartile Range) value, a root mean square (RMS) value, and the like of a predetermined number of sampled sensor data , But is not limited thereto. The feature may be extracted by arbitrary processing on the sensor data.

Then, in step S30, the user context is identified based on the extracted features. The user context represents a user's situation or information defining the situation. The user context may be related to the type of action that can specify the user's movement. For example, the user context may be classified into various types of exercise operations such as push-up, biceps curls, and rowing, but is not limited thereto.

For example, the user context can be identified using the KNN algorithm. 2 is a diagram schematically illustrating identification of a user context using a KNN algorithm. Referring to FIG. 2, a plurality of features are processed using the KNN algorithm. By the KNN algorithm, a plurality of features can be compared with pre-classified or learned features and classified into groups having the same or similar tendencies. By combining the classification results, a specific user context can be derived. the k value, and the distance between neighboring features, and the like can be variously adjusted according to the embodiment. Although not explicitly shown, a weight relating to high importance and distance to some features may be applied. The detailed description of the KNN algorithm is omitted because it may obscure the gist of the present invention.

On the other hand, for the identification of the user context, any unillustrated machine learning algorithm may be used.

Then, in step S40, a motion counting algorithm is determined based on the identified user context. At this time, the autocorrelation parameters of the motion counting algorithm may be determined together based on the identified user context. The autocorrelation parameters can be determined differently depending on the user context. That is, a first autocorrelation parameter may be determined for a first user context, and a second autocorrelation parameter may be determined for a second user context. The autocorrelation parameter may be used to remove noise or counting errors within the motion counting algorithm.

For example, the autocorrelation parameter may correspond to a repetition period of the peaks included in the sensor data. The peak may be an argument or an element for counting the user's movement as described below. 3 is a diagram schematically illustrating determination of autocorrelation parameters based on a user context. Referring to FIG. 3, a plurality of peaks p0 to p2 may be included in sensor data in a predetermined time range. The peak can be selected based on the time interval between the amplitude and the previous peak. On the other hand, the repetitive pattern of peaks may be different depending on the user context (user's action). For example, when the user performs the push-up operation, when the repetition period of the peaks, that is, the time interval between the previous peak and the current peak is 100 msec, considering the repetition pattern of the general peak appearing in the push- , The current peak is presumed to be due to noise or error, and is not used for counting. In the example shown in FIG. 3, if the reference time interval with respect to the repetition period of the peak is tr1, then p1 can be selected as the peak for counting after p0, but if the reference time interval is tr2, And p2 may be selected as the peak for counting.

At least some motion counting algorithms may include filtering for sensor data in a predetermined frequency range of sensor data to remove noise or counting errors. And, when the motion counting algorithm is determined, the filtering setting may be determined together based on the identified user context. The filtering settings can be determined differently depending on the user context. That is, a first filtering setting may be determined for a first user context, and a second filtering setting may be determined for a second user context.

4 is a diagram schematically illustrating determining filtering settings for sensor data based on a user context. Referring to FIG. 4, a plurality of filtering settings are shown that define a predetermined passband. In the example shown in Fig. 4, it is confirmed that fl1 and fh1 are set to the cut-off frequency according to the first filtering setting, and pass-bands when fl2 and fh2 are set to the cut-off frequency according to the second filtering setting . Although a band-pass filter is shown as an example in FIG. 4, the present invention is not limited to this, and a filtering setting may be provided in substantially the same manner for a high-pass filter and a low-pass filter.

On the other hand, a separate motion counting algorithm may be determined based on the identified user context (in the case of certain operations that move in a completely different way than other operations). In this case, a first motion counting algorithm may be determined for a first user context, and a second motion counting algorithm may be determined for a second user context.

Subsequently, in step S50, the motion of the user is counted based on the sensor data, using the determined motion counting algorithm. For example, the sensor data of the axis having the largest change among the sensor data of the three axes is selected, and the peaks in the sensor data of the selected axis can be counted. The method of counting movements can be variously modified according to specific embodiments.

5 is a block diagram schematically illustrating a wearable device for performing a user context-based motion counting method according to an embodiment of the present invention.

5, the wearable apparatus 100 includes an acceleration sensor 110, a gyro sensor 120, a storage unit 130, an input unit 140, an output unit 150, a control unit 160, a power supply unit 170 ).

The acceleration sensor 110 detects acceleration in three axial directions (e.g., X-axis, Y-axis, and Z-axis). Although only one acceleration sensor is shown in FIG. 5, the present invention is not limited thereto, and a plurality of acceleration sensors capable of detecting acceleration in one axial direction may be provided. The acceleration sensor 110 may transmit the acceleration data in the three axial directions to the controller 160.

The gyro sensor 120 detects the angular velocity in three axial directions (for example, a pitch axis, a yaw axis, and a roll axis). Although only one gyro sensor is shown in Fig. 5, the present invention is not limited thereto, and a plurality of gyro sensors each capable of detecting an angular velocity in the axial direction may be provided. The gyro sensor 120 can transmit angular velocity data in three axial directions to the controller 160. [

The storage unit 130 stores various data and commands. The storage unit 130 may store various software modules including system software for operation of the wearable device 100 and applications in which a user context based motion counting method according to an embodiment of the present invention is implemented. The storage unit 130 may be any of a variety of storage devices such as a random access memory (RAM), a read only memory (ROM), an erasable-programmable ROM (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a removable disk, And may include any form of computer readable recording medium.

The input unit 140 receives various information from the user. The input unit 140 may include various input means such as a key, a button, a switch, a wheel, and a touch pad.

The output unit 150 notifies the user of various types of information. The output unit 150 may output information in the form of text, image, or voice. To this end, the output unit 150 may include a display module 151 and a speaker module 152. The display module 151 may be a liquid crystal display (LCD), a thin film transistor (TFT) LCD, an organic light emitting diode (OLED), a flexible display, a three-dimensional display, an electronic ink display, And may be provided in any form.

The controller 160 controls the overall operation of the wearable device 100 by controlling other components. The control unit 160 may execute various software modules including system software for the operation of the wearable device 100 and an application in which a user context based motion counting method according to an embodiment of the present invention is implemented. The controller 160 may filter the sensor data transmitted from the sensors 110 and 120, including a high pass filter 161, a low pass filter 162 and a band pass filter 163. The control unit 160 may filter at least one of the plurality of filters 161 to 163 to filter the sensor data. The high pass filter 161, the low pass filter 162 and the band pass filter 163 may be provided as separate components outside the control unit 160 or as internal components of the respective sensors 110 and 120 May be provided.

The power supply unit 170 supplies power necessary for the operations of the acceleration sensor 110, the gyro sensor 120, the storage unit 130, the input unit 140, the output unit 150, and the controller 160. The power supply unit 170 may include an internal battery or may convert a power supplied from the outside into a power suitable for the components.

On the other hand, the components shown in Fig. 5 are not essential, so that the wearable device 100 can include more components or can be modified to include fewer components.

FIG. 6 is a view schematically showing a state in which the wearable device of FIG. 5 is worn by the wearer's body.

Referring to FIG. 6, the wearable device 100 described with reference to FIG. 5 may be worn on the wearer's body. For example, the wearable device 100 may be worn on the limbs, such as the user's arms or legs, or worn on the torso of the user, but is not limited thereto and may include other parts (head, hands, And can be worn on at least a part of the user's body. Alternatively, the wearable device 100 may be provided inside any item (hat, glove, shoes, etc.) worn on the wearer's body.

7 is a block diagram schematically illustrating a wearable device for performing a user context-based motion counting method according to another embodiment of the present invention. For convenience of description, the same components as those of the wearable device 100 described with reference to FIG. 5 will not be described.

7, the wearable device 200 further includes a wireless communication unit 230 and a vibration unit 260 as compared with the wearable device 100 described with reference to FIG.

The wireless communication unit 230 may wirelessly communicate with an external device (such as various servers or user terminals). The wireless communication unit 230 can wirelessly communicate with an external device using a wireless communication method such as mobile communication, WiBro, WiFi, Bluetooth, Zigbee, ultrasonic, infrared, and RF have. The wireless communication unit 230 may transmit the data and / or information received from the external device to the control unit 280 and may transmit the data and / or information transmitted from the control unit 280 to the external device. For this, the wireless communication unit 230 may include a mobile communication module, a short-range communication module, and the like.

The vibration unit 260 may perform a vibration notification for notifying the user of various information.

The wearable device 200 can transmit and receive various information such as sensor data, user context, motion counting algorithm, and motion counting result to various servers or user devices (not shown).

A wearable device according to an embodiment of the present invention may be provided in any computer system that can be worn on the body of an unillustrated user.

FIG. 8 is a block diagram schematically illustrating a sensor hub for performing a user context-based motion counting method according to an embodiment of the present invention.

Referring to FIG. 8, the sensor hub 1000 may include a processing apparatus 1100, a MEMS apparatus 1200, and an application specific integrated circuit (ASIC) apparatus 1300. The MEMS device 1200 may be, but is not limited to, an acceleration sensor or a gyro sensor. The ASIC device 1300 may process the sensing signal of the MEMS device 1200. The processing device 1100, on behalf of the application processor, may function as a coprocessor to professionally perform sensor data processing. The processing device 1100 may perform various software modules including an application in which a user context based motion counting method according to an embodiment of the present invention is implemented.

9 is a block diagram schematically illustrating a sensor hub for performing a user context-based motion counting method according to another embodiment of the present invention.

Referring to FIG. 9, the sensor hub 2000 may include a plurality of MEMS devices 2200 and 2400 and a plurality of ASIC devices 2300 and 2500. The first MEMS device 2200 may be an acceleration sensor and the second MEMS device 2400 may be a gyro sensor, but is not limited thereto. The plurality of ASIC devices 2300 and 2500 can process the sensing signals of the corresponding MEMS devices 2200 and 2400, respectively. The processing device 2100, on behalf of the application processor, may serve as a coprocessor for professionally performing sensor data processing. The processing device 2100 may perform various software modules including an application in which a user context based motion counting method according to an embodiment of the present invention is implemented. As shown, three or more MEMS devices and ASIC devices may be provided in the sensor hub 2000.

The methods described in connection with the embodiments of the present invention may be implemented with software modules executed by a processor. The software modules may reside in RAM, ROM, EPROM, EEPROM, flash memory, hard disk, removable disk, CD-ROM, or any form of computer readable recording medium known in the art .

While the present invention has been described in connection with what is presently considered to be practical exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, It will be understood. Therefore, it should be understood that the above-described embodiments are illustrative in all aspects and not restrictive.

Claims

Extracting at least one feature from the sensor data;
Identifying a user context based on the extracted features;
Determining a motion counting algorithm based on the identified user context; And
Counting a user's movement based on the sensor data using the determined motion counting algorithm,
Wherein determining the motion counting algorithm comprises:
Determining an autocorrelation parameter of a motion counting algorithm based on the identified user context, determining a first autocorrelation parameter for a first user context, and determining a second autocorrelation parameter for a second user context Wherein the user context-based motion counting method comprises:

The method according to claim 1,
The extracted feature may be,
And at least one statistical value of the sensor data.

The method according to claim 1,
Wherein identifying the user context comprises:
Wherein the user context is identified based on the extracted feature using a KNN (K-Nearest Neighborhood) algorithm.

delete

The method according to claim 1,
The autocorrelation parameter may comprise:
Corresponding to a repetition period of a peak included in the sensor data.

The method according to claim 1,
Wherein determining the motion counting algorithm comprises:
Determine a first motion-counting algorithm for a first user context based on the identified user context, and determine a second motion-counting algorithm for a second user context.

The method according to claim 1,
The motion counting algorithm includes:
And filtering the sensor data in a predetermined frequency range of the sensor data,
Wherein determining the motion counting algorithm comprises:
Determining a filtering setting for sensor data in a predetermined frequency range of the sensor data based on the identified user context, determining a first filtering setting for a first user context, and determining a second filtering setting for a second user context, And determining a setting based on the user context.

The method according to claim 1,
Further comprising the step of obtaining the sensor data from an acceleration sensor or a gyro sensor.

9. A sensor device for performing a user context-based motion counting method according to any one of claims 1 to 3 and 5 to 8.

10. A wearable apparatus for performing a user context-based motion counting method according to any one of claims 1 to 3, and 5 to 8.