WO2015084082A1

WO2015084082A1 - Method, device and system for recognizing posture or motion, and non-temporary computer-readable recording medium

Info

Publication number: WO2015084082A1
Application number: PCT/KR2014/011851
Authority: WO
Inventors: 김선혜; 황성재
Original assignee: 주식회사 퓨처플레이
Priority date: 2013-12-04
Filing date: 2014-12-04
Publication date: 2015-06-11
Also published as: KR20160108126A

Abstract

According to an aspect of the present invention, a method for recognizing a posture or motion is provided, the method comprising the steps of: acquiring information on a direction or motion of a first device which is worn on a first part of a user's body and a direction or motion of a second device which is worn on a second part of the user's body; and recognizing a posture which the user takes or a motion which the user does, with reference to a relative relationship between the direction or motion of the first device and the direction or motion of the second device.

Description

Method, device, system and non-transitory computer readable recording medium for recognizing posture or motion

The present invention relates to methods, devices, systems, and non-transitory computer readable recording media for recognizing posture or motion.

In recent years, mobile smart devices having various communication and sensing functions and powerful computing functions such as smart phones and smart pads have been widely used. Among such mobile smart devices, wearable devices (eg, smart glasses, smart watches, smart bands, smart devices in the form of rings or brooches) that can be worn and carried by the user directly on the body or clothing Smart devices that are attached or embedded).

Recently, a technology for recognizing a posture or a motion of a user or a type of a user's behavior using a wearable device that can be worn on a user's body has been introduced. However, according to the conventional technology introduced so far, there are limitations such as not only implementing a pedometer-level function of recognizing simple and repetitive movements such as walking or not accurately recognizing delicate postures and complicated movements. .

The object of the present invention is to solve all the above-mentioned problems.

In addition, the present invention obtains information about the direction or movement of the first device worn on the first body part of the user and information about the direction or movement of the second device worn on the second body part of the user, By referring to a relative relationship between the direction or movement of the device and the direction or movement of the second device, the posture or action performed by the user can be recognized to accurately recognize the posture or motion of the user wearing the wearable device. To do so for other purposes.

Representative configuration of the present invention for achieving the above object is as follows.

According to an aspect of the present invention, a method for recognizing a posture or motion, comprising: information about the direction or movement of the first device worn on the first body part of the user and a second worn on the second body part of the user Acquiring information about a direction or movement of a device; and referring to a relative relationship between the direction or movement of the first device and the direction or movement of the second device; A method is provided that includes recognizing an action.

According to another aspect of the present invention, there is provided a device for recognizing a posture or motion, comprising technical means for obtaining information regarding a direction or movement of the device worn on a body part of a user, and a direction or movement of the device and the user A device is provided that includes a program module for recognizing a posture taken by a user or an action performed by the user, with reference to a relative relationship between the direction or movement of another device worn on another body part of.

According to still another aspect of the present invention, there is provided a system for recognizing a posture or motion, comprising information on a direction or movement of a first device worn on a first body part of a user and a second body part worn on the second body part of the user. Obtaining information about the direction or movement of the second device and referring to the relative relationship between the direction or movement of the first device and the direction or movement of the second device, There is provided a system including a control unit for recognizing an operation and a storage for storing information provided from at least one of the first device and the second device.

In addition, there is further provided a non-transitory computer readable recording medium for recording another method, device, system, and computer program for executing the method.

According to the present invention, an effect of enabling a user to accurately recognize a posture or motion of a user at high resolution by using a relative relationship between directions or movements of two or more devices worn by the user is achieved.

In addition, according to the present invention, an effect of enabling the external device to be manipulated by providing feedback to the user based on the posture or operation of the user, or by the user taking a specific posture or performing a specific operation.

1 is a diagram illustrating a schematic configuration of an entire system for recognizing a posture or an operation of a user according to an embodiment of the present invention.

2 to 5 are diagrams exemplarily illustrating a configuration for recognizing a posture or an operation of a user according to an exemplary embodiment of the present invention.

100: network

200: recognition system

310: first device

320: second device

DETAILED DESCRIPTION The following detailed description of the invention refers to the accompanying drawings that show, by way of illustration, specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that the various embodiments of the present invention are different but need not be mutually exclusive. For example, certain shapes, structures, and characteristics described herein may be embodied in other embodiments without departing from the spirit and scope of the invention with respect to one embodiment. In addition, it is to be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the invention. The following detailed description, therefore, is not to be taken in a limiting sense, and the scope of the present invention, if properly described, is defined only by the appended claims, along with the full range of equivalents to which such claims are entitled. Like reference numerals in the drawings refer to the same or similar functions throughout the several aspects.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement the present invention.

Configuration of the entire system

As shown in FIG. 1, the entire system according to an embodiment of the present invention may include a communication network 100, a recognition system 200, and a plurality of

devices

310 and 320.

First, the communication network 100 according to an embodiment of the present invention may be configured regardless of a communication mode such as wired communication or wireless communication, and includes a local area network (LAN) and a metropolitan area network (MAN). ), And various communication networks such as a wide area network (WAN). Preferably, the communication network 100 as used herein may be a known Internet or World Wide Web (WWW). However, the communication network 100 may include, at least in part, a known wired / wireless data communication network, a known telephone network, or a known wired / wireless television communication network without being limited thereto.

Next, the recognition system 200 according to an embodiment of the present invention may be a digital device having memory capability and a microprocessor. The recognition system 200 may be a server system. The recognition system 200 has a function of interfacing one of the

devices

310 and 320 to the other through the communication network 100 to provide predetermined information or control commands to the other, or to receive the predetermined information or control commands from the other. Can be performed.

To this end, the recognition system 200, as described in detail below, information about the direction or movement of the first device worn on the first body part of the user and the second device worn on the second body part of the user The wearable device may be obtained by acquiring information about a direction or a motion of and recognizing a posture or an action performed by the user with reference to a relative relationship between the direction or motion of the first device and the direction or motion of the second device. A function of accurately recognizing a posture or an operation of the user who wears the mobile phone may be performed.

Recognizing a posture or motion of the user may be performed by a controller (not shown) included by the recognition system 200. Such a controller may exist in the form of a program module in the recognition system 200. Such program modules may take the form of operating systems, application modules or other program modules. The program module may also be stored in a remote storage device capable of communicating with the recognition system 200. Meanwhile, the program module includes, but is not limited to, routines, subroutines, programs, objects, components, data structures, etc. that perform particular tasks or execute particular abstract data types, which will be described later, according to the present invention.

In addition, the recognition system 200 stores information regarding a direction or a movement provided from at least one of the plurality of

devices

310 and 320 worn by the user, and this is stored in at least one of the plurality of

devices

310 and 320. It can further perform the function to be used by. The above-described storage may be performed by a storage (not shown) included by the recognition system 200. Such storage is a concept that includes a computer readable recording medium, and may be a broad database including not only a narrow database but also a file system based data record.

The function of the recognition system 200 will be described in more detail below. On the other hand, as described above with respect to the recognition system 200, this description is illustrative, a plurality of devices 310 that the user is wearing at least some of the functions or components required for the recognition system 200 as needed It will be apparent to those skilled in the art that the method may be realized or included in at least one of

Finally, the plurality of

devices

310 and 320 according to an embodiment of the present invention may be matched to the recognition system 200 or the plurality of devices 310 and 320 (preferably, the plurality of

devices

310 and 320). ) Is a digital device that can communicate with each other after being separated from each other or externalized), and any digital device having a computing capability with a memory means and a microprocessor can be used. It can be employed as the

device

310, 320 according to the invention. The

devices

310 and 320 may be so-called wearable devices such as smart glasses, smart watches, smart bands, smart rings, etc., or smartphones, smart pads, desktop computers, notebook computers, workstations, PDAs, web pads, mobile phones, It may be a more traditional device such as a button, mouse, keyboard, electronic pen, and the like.

In particular, according to an embodiment of the present invention, the

device

310, 320 may include at least one technical means for obtaining physical information about the direction or movement of the device (310, 320). Examples of such technical means are known components, such as motion sensors, acceleration sensors, gyro sensors, magnetic sensors, positioning modules (GPS modules, beacon-based positioning (identification) modules, etc.), barometers, distance sensors, cameras , Ultrasonic distance sensors (which may include stereo speakers or stereo microphones), light wave distance sensors, and the like.

In addition, the

devices

310 and 320 process the above physical information to provide information or control commands to

other devices

310 and 320, or receive information or control commands from

other devices

310 and 320, or the like. An application program for generating such information or control command may be further included. Such an application may exist in the form of a program module in the

corresponding devices

310 and 320. The nature of the program module may be generally similar to the controller of the recognition system 200 as described above. Here, the application may be replaced with a hardware device or a firmware device, at least a part of which may perform a function substantially the same or equivalent thereto.

Meanwhile, according to an embodiment of the present invention, the first device 310 and the second device 320 may have a predetermined association (for example, an association that belongs to the same user, or an association that functions for the same user). The first device 310 and the second device 320 when it is recognized that they have an association that is located substantially close to each other, or that one of them is reasonable to authenticate or allow the other. A predetermined connection may be formed between the two terminals, which may be performed by the recognition system 200 or may be performed by the first device 310 and the second device 320.

Example

Hereinafter, a detailed example of a method of recognizing a posture or an operation of a user wearing the plurality of

devices

310 and 320 will be described in detail according to an exemplary embodiment.

According to one embodiment of the invention, the recognition system 200 is a predetermined time for the information on the direction or movement of each of the first device 310 and the second device 320 is worn on different body parts of the user Acquisition in the section, with reference to the obtained information can recognize what kind of posture the user is taking or what action.

Specifically, the recognition system 200 according to an embodiment of the present invention may include information about a relationship between a direction of the first device 310 and a direction of the second device 320 (for example, the first device ( The posture of the user can be recognized with reference to the relationship between the normal direction of the plane specifying 310 and the normal direction of the plane specifying the second device 320). Here, the relationship between the direction of the first device 310 and the direction of the second device 320 is a vector defining the direction of the first device 310 and a vector defining the direction of the second device 310 It can be specified as an angle to form.

In addition, the recognition system 200 according to an exemplary embodiment of the present invention may provide information about a relationship between the movement of the first device 310 and the movement of the second device 320 (eg, the first device 310). ) And the action that the user is taking or the action that the user is performing with reference to a change in the distance between the second device 320 and a direction in which the first device 310 and the second device 320 are closer or farther away from each other. I can recognize it.

First, referring to FIG. 2, it may be assumed that a user who wears the first device 310 and the second device 320 on both wrists is holding an object 410 with both hands.

Specifically, referring to FIG. 2A, when the object 410 held by both hands is horizontal with respect to the ground, the front of the first device 310 worn on the user's left wrist is identified. The second device 320 worn on the user's right wrist and the normal direction of the plane (i.e., the direction of the Z axis when the plane specifying the front of the first device 310 is defined by the X and Y axes). The normal direction of the plane specifying the front face of (i.e., the direction of the Z 'axis when the plane specifying the front face of the second device 320 is defined by the X' axis and the Y 'axis) has a symmetric relationship with each other. do.

Also, referring to FIG. 2B, when the object 410 held by both hands is not level with respect to the ground, the front of the first device 310 worn on the user's left wrist is shown. The second device 320 worn on the user's right wrist and the normal direction of the specifying plane (i.e., the direction of the Z axis when the plane specifying the front of the first device 310 is defined by the X and Y axes). The normal direction of the plane specifying the front face of the plane (i.e., the direction of the Z 'axis when the plane specifying the front surface of the second device 320 is defined by the X' axis and the Y 'axis) has an asymmetric relationship with each other. Is achieved.

Accordingly, the recognition system 200 according to an exemplary embodiment of the present invention provides a user with an object based on whether the direction of the first device 310 and the direction of the second device 320 have a symmetric relationship with each other. It can recognize whether it is holding horizontally (ie straight).

Next, referring to FIG. 3, it is assumed that a user who wears the first device 310 and the second device 320 on both wrists respectively holds the steering wheel 420 of the vehicle with both hands. can do.

Specifically, referring to FIGS. 3A and 3B, when the user performs the operation of turning the steering wheel 420 while holding the steering wheel 420 with both hands, the user may wear the first wrist worn on the user's left wrist. While the normal direction of the plane specifying the front of the first device 310 and the normal direction of the plane specifying the front of the second device 320 worn on the user's left wrist are maintained in a symmetrical relationship with each other, The first device 310 and the second device 320 are moved in opposite directions.

Therefore, in the recognition system 200 according to an exemplary embodiment, the first device 310 is maintained in a symmetrical relationship between the direction of the first device 310 and the direction of the second device 320. ) Based on whether or not the second device 320 moves in the opposite direction, it may be recognized that the user is operating the steering wheel 420.

Next, referring to FIG. 4, it may be assumed that a user who wears the first device 310 and the second device 320 on both wrists performs an operation in which both hands move closer or closer. In this case, the recognition system 200 according to an embodiment of the present invention may refer to a distance between the first device 310 and the second device 320 that is increased by the user. It may be recognized that the user performs a zoom in operation. On the contrary, the user may pinch out or zoom out with reference to the shortening of the distance between the first device 310 and the second device 320. It can be recognized that the (Zoom Out) operation is performed.

Meanwhile, according to an embodiment of the present invention, the plurality of

devices

310 and 320 may be worn on various body parts, and the user may be recognized according to where the body parts on which the

devices

310 and 320 are worn are located. Posture or motion may vary.

First, referring to FIG. 6A, it may be assumed that the first device 310 and the second device 320 are worn on a wrist and ankle of a user, respectively. In this case, one embodiment of the present invention The recognition system 200 according to the user's upper body with reference to the information on the direction or movement of the first device 310 worn on the wrist and the information on the direction or movement of the second device 320 worn on the ankle. Recognize motion and lower body motion.

Next, referring to FIG. 6B, it may be assumed that the first device 310 and the second device 320 are worn on the user's right wrist and left wrist, respectively. In this case, The recognition system 200 according to an exemplary embodiment may refer to information about the direction or movement of the first device 310 worn on the right wrist and information about the direction or movement of the second device 320 worn on the left wrist. It is possible to recognize how balanced the user performs left and right. For example, when the user detects that the first device 310 worn on the left wrist moves faster than the second device 320 worn on the right wrist, the user balances left and right in walking. It can recognize that it is not doing and provide feedback to the user.

Embodiments according to the present invention described above may be implemented in the form of program instructions that may be executed by various computer components, and may be recorded on a non-transitory computer readable recording medium. The non-transitory computer readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the non-transitory computer readable recording medium may be those specially designed and configured for the present invention, or may be known and available to those skilled in the computer software arts. Examples of non-transitory computer readable recording media include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROMs, DVDs, magnetic-optical media such as floppy disks ( magneto-optical media) and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules to perform the process according to the invention, and vice versa.

Although the present invention has been described by specific embodiments such as specific components and the like, but the embodiments and the drawings are provided to assist in a more general understanding of the present invention, the present invention is not limited to the above embodiments. For those skilled in the art, various modifications and variations can be made from these descriptions.

Accordingly, the spirit of the present invention should not be limited to the above-described embodiments, and all of the equivalents or equivalents of the claims, as well as the appended claims, fall within the scope of the spirit of the present invention. I will say.

Claims

As a method for recognizing posture or motion,

Acquiring information about the direction or movement of the first device worn on the first body part of the user and information about the direction or movement of the second device worn on the second body part of the user; and

Recognizing a posture taken by the user or an operation performed by the user with reference to a relative relationship between the direction or movement of the first device and the direction or movement of the second device

How to include.
The method of claim 1,

In the recognition step,

A method of recognizing a posture taken by the user or an operation performed by the user with reference to a relationship between the direction of the first device and the direction of the second device specified based on at least one coordinate axis.
The method of claim 1,

In the recognition step,

And recognizing an action performed by the user with reference to a change in distance between the first device and the second device.
The method of claim 1,

In the recognition step,

And recognizing an operation performed by the user with reference to a direction in which the first device and the second device are close to or far from each other.
The method of claim 1,

Wherein the first body part and the second body part are both wrists of the user.
The method of claim 1,

Wherein the first body part is the wrist of the user and the second body part is the ankle of the user.
A non-transitory computer readable recording medium having recorded thereon a computer program for executing the method according to any one of claims 1 to 6.
A device for recognizing posture or motion,

Technical means for obtaining information about the direction or movement of the device worn on a user's body part, and

A program module for recognizing a posture taken by the user or an action performed by the user with reference to a relative relationship between the direction or movement of the device and the direction or movement of another device worn on another body part of the user

Device comprising a.
A system for recognizing posture or movement,

Obtaining information about the direction or movement of the first device worn on the first body part of the user and information about the direction or movement of the second device worn on the second body part of the user; A controller for recognizing a posture taken by the user or an operation performed by the user with reference to a relative relationship between a direction or a movement and a direction or movement of the second device; and

A storage storing information received from at least one of the first device and the second device

System comprising.