KR20180028358A - Terminal and method of controlling the same - Google Patents

Abstract

According to the present specification, a terminal and a control method thereof are disclosed. According to an embodiment of the present invention, a terminal comprises: a first sensor; a second sensor for sensing situation recognition data; and a control unit for determining whether or not a user is in a first state by extracting first data from sensing data of the first sensor, determining whether or not the user is in a second state by extracting second data from sensing data of the second sensor when the user is in the first state according to the determination result, and generating a control command when the user is in the second state according to the determination result so that the control command is transmitted to an external terminal.

Description

[0001] TERMINAL AND METHOD OF CONTROLLING THE SAME [0002]

The present invention relates to a terminal and its control method, and more particularly to service processing based on data obtained through at least one sensor.

A terminal can be divided into a mobile / portable terminal and a stationary terminal depending on whether the terminal is movable or not. The mobile terminal can be divided into a handheld terminal and a vehicle mounted terminal according to whether the user can directly carry the mobile terminal.

The functions of mobile terminals are diversified. For example, there are data and voice communication, photographing and video shooting through a camera, voice recording, music file playback through a speaker system, and outputting an image or video on a display unit. Some terminals are equipped with an electronic game play function or a multimedia player function. In particular, recently mobile terminals can receive multicast signals that provide visual content such as broadcast and video or television programs.

Such a terminal is being implemented in the form of a multimedia device having a complex function such as photographing and photographing of a moving picture, reproduction of a music or a moving picture file, reception of a game, and broadcasting, for various functions.

In order to support and enhance the functionality of such terminals, it may be considered to improve the structural and / or software parts of the terminal.

Conventionally, only a momentary fall motion of the user is detected and reported to the manager. For example, in the conventional system, the fall of the user can be detected only when a certain impact is sensed, and it is difficult to prevent the user from knowing the crisis situation before the fall and prevent it from occurring. In other words, there was a problem because it is a system to detect and report to the manager only when a fall situation occurs. Therefore, there was a problem when it was not possible to recognize a crisis situation or an emergency situation with the intention of the person, such as an infant, an elderly person, or an uncomfortable body, and to respond appropriately or avoid or report it. In addition, there has been a problem of erroneous recognition of the falling condition and malfunction due to such a false sense by sensing a falling condition due to a simple impact.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems, and it is an object of the present invention to recognize a circumstance recognition of a user's situation before and after an event, We will do it.

Another object of the present invention is to more precisely and accurately recognize a situation related to the event based on data obtained through at least one sensor.

Another problem is to increase the efficiency and reliability of a system or a terminal by providing an advanced service by recognizing the user's situation accurately before, during, and after an event occurs.

The technical problem to be solved by the present invention is not limited to the above-described technical problems and other technical problems which are not mentioned can be clearly understood by those skilled in the art from the following description .

Various embodiments (s) of a terminal and a control method thereof according to the present invention are disclosed herein.

A terminal according to an embodiment of the present invention includes a first sensor; A second sensor for sensing context aware data; And extracting first data from the sensing data of the first sensor to determine whether the user is in a first state and extracting second data from the sensing data of the second sensor if the user is in the first state, And if the user is in the second state, generates a control command and transmits the control command to the external terminal.

The technical solutions obtained by the present invention are not limited to the above-mentioned solutions, and other solutions not mentioned are clearly described to those skilled in the art from the following description. It can be understood.

According to the present invention, there are the following effects.

According to at least one embodiment of the various embodiments of the present invention, there is an effect that not only the event that occurs to the user but also the situation of the user before and after the occurrence of the event can be recognized and appropriately responded.

According to at least one embodiment of the various embodiments of the present invention, there is an effect that a situation related to the event can be recognized more precisely and accurately based on data obtained through at least one sensor.

According to at least one embodiment of the various embodiments of the present invention, the efficiency of the system or the terminal is improved and the reliability is improved by providing the advanced service by recognizing the user's situation accurately before, during, and after the event occurrence There is an effect that can be.

The effects obtained by the present invention are not limited to the above-mentioned effects, and other effects not mentioned can be clearly understood by those skilled in the art from the following description will be.

FIG. 1A is a schematic diagram of a digital system including a smart control ring device according to the present invention,
1B is a view showing the appearance of a smart control ring device related to the present invention,
2 is a view illustrating a shape or appearance of a terminal according to an embodiment of the present invention;
3 is a configuration block diagram of a mobile terminal according to the present invention;
4 is a perspective view showing an example of a watch-type terminal according to another embodiment of the present invention,
5 is a perspective view illustrating an example of a glass-type terminal according to another embodiment of the present invention.
6 is a block diagram of a smart control device according to an embodiment of the present invention.
7 is a flowchart illustrating a data processing method of a terminal according to an embodiment of the present invention.
FIG. 8 illustrates a data graph for falling motion according to an embodiment of the present invention; FIG.
FIG. 9 is a view illustrating a method of measuring a posture associated with an fall algorithm according to an embodiment of the present invention.
FIG. 10 is a view for explaining a place as one of parameters of context awareness data according to an embodiment of the present invention; FIG.
FIG. 11 is a diagram for explaining an everyday life pattern and an abnormality as parameters of context-aware data according to an embodiment of the present invention,
12 is a view for explaining a head direction of a user as one of parameters of context awareness data according to an embodiment of the present invention,
FIG. 13 is a view for explaining voice or audio as one of parameters of context-aware data according to an embodiment of the present invention;
FIG. 14 is a view for explaining a user's location search as one of the parameters of the context-aware data according to an embodiment of the present invention;
FIG. 15 is a graph showing a breathing / heartbeat data graph according to an embodiment of the present invention; FIG.
16 and 17 are diagrams for explaining states constituting a fall algorithm according to an embodiment of the present invention,
FIG. 18 is a view for explaining whether a user exists or not in one of the context-aware parameters according to the present invention;
FIG. 19 is a flowchart illustrating a fall detection algorithm according to an embodiment of the present invention;
FIG. 20 is a graph showing a graph of data on acceleration and buffer data,
21 is a flowchart illustrating a motion determination algorithm according to an embodiment of the present invention.
FIG. 22 is a view showing an example of a heart rate-related data graph related to FIG. 21,
FIG. 23 is a graph illustrating a data graph for illustrating an algorithm for distinguishing sleeping and non-moving scenes according to an embodiment of the present invention,
24 to 26 are flowcharts illustrating an algorithm for identifying a user in an abnormal state according to an embodiment of the present invention,
27 to 29 are diagrams for explaining a correlation between a lying posture and an abnormality determination algorithm according to an embodiment of the present invention,
30 is a flowchart illustrating an abnormality determination method according to another embodiment of the present invention, and FIG.
31 is a data graph related to the abnormality determination of FIG.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, wherein like reference numerals are used to designate identical or similar elements, and redundant description thereof will be omitted. The suffix "module" and " part "for the components used in the following description are given or mixed in consideration of ease of specification, and do not have their own meaning or role. In the following description of the embodiments of the present invention, a detailed description of related arts will be omitted when it is determined that the gist of the embodiments disclosed herein may be blurred. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are intended to provide further explanation of the invention as claimed. , ≪ / RTI > equivalents, and alternatives.

Terms including ordinals, such as first, second, etc., may be used to describe various elements, but the elements are not limited to these terms. The terms are used only for the purpose of distinguishing one component from another.

It is to be understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, . On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between.

The singular expressions include plural expressions unless the context clearly dictates otherwise.

In the present application, the terms "comprises", "having", and the like are used to specify that a feature, a number, a step, an operation, an element, a component, But do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

Although the terminal described in the present specification is described as an example of a mobile terminal, it is not limited thereto. For example, the mobile terminal may be a mobile phone, a smart phone, a notebook computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation device, a slate PC, a tablet PC, an ultrabook, a wearable device (e.g., a smart watch, a glass glass, a head mounted display (HMD)), and the like. However, in order to facilitate understanding of the present invention, and for convenience of explanation, the terminal will be described with reference to the terminal of FIG. 2 to be described later, for example. The terminal of Fig. 2 is an example implemented in the form of being attached to, for example, a user's body, clothing, or the like.

The " smart controlling device " described in the present specification is a device that transmits data to at least one device connected or connectable based on various inputs such as voice, text, image, motion, The term " device " The smart control ring device may include all electronic devices equipped with an artificial intelligence (AI) function and / or capable of voice recognition. The smart control ring device may include an audio device capable of voice recognition and controlling an operation by transmitting a control command to a predetermined device according to a voice command of an input user, for example. Hereinafter, an audio device (or smart speaker) equipped with an artificial intelligence (function) and capable of voice recognition will be described as an example of the present invention for the purpose of facilitating understanding of the present invention and for convenience of explanation. It should be noted, however, that the present invention is not limited thereto.

In this specification, the present invention is based on data obtained through at least one sensor for various events that can occur in everyday life, based on not only simple detection of the event but also context awareness So as to accurately detect the event and reduce malfunctions caused by a false sense. In other words, the present invention predicts occurrence of an event not only at the time of the occurrence of the event but also before the occurrence of the event, thereby preventing the influence of the occurrence of the event. Further, in the present invention, when an event occurs, it can be precisely detected and appropriately prepared or responded to after the occurrence of the event, so that the effect of event occurrence can be minimized, or the occurrence of an additional event due to the event occurrence, Can be minimized. Therefore, according to the present invention, the risk factors of event occurrence can be predicted and guided by the present invention, and can be associated with related training and the like.

Hereinafter, an event refers to a concept including all the various situations that can occur to a user in daily life, and in particular, a situation in which a situation causes a risk to the user or accompanies the event. In order to facilitate the understanding of the present invention and for convenience of the applicant's explanation, the following description will be made by taking as an example a case where a child, elderly person, a person who is physically uncomfortable, It should be noted, however, that the scope of the present invention is not limited to the above examples.

FIG. 1A is a schematic view of a digital system including a smart control ring device according to the present invention, FIG. 1B is a view showing the appearance of a smart control ring device related to the present invention, FIG. FIG. 6 is a block diagram of a configuration of a smart control device 110 according to an embodiment of the present invention. Referring to FIG.

In FIG. 1A, for convenience of explanation, a digital system including various devices including a smart control ring device 110 connected to or connectable with each other through a home network is exemplified. The smart control ring device 110 may perform data communication with at least one external device via local or remote wired / wireless communication, and may control the at least one external device based thereon. For example, the smart control ring device 110 may perform the function of a controller in the digital system of FIG. Meanwhile, the smart control device 110 may perform data communication with an external server 140 as shown in FIG. At this time, data communication with the external server 140 may be performed by a device other than the smart control device 110, and the smart control device 110 may indirectly communicate with the external server 140 through the other device Data communication may also be performed. Fig data communication takes place within an external server or a digital system of Figure 1 is Y-fi (Wi-Fi; Wireless-Fidelity ), Bluetooth (Bluetooth ^TM), jet-wave (Z-wave), Infrared Communication, ZigBee, RS and communication Protocol. ≪ / RTI > However, the present invention is not limited to the communication protocol described above, and a communication protocol or the like described later may be further used. Meanwhile, the data communication may be performed directly between the device and the device, or indirectly through a device such as a relay, a gap filler, or the like.

In FIG. 1, the smart control device 110 and the other devices 122, 124, 126, 128, and 130 constituting the digital system are illustrated as belonging to the same network, but they need not necessarily be limited to the same network none. 1, electronic devices such as a digital TV 122, a smart phone 124, an electric or robot cleaner 126, a refrigerator 128, a washing machine 130, and the like are illustrated. However, the present invention is not limited to these electronic devices.

1B, the smart control ring device 110 may include a user input unit 163, an acoustic output unit 152, and a light output unit 154 on the outer surface of the body unit. The user input unit 163 may be configured to receive a control command from a user, and may be provided in plurality. Hereinafter, a plurality of user input units will be referred to as a first user input unit 163a, a second user input unit 163b, and a third user input unit 163c, respectively. Likewise, a plurality of optical output sections 154 may be provided, and will be referred to as a first optical output section 154a and a second optical output section 154b, respectively. When the plurality of user input units and optical output units are collectively referred to, the reference numerals 163 and 154 are used.

The body part may be cylindrical, and may have a function as a ring-shaped body. The size of the body part can be determined considering the design. On the other hand, the shape of the body part can be variously changed.

The body portion includes a first region 170 forming a side face of the cylinder, a second region 180 forming a bottom face of the cylinder, and a third region 180 facing the second region 180, Area 190. The < / RTI > The second region 180 and the third region 190 may have the same area or different areas.

The first region 170 may be referred to as the outer surface, the second region 180 and the third region 190 may be referred to as the outer upper surface and the outer bottom surface. Hereinafter, the first, second, The terms are used to describe them.

The first region 170 may include a third user input portion 163c, a second optical output portion 154b, an infrared ray output portion 155, and an acoustic output portion 152. For example, the second light output section 154c and the sound output section 152 may be formed apart from each other. Alternatively, at least a part of the second light output section 154c may have a layer structure with the sound output section 152 and overlap each other. These matters can be easily changed by the designer's design.

The second light output portion 154b and the acoustic output portion 152 may be formed to surround the first region 170 of the body portion. Accordingly, the sound output unit 152 is configured to output sound in all directions with respect to the body unit, and the second light output unit 154c outputs light in all directions with respect to the body unit.

The third user input unit 163c may be disposed at the top of the first area 170. [ The third user input unit 163c may be formed to rotate around the center of the body. Therefore, the user can rotate the third user input portion 163c to make the volume of the smart control ring device 110 large or small.

The infrared ray output unit 155 may be disposed at a position where an infrared ray signal can be output in all directions. For example, an infrared ray output unit may be disposed at an upper end of the first area 170. As another example, the first region 170 may be disposed at an upper portion of the first region 170 so as to be rotatable. Accordingly, the infrared ray output unit 155 can output an infrared ray signal so that an infrared ray signal reaches an external device located at an arbitrary position. On the other hand, the arrangement position of the infrared ray output portion can be changed to a position where the infrared ray signal can be output in all directions by the design of a person skilled in the art.

In the second region 180, a display unit 151, first and second user input units 163a and 163b, a first light output unit 154a, and a temperature / humidity sensor may be disposed.

The display unit 151 may be disposed at the center of the second area 180 so as to secure the time of the user. The first and second user input units 163a and 163b may be arranged to receive a user input in a peripheral area of the display unit 151. [

The first and second user input units 163a and 163b may be formed in a button manner to operate by a pressing operation or may be formed in a touch manner to operate by a touch operation. The first and second user input units 163a and 163b may be configured to perform different functions. For example, the first user input unit 163a is a button for inputting a control command for stopping speech recognition, and the second user input unit 163b is a button for inputting a control command for turning on / .

The first light output portion 154a may be formed along the outer edge of the second region 180. [ That is, the first light output portion 154a may have a band shape surrounding the outer region of the second region 180. [ For example, if the second region 180 is circular, the first light output portion 154a may be in the form of a band surrounding the circle.

The light output unit 154 may be formed to emit light from the light source. A light emitting diode (LED) may be used as such a light source. The light source is located on the inner circumferential surface of the light output section 154, and the light output from the light source passes through the light output section 154 to illuminate the outside. The light output portion 154 is made of a transparent or semitransparent material through which light can pass.

The light output unit 154 may output light information related to an event occurring in the smart control ring device 110 as light. For example, when speech recognition is being performed in the smart control ring device 110, red light may be output. Further, when the smart controlling device 110 is waiting for a correction command, it can output yellow light.

The temperature / humidity sensor may be disposed in a second area 180 that can be in direct contact with the outside so as to sense external temperature and humidity.

Although not shown, the third area 190 may further include a power supply for receiving power from the outside, an interface for transmitting and receiving data to and from an external device, and an audio input (microphone) for receiving sound.

As an example of controlling external devices according to the present invention, the smart controlling device 110 may turn on the air conditioner or adjust the temperature of the air conditioner through infrared communication. The smart control ring device 110 may serve as a controller for controlling an external device under the Internet of Things (IOT).

Referring to FIG. 2, the appearance and shape of a terminal according to an embodiment of the present invention will be described.

The terminal of FIG. 2A is a terminal implemented, for example, in the form of a neckband. Referring to FIG. 2A, the terminal includes a neckband 210 and a cradle 230. The neckband 210 may be coupled to the cradle 230 so that the neckband 210 may be mounted on the cradle 230 and the cradle 230 may be mounted on the neckband 210 to supply power to the neckband 210. For example, the neckband 210 may be bent in a form that can be mounted on the neck of a user (person), or may be realized in a wired form. On the other hand, both ends of the neckband 210 may be combined or included with a configuration 222, 224 for sensing a living body signal through contact or non-contact with a user's body. As described above, the cradle 230 receives the sensed living body signal of the neckband 210 as a communication device itself, in addition to power supply to the neckband 210, either by wire or wirelessly, and transmits the received living body signal (A terminal, a server, etc. of an administrator) on the basis of the received information. The transmission of the warning signal, the control command, and the like is not necessarily performed through the cradle 230, but may be performed through the neckband 210.

The terminal of FIG. 2B is similar to the terminal 210 of FIG. 2A described above, but is implemented in a form similar to a necklace rather than a neckband. Such a terminal may be implemented as a line and a sensing unit that can be fixed through a user's neck or the like.

The terminal of FIG. 2C is an example implemented in a form that can be attached to a body of a user or clothes of a user, unlike the above-described FIGs. 2A to 2B.

As described above, both the terminal of FIG. 2B and the terminal of FIG. 2C are not shown, but may be powered through a charging arrangement such as the cradle 230 described in FIG. 2A. In addition, it will be apparent to those skilled in the art that the terminal according to the present invention can be implemented in other forms.

Hereinafter, a terminal according to the present invention will be described with reference to FIGS. 1, 2 and 6. FIG.

6, the terminal includes at least one of a signal receiving unit 610, a control unit 620, a signal processing unit 630, a memory 640, and a signal output unit 650. Here, the controller 620 may include an analyzer 622, an AI processor 624, and the like. In other words, the analyzing unit 622 and the AI processing unit 624 may be individually configured or may be modularized to have one configuration.

The signal receiving unit 610 receives a voice command of the user and a sensing signal through the sensor. At this time, the voice command may be classified into a start signal such as 'Alice' and an operation signal such as an instruction other than the start signal, a request, or the like. Hereinafter, the start signal may be variously referred to as start data, voice command trigger signal, voice command trigger data, trigger signal, trigger data, and the like. At this time, the signal and data may be used in substantially the same meaning, for example, although there is a difference in form or format. Therefore, unless otherwise specified, the signal and data in this specification may be interpreted in the same sense. On the other hand, the data in the input signal excluding the trigger data is referred to as command data (or real data). Such command data means, for example, actual command data of a speaking person (user). The signal receiving unit 610 transmits the received voice command to the controller 620. At this time, the trigger data in the transmitted voice command can be excluded. For example, when the trigger data is included or not input, the signal receiving unit 610 may ignore the received signal or may not transmit the signal to the controller 620. In the meantime, the signal receiving unit 610 has exemplified the voice command in the above, but the present invention is not limited thereto. Accordingly, the signal receiving unit 610 can receive various signals such as text, motion, touch, and the like, and can include an interface therefor.

The control unit 620 includes a recognition unit 622, an AI processing unit 624, and the like, as described above. The recognition unit 622 recognizes the trigger data and the command data in the input signal received from the signal receiving unit 210 according to the input signal format. For example, if the input signal through the signal receiving unit 610 is a voice format, the recognizing unit 622 recognizes the input signal using hardware or software such as a voice recognition function, a tool, a module, and an engine. At this time, the recognition unit 622 converts the voice input signal into a text format based on STT (Sound to Text) (or TTS (Text to Sound) in some cases), and then transfers the voice input signal to the AI processing unit 624 And the like. The AI processing unit 624 analyzes the input signal data transmitted from the recognizing unit 622. For example, when the trigger data in the received input signal is recognized or analyzed, the recognition unit 622 or the AI processing unit 624 may detect the adjacent display device according to the present invention, can do. The AI processing unit 624 analyzes the input signal recognized by the recognition unit 622 and generates response data. In the above, the analysis may mean, for example, judging whether or not a valid input signal, attribute, type, and type of trigger or command data.

Meanwhile, the AI processing unit 624 plays a role of processing information based on artificial intelligence technology. The AI processing unit 624 includes at least one or more of AI learning, information inference, information perception, Modules.

The AI processing unit 624 can use a machine running technology to learn a large amount of information (big data, big data) such as information stored in the terminal, environment information around the terminal, information stored in a communicable external storage, Inference, and processing. The AI processing unit 624 predicts (or inferences) the operation of at least one executable terminal by using the learned information using the machine learning technique, and the AI processing unit 624 determines the most feasible The terminal can be controlled so that the operation is performed.

The machine learning technique collects and learns a large amount of information based on at least one algorithm, and determines and predicts information based on the learned information. The learning of information is an operation that grasps the characteristics, rules, and judgment criteria of information, quantifies the relationship between information and information, and predicts new data using a quantified pattern.

The algorithms used by such machine learning techniques may be statistical based algorithms, for example, a decision tree that uses the tree structure type as a prediction model, an artificial tree that mimics the neural network structure and function of the organism Neural networks, genetic programming based on biological evolutionary algorithms, clustering to distribute observed examples to a subset of clusters, Monte Carlo computation of probability values through randomly extracted random numbers, Method (Monter carlo method) or the like.

As a field of machine learning technology, deep running technology is a technology that performs at least one of learning, judging, and processing information using an artificial neural network algorithm. An artificial neural network can have a structure that links layers and layers, and conveys data between layers and layers. This deep learning technique can learn vast amount of information through artificial neural network using GPU (graphic processing unit) optimized for parallel computing.

Meanwhile, the AI processing unit 624 collects (detects, monitors, extracts, and detects signals, data, information, and the like) input or output from the components of the terminal to collect vast amounts of information for applying the machine learning technology , Receiving). The AI processing unit 624 can collect (detect, monitor, extract, detect, and receive) data, information, and the like stored in an external storage (for example, a cloud server) More specifically, the collection of information can be understood as including the operation of sensing information through a sensor, extracting information stored in the memory 640, or receiving information from an external storage through communication.

The AI processing unit 624 can detect information in the terminal, surrounding environment information surrounding the terminal, and user information through a sensing unit (not shown). In addition, the AI processing unit 624 can receive broadcast signals and / or broadcast-related information, wireless signals, wireless data, and the like through a wireless communication unit (not shown). The AI processing unit 624 can receive image information (or signal), audio information (or signal), data, or information input from the user from the input unit.

The AI processing unit 624 collects a large amount of information in real time in the background, learns it, and stores information (for example, a knowledge graph, a command policy, a personalization database, 640).

When the operation of the terminal is predicted based on the information learned using the machine learning technology, the AI processing unit 624 controls the elements of the terminal, executes the predicted operation, To the control unit 620. [0064] The control unit 620 can execute the predicted operation by controlling the terminal based on the control command.

On the other hand, when the specific operation is performed, the AI processing unit 624 analyzes the history information indicating the performance of the specific operation through the machine learning technique, and updates the existing learned information based on the analysis information have. Accordingly, the AI processing unit 224 can improve the accuracy of information prediction.

In the present specification, the recognition unit 622 and / or the AI processing unit 624 may be understood as the same components as the control unit 620. [ In this case, the functions performed in the control unit 620 described in this specification can be expressed as being performed in the recognition unit 622 and / or the AI processing unit 624, and vice versa.

Alternatively, in the present specification, the recognition unit 622 and / or the AI processing unit 624 and the control unit 620 can be understood as separate components. In this case, the recognition unit 622 and / or the AI processing unit 624 can perform various controls on the terminal by exchanging data with the control unit 620. The control unit 620 may perform at least one function on the terminal or control at least one of the components of the terminal based on the results obtained from the AI processing unit 624. [ Furthermore, the AI processing unit 624 can also be operated under the control of the control unit 620. [

The memory 640 stores necessary data related to analysis of the received voice command, data processing of the terminal such as the control command, and the like. Also, the memory 640 stores data supporting various functions of the terminal. The memory 640 stores a plurality of application programs or applications running on the terminal, data for operation of the terminal, commands, data for operation of the AI processing unit 624 (for example, At least one algorithm information for machine learning, etc.). At least some of these applications may be downloaded from an external server via wireless communication. Meanwhile, the application program may be stored in the memory 640, installed on the terminal, and operated by the control unit 620 to perform the operation (or the function) of the terminal.

The signal processing unit 630 generates the corresponding signal under the control of the control unit 620. For example, the aforementioned control command may be included in the generated signal. In addition, the signal processing unit 630 can process various data required by other terminals.

The signal output unit 650 transmits a signal processed by the signal processing unit 630 to an internal or external device according to a predetermined output method. At this time, the output method may be determined or changed according to at least one of a communication protocol, a destination device, a kind of an output signal, and the like.

Although not shown, the terminal may further include a communication interface necessary for transmitting / receiving signals / data as required.

Hereinafter, a fixed device such as a network TV, a Hybrid Broadcast Broadband TV (HBBTV), a Smart TV, an IPTV (Internet Protocol TV), a PC (Personal Computer), etc. may be included as one of terminals for performing data communication with a terminal according to the present invention have. Meanwhile, the terminal may be implemented in the form of a sign with only a display panel or a SET with another configuration, for example, a set-top box (STB).

The terminal may be a general OS (Operating System), a web OS (OS), or the like. The terminal may add various services and applications to an OS kernel, a Linux kernel, Modify, update, etc., thereby providing a more user-friendly environment.

The terminal may also be referred to as a display device in the sense that it includes all of the devices including a display. Hereinafter, FIG. 4 illustrates a digital TV as an example of a display device, and FIG. 5 illustrates a mobile terminal as another example of a display device. However, as described above, the terminal is not limited to Figs. 4 and 5, and may include all devices including a display.

It will be readily apparent to those skilled in the art that the configurations of FIGS. 3A through 4B may be applied to a mobile terminal, which will be described later, except when the configuration is applicable only to a fixed terminal such as the digital TV 400. [

For example, the mobile terminal described herein may include a voice recognition speaker, a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP) A slate PC, a tablet PC, an ultrabook, a wearable device (e.g., a smart watch, a glass glass, an HMD (head mounted) display)), and the like.

Hereinafter, a smart control device 110 according to an embodiment of the present invention will be described.

3 is a configuration block diagram of a mobile terminal according to the present invention.

3A, the mobile terminal 300 includes a wireless communication unit 310, an input unit 320, a sensing unit 340, an output unit 350, an interface unit 360, a memory 370, a controller 380 A power supply unit 390, and the like. The components shown in FIG. 3A are not essential for implementing a mobile terminal, so that the mobile terminal described herein may have more or fewer components than the components listed above.

The wireless communication unit 310 may be connected between the mobile terminal 300 and the wireless communication system or between the mobile terminal 300 and another mobile terminal 300 or between the mobile terminal 300 and the external server 300. [ And at least one module that enables wireless communication between the base station and the base station. In addition, the wireless communication unit 310 may include one or more modules for connecting the mobile terminal 300 to one or more networks.

The wireless communication unit 310 may include at least one of a broadcast receiving module 311, a mobile communication module 312, a wireless Internet module 313, a short distance communication module 314, and a location information module 315 .

The input unit 320 includes a camera 321 or an image input unit for inputting a video signal, a microphone 322 for inputting an audio signal or an audio input unit and a user input unit 323 for receiving information from a user A touch key, a mechanical key, and the like). The voice data or image data collected by the input unit 320 may be analyzed and processed by the user's control command.

The sensing unit 340 may include at least one sensor for sensing at least one of information in the mobile terminal, surrounding environment information surrounding the mobile terminal, and user information. For example, the sensing unit 340 may include a proximity sensor 341, an illumination sensor 342, a touch sensor, an acceleration sensor, a magnetic sensor, A G-sensor, a gyroscope sensor, a motion sensor, an RGB sensor, an infrared sensor, a finger scan sensor, an ultrasonic sensor, A microphone 322, a battery gauge, an environmental sensor (for example, a barometer, a hygrometer, a thermometer, a radiation detection sensor, a temperature sensor, A thermal sensor, a gas sensor, etc.), a chemical sensor (e.g., an electronic nose, a health-care sensor, a biometric sensor, etc.). Meanwhile, the mobile terminal disclosed in the present specification can combine and utilize information sensed by at least two of the sensors.

The output unit 350 is for generating output related to the visual, auditory or tactile sense and includes a display unit 351, an acoustic output unit 552, a haptrip module 353, a light output unit 354, an infrared output unit 355 < / RTI > The display unit 351 may have a mutual layer structure with the touch sensor or may be integrally formed to realize a touch screen. The touch screen functions as a user input unit 323 that provides an input interface between the mobile terminal 300 and a user and can provide an output interface between the mobile terminal 300 and a user.

The interface unit 360 serves as a pathway to various kinds of external devices connected to the mobile terminal 300. The interface unit 360 may include a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device having an identification module, an audio I / ) Port, a video I / O port, and an earphone port. In the mobile terminal 300, corresponding to the connection of the external device to the interface unit 360, it is possible to perform appropriate control related to the connected external device.

In addition, the memory 370 stores data supporting various functions of the mobile terminal 300. The memory 370 may store a plurality of application programs driven by the mobile terminal 300, data for operation of the mobile terminal 300, and commands. At least some of these applications may be downloaded from an external server via wireless communication. At least some of these application programs may be present on the mobile terminal 300 from the time of shipment for the basic functions (e.g., call incoming, outgoing, message reception, origination) of the mobile terminal 300 . Meanwhile, the application program may be stored in the memory 370, installed on the mobile terminal 300, and operated by the control unit 380 to perform the operation (or function) of the mobile terminal.

The control unit 380 controls the overall operation of the mobile terminal 300, in addition to the operations related to the application program. The control unit 380 may process or process signals, data, information, and the like input or output through the above-described components, or may drive an application program stored in the memory 370 to provide or process appropriate information or functions to the user.

In addition, the controller 380 may control at least some of the components illustrated in FIG. 3 to drive an application program stored in the memory 370. In addition, the controller 380 can operate at least two of the components included in the mobile terminal 300 in combination with each other for driving the application program.

The power supply unit 390 receives external power and internal power under the control of the controller 380 and supplies power to the respective components included in the mobile terminal 300. The power supply unit 390 includes a battery, which may be an internal battery or a replaceable battery.

At least some of the components may operate in cooperation with one another to implement a method of operation, control, or control of a mobile terminal according to various embodiments described below. In addition, the operation, control, or control method of the mobile terminal may be implemented on the mobile terminal by driving at least one application program stored in the memory 370. [

Hereinafter, the components listed above will be described in more detail with reference to FIG. 5 before explaining various embodiments implemented through the mobile terminal 300 as described above.

Referring to the wireless communication unit 310, the broadcast receiving module 311 of the wireless communication unit 310 receives broadcast signals and / or broadcast-related information from an external broadcast management server through a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel. Two or more broadcast receiving modules may be provided to the mobile terminal 300 for simultaneous broadcast reception or broadcast channel switching for at least two broadcast channels.

The mobile communication module 312 may be a mobile communication module that is capable of communicating with a plurality of mobile communication devices such as a mobile communication module, a mobile communication module, a mobile communication module, a mobile communication module, (Enhanced Voice-Data Optimized or Enhanced Voice-Data Only), Wideband CDMA (WCDMA), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Long Term Evolution (LTE-Advanced), or the like) on a mobile communication network.

The wireless signal may include various types of data depending on a voice call signal, a video call signal or a text / multimedia message transmission / reception.

The wireless Internet module 313 is a module for wireless Internet access, and may be embedded in the mobile terminal 300 or externally. The wireless Internet module 313 is configured to transmit and receive wireless signals in a communication network according to wireless Internet technologies.

Wireless Internet technologies include, for example, wireless LAN (WLAN), wireless fidelity (Wi-Fi), wireless fidelity (Wi-Fi) Direct, DLNA (Digital Living Network Alliance), WiBro The wireless Internet module 513 transmits and receives data according to at least one wireless Internet technology in a range including Internet technologies not listed in the above. do.

In view of wireless Internet access by WiBro, HSDPA, HSUPA, GSM, CDMA, WCDMA, LTE, LTE-A and the like through a mobile communication network, the wireless Internet module 313 May be understood as a kind of the mobile communication module 312.

The short-range communication module 314 is for short-range communication. The short-range communication module 314 may be a Bluetooth ™, a Radio Frequency Identification (RFID), an Infrared Data Association (IrDA), an Ultra Wideband (UWB) (Near Field Communication), Wi-Fi, Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technology. The short range communication module 314 may be connected to the wireless communication system between the mobile terminal 300 and another mobile terminal 300 or between the mobile terminal 300 and the mobile terminal 300 through wireless local area networks ) And a network in which another mobile terminal (or an external server) is located. The short-range wireless communication network may be a short-range wireless personal area network.

Herein, another mobile terminal 300 is a wearable device (e.g., a smart watch, a smart phone, etc.) capable of exchanging data with the mobile terminal 300 according to the present invention Glass (smart glass), head mounted display (HMD)). The short range communication module 314 may detect (or recognize) a wearable device capable of communicating with the mobile terminal 300 around the mobile terminal 300. [ If the detected wearable device is a device authenticated to communicate with the mobile terminal 300 according to the present invention, the control unit 380 may transmit at least a part of the data processed by the mobile terminal 300 to the short- 314 to the wearable device. Therefore, the user of the wearable device can use the data processed by the mobile terminal 300 through the wearable device. For example, according to this, when a telephone is received in the mobile terminal 300, the user performs a telephone conversation via the wearable device, or when a message is received in the mobile terminal 300, It is possible to check the message.

The position information module 315 is a module for obtaining the position (or current position) of the mobile terminal, and a representative example thereof is a Global Positioning System (GPS) module or a Wi-Fi module. For example, when the mobile terminal utilizes the GPS module, it can acquire the position of the mobile terminal by using a signal transmitted from the GPS satellite. As another example, when the mobile terminal utilizes the Wi-Fi module, it can acquire the position of the mobile terminal based on information of a wireless access point (AP) that transmits or receives the wireless signal with the Wi-Fi module. Optionally, the location information module 315 may perform any of the other functions of the wireless communication unit 310 to obtain data relating to the location of the mobile terminal, in addition or alternatively. The position information module 315 is a module used for obtaining the position (or current position) of the mobile terminal, and is not limited to the module for directly calculating or acquiring the position of the mobile terminal.

The input unit 320 is for inputting image information (or signal), audio information (or signal), data, or information input from a user. For inputting image information, Or a plurality of cameras 321. The camera 321 processes an image frame such as a still image or moving image obtained by the image sensor in the video communication mode or the photographing mode. The processed image frame may be displayed on the display unit 351 or stored in the memory 370. [ A plurality of cameras 321 provided in the mobile terminal 300 may be arranged to have a matrix structure and various angles or foci may be provided to the mobile terminal 300 through the camera 321 having a matrix structure A plurality of pieces of image information can be input. In addition, the plurality of cameras 321 may be arranged in a stereo structure to acquire a left image and a right image for realizing the stereoscopic image.

The microphone 322 processes the external acoustic signal into electrical voice data. The processed voice data can be utilized variously according to a function (or a running application program) being executed in the mobile terminal 300. Meanwhile, various noise elimination algorithms may be implemented in the microphone 322 to remove noise generated in receiving an external sound signal.

The user input unit 323 is for receiving information from the user. When information is input through the user input unit 323, the control unit 380 can control the operation of the mobile terminal 300 to correspond to the input information . The user input unit 323 may include a mechanical input unit (or a mechanical key such as a button located at the front / rear or side of the mobile terminal 300, a dome switch, a jog wheel, Jog switches, etc.) and touch-type input means. For example, the touch-type input means may comprise a virtual key, a soft key or a visual key displayed on the touch screen through software processing, And a touch key disposed on the touch panel. Meanwhile, the virtual key or the visual key can be displayed on the touch screen having various forms, for example, graphics, text, icons, video, or a combination thereof.

Meanwhile, the sensing unit 340 senses at least one of information in the mobile terminal, surrounding environment information surrounding the mobile terminal, and user information, and generates a corresponding sensing signal. The control unit 380 may control the driving or operation of the mobile terminal 300 or may perform data processing, function or operation related to the application program installed in the mobile terminal 300 based on the sensing signal. Representative sensors among various sensors that may be included in the sensing unit 340 will be described in more detail.

First, the proximity sensor 341 refers to a sensor that detects the presence of an object approaching a predetermined detection surface, or the presence of an object in the vicinity of the detection surface without mechanical contact by using electromagnetic force or infrared rays. The proximity sensor 341 may be disposed in the inner area of the mobile terminal or the proximity sensor 341 near the touch screen.

Examples of the proximity sensor 341 include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, and an infrared proximity sensor. In the case where the touch screen is electrostatic, the proximity sensor 341 can be configured to detect the proximity of the object with a change in the electric field along the proximity of the object having conductivity. In this case, the touch screen (or touch sensor) itself may be classified as a proximity sensor.

On the other hand, for convenience of explanation, the act of recognizing that the object is located on the touch screen in proximity with no object touching the touch screen is referred to as "proximity touch & The act of actually touching an object on the screen is called a "contact touch. &Quot; The position at which the object is closely touched on the touch screen means a position where the object corresponds to the touch screen vertically when the object is touched. The proximity sensor 341 is capable of sensing a proximity touch and a proximity touch pattern (e.g., a proximity touch distance, a proximity touch direction, a proximity touch speed, a proximity touch time, a proximity touch position, have. The control unit 380 processes the data (or information) corresponding to the proximity touch operation and the proximity touch pattern sensed through the proximity sensor 341 as described above, and further stores the visual information corresponding to the processed data It can be output on the touch screen. Further, the control unit 380 can control the mobile terminal 300 such that different operations or data (or information) are processed depending on whether the touch to the same point on the touch screen is a proximity touch or a contact touch .

The touch sensor uses a touch (or touch input) applied to the touch screen (or the display unit 351) by using at least one of various touch methods such as a resistance film type, a capacitive type, an infrared type, an ultrasonic type, Detection.

For example, the touch sensor may be configured to convert a change in a pressure applied to a specific portion of the touch screen or a capacitance generated in a specific portion to an electrical input signal. The touch sensor may be configured to detect a position, an area, a pressure at the time of touch, a capacitance at the time of touch, and the like where a touch object touching the touch screen is touched on the touch sensor. Here, the touch object may be a finger, a touch pen, a stylus pen, a pointer, or the like as an object to which a touch is applied to the touch sensor.

Thus, when there is a touch input to the touch sensor, the corresponding signal (s) is sent to the touch controller. The touch controller processes the signal (s) and transmits corresponding data to the controller 380. Thus, the control unit 380 can know which area of the display unit 351 is touched or the like. Here, the touch controller may be a separate component from the control unit 380, and may be the control unit 380 itself.

Meanwhile, the control unit 380 can perform different controls or perform the same control according to the type of the touch object, which touches the touch screen (or a touch key provided in the touch screen). Whether to perform different controls or to perform the same control according to the type of the touch object may be determined according to the current state of the mobile terminal 300 or an application program being executed.

On the other hand, the touch sensors and the proximity sensors discussed above can be used independently or in combination to provide a short touch (touch), a long touch, a multi touch, a drag touch ), Flick touch, pinch-in touch, pinch-out touch, swipe touch, hovering touch, and the like. Touch can be sensed.

The ultrasonic sensor can recognize the position information of the object to be sensed by using ultrasonic waves. Meanwhile, the controller 380 can calculate the position of the wave generating source through the information sensed by the optical sensor and the plurality of ultrasonic sensors. The position of the wave source can be calculated using the fact that the light is much faster than the ultrasonic wave, that is, the time when the light reaches the optical sensor is much faster than the time the ultrasonic wave reaches the ultrasonic sensor. More specifically, the position of the wave generating source can be calculated using the time difference with the time when the ultrasonic wave reaches the reference signal.

The camera 321 includes at least one of a camera sensor (for example, a CCD, a CMOS, etc.), a photo sensor (or an image sensor), and a laser sensor.

The camera 321 and the laser sensor may be combined with each other to sense the touch of the sensing object with respect to the three-dimensional stereoscopic image. The photosensor can be laminated to the display element, which is adapted to scan the movement of the object to be detected proximate to the touch screen. More specifically, the photo sensor mounts a photodiode and a transistor (TR) in a row / column, and scans the contents loaded on the photosensor using an electrical signal that varies according to the amount of light applied to the photodiode . That is, the photo sensor performs coordinate calculation of the object to be sensed according to the amount of change of light, and position information of the object to be sensed can be obtained through the calculation.

The display unit 351 displays (outputs) information to be processed by the mobile terminal 300. For example, the display unit 351 can display UI (User Interface) and GUI (Graphic User Interface) information according to execution screen information of an application program driven by the mobile terminal 300 or such execution screen information .

In addition, the display unit 351 may be configured as a stereoscopic display unit for displaying a stereoscopic image.

In the stereoscopic display unit, a three-dimensional display system such as a stereoscopic system (glasses system), an autostereoscopic system (no-glasses system), and a projection system (holographic system) can be applied.

The sound output unit 352 can output audio data received from the wireless communication unit 310 or stored in the memory 370 in a call signal reception mode, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, The sound output unit 352 also outputs sound signals related to functions (e.g., call signal reception sound, message reception sound, etc.) performed in the mobile terminal 300. [ The sound output unit 352 may include a receiver, a speaker, a buzzer, and the like.

The haptic module 353 generates various tactile effects that the user can feel. A typical example of the haptic effect generated by the haptic module 353 may be vibration. The intensity and pattern of the vibration generated in the haptic module 353 can be controlled by the user's selection or setting of the control unit. For example, the haptic module 353 may combine and output different vibrations or sequentially output the vibrations.

In addition to vibration, the haptic module 353 may be configured to perform various functions such as a pin arrangement vertically moving with respect to the contact skin surface, a spraying force or suction force of the air through the injection port or the suction port, a touch on the skin surface, And various tactile effects such as an effect of reproducing a cold sensation using an endothermic or exothermic element can be generated.

The haptic module 353 can transmit a tactile effect through direct contact, and can be implemented so that a user can feel a tactile effect through a muscular sense such as a finger or an arm. The haptic module 353 may include two or more haptic modules according to the configuration of the mobile terminal 300.

The light output unit 354 outputs a signal for notifying the occurrence of an event using the light of the light source of the mobile terminal 300. Examples of events that occur in the mobile terminal 300 may include message reception, call signal reception, missed call, alarm, schedule notification, email reception, information reception through an application, and the like.

The signal output from the light output unit 354 is implemented as the mobile terminal emits light of a single color or a plurality of colors to the front or rear surface. The signal output may be terminated by the mobile terminal detecting the event confirmation of the user.

The infrared ray output unit 355 may output an infrared ray signal for controlling an external device. The external device is an apparatus equipped with an infrared receiver, and can be controlled such that the power is turned on / off according to an infrared signal. For example, the external device may be a TV, a light, an air conditioner, a refrigerator, a washing machine, a boiler, a switch, a plug, a gas lock, and a home CCTV.

The infrared signal can be realized by emitting infrared light from the light emitting portion. For example, the light emitting portion may be implemented as an infrared light emitting diode. Such an infrared signal can be output based on a user request.

The interface unit 360 serves as a pathway to all external devices connected to the mobile terminal 300. The interface unit 360 receives data from an external device or supplies power to each component in the mobile terminal 300 or transmits data in the mobile terminal 300 to an external device. For example, you can connect a device with a wired / wireless headset port, external charger port, wired / wireless data port, memory card port, identification module, audio I / O port, video I / O port, May be included in the interface unit 360.

The identification module is a chip for storing various information for authenticating the usage right of the mobile terminal 300 and includes a user identification module (UIM), a subscriber identity module (SIM) A universal subscriber identity module (USIM), and the like. Devices with identification modules (hereinafter referred to as "identification devices") can be manufactured in a smart card format. Accordingly, the identification device can be connected to the terminal 300 through the interface unit 360. [

When the mobile terminal 300 is connected to the external cradle, the interface unit 360 may be a path through which the power from the cradle is supplied to the mobile terminal 300, May be a channel through which the mobile terminal 300 is transmitted. Various command signals or power from the cradle can be operated as a signal for recognizing that the mobile terminal 300 is correctly mounted on the cradle.

The memory 370 may store a program for the operation of the control unit 380 and temporarily store input / output data (e.g., a phone book, a message, a still image, a moving picture, etc.). The memory 370 may store data on vibration and sound of various patterns outputted when a touch is input on the touch screen.

The memory 370 may be a flash memory type, a hard disk type, a solid state disk type, an SDD type (Silicon Disk Drive type), a multimedia card micro type ), Card type memory (e.g., SD or XD memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read memory, a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, and / or an optical disk. The mobile terminal 300 may operate in association with a web storage that performs the storage function of the memory 370 over the Internet.

Meanwhile, as described above, the control unit 380 controls an operation related to an application program and an overall operation of the mobile terminal 300. [ For example, when the state of the mobile terminal meets a set condition, the controller 380 can execute or release a lock state for restricting input of a user's control command to applications.

The control unit 380 performs control and processing related to voice communication, data communication, video call, and the like, or performs pattern recognition processing to recognize handwriting input or drawing input performed on the touch screen as characters and images, respectively . Further, the controller 380 may control any one or a plurality of the above-described components in order to implement various embodiments described below on the mobile terminal 300 according to the present invention.

The power supply unit 390 receives external power and internal power under the control of the controller 380 and supplies power necessary for operation of the respective components. The power supply unit 390 includes a battery, the battery may be an internal battery configured to be chargeable, and may be detachably coupled to the terminal body for charging or the like.

In addition, the power supply unit 390 may include a connection port, and the connection port may be configured as an example of an interface 360 through which an external charger for supplying power for charging the battery is electrically connected.

As another example, the power supply unit 390 may be configured to charge the battery in a wireless manner without using the connection port. In this case, the power supply unit 390 may use at least one of an inductive coupling method based on a magnetic induction phenomenon from an external wireless power transmission apparatus or a magnetic resonance coupling method based on an electromagnetic resonance phenomenon Power can be delivered.

In the following, various embodiments may be embodied in a recording medium readable by a computer or similar device using, for example, software, hardware, or a combination thereof.

In the above description, the terminal may be called various names such as an audio device, a video device, and the like according to its implementation.

On the other hand, the terminal can be extended to a wearable device which can be worn on the body beyond the dimension that the user holds mainly in the hand. Such wearable devices include smart watch, smart glass, and HMD. Hereinafter, examples of terminals extended to a wearable device will be described.

The wearable device can be made to be capable of mutually exchanging (or interlocking) data with the other terminal 100. The short range communication module 114 can detect (or recognize) a wearable device capable of communicating around the terminal 100. [ If the detected wearable device is a device authenticated to communicate with the terminal 100, the control unit 180 transmits at least a part of the data processed by the terminal 100 to the wearable device through the short distance communication module 114 . Accordingly, the user can use the data processed by the terminal 100 through the wearable device. For example, when a phone call is received in the terminal 100, it is possible to make a phone call through the wearable device or to confirm the received message through the wearable device when the message is received in the terminal 100. [

4 is a perspective view showing an example of a watch-type terminal 200 according to another embodiment of the present invention.

4, a watch-type terminal 400 includes a body 401 having a display unit 451 and a band 402 connected to the body 401 and configured to be worn on the wrist. In general, the terminal 400 may include features of the terminals of Figures 3A-3C or similar features.

The main body 401 includes a case which forms an appearance. As shown, the case may include a first case 401a and a second case 401b that provide an internal space for accommodating various electronic components. However, the present invention is not limited to this, and one case may be configured to provide the internal space, so that a unibody terminal 400 may be realized.

The watch-type terminal 200 is configured to be capable of wireless communication, and the main body 401 may be provided with an antenna for the wireless communication. On the other hand, the antenna can expand its performance by using a case. For example, a case comprising a conductive material may be configured to electrically connect with the antenna to extend the ground or radiating area.

A display unit 451 is disposed on a front surface of the main body 401 to output information, and a touch sensor is provided on the display unit 451 to implement a touch screen. As shown in the figure, the window 451a of the display unit may be mounted on the first case 401a to form a front surface of the terminal body together with the first case 401a.

The main body 401 may include an audio output unit 452, a camera 421, a microphone 422, a user input unit 423, and the like. When the display unit 451 is implemented as a touch screen, the display unit 451 may function as a user input unit 423, so that the main body 401 may not have a separate key.

The band 402 is worn on the wrist so as to enclose the wrist, and may be formed of a flexible material for easy wearing. As an example, the band 402 may be formed of leather, rubber, silicone, synthetic resin, or the like. The band 402 may be detachably attached to the main body 401 and may be configured to be replaceable by various types of bands according to the user's preference.

On the other hand, the band 402 can be used to expand the performance of the antenna. For example, the band may include a ground extension (not shown) that is electrically connected to the antenna and extends the ground region.

The band 402 may be provided with a fastener 402a. The fastener 402a may be embodied by a buckle, a snap-fit hook structure, or a velcro (trademark), and may include a stretchable section or material have. 4 shows an example in which the fastener 402a is embodied as a buckle.

5 is a perspective view illustrating an example of a glass-type terminal 300 according to another embodiment of the present invention.

The glass-type terminal 500 is configured to be worn on the head of a human body, and a frame portion (a case, a housing, etc.) may be provided. The frame portion may be formed of a flexible material to facilitate wearing. 5 illustrates that the frame portion includes a first frame 501 and a second frame 502 of different materials. In general, terminal 500 may include features of terminal 300 of Figures 3A-3C or similar features.

The frame portion is supported on the head portion, and a space for mounting various components is provided. As shown in the figure, electronic components such as a control module 580, an audio output module 552, and the like may be mounted on the frame portion. Further, a lens 503 covering at least one of the left and right eyes may be detachably mounted on the frame portion.

The control module 580 controls various electronic components included in the terminal 500. The control module 580 can be understood as a configuration corresponding to the control unit 380 described above. 5 illustrates that the control module 580 is installed in the frame portion on one side of the head. However, the position of the control module 580 is not limited thereto.

The display unit 551 may be implemented as a head-mounted display (HMD). The HMD type refers to a display method that is mounted on the head and displays images directly in front of the user's eyes. When the user wears the glass-type mobile terminal 500, the display unit 551 may be disposed so as to correspond to at least one of the left and right eyes so as to provide images directly to the user's eyes. In this figure, the display unit 551 is located at a portion corresponding to the right eye so that an image can be output toward the user's right eye.

The display unit 551 can project an image using the prism to the user's eyes. Further, the prism may be formed to be transmissive so that the user can view the projected image and the general view of the front (the range that the user views through the eyes) together.

The image output through the display unit 551 may be overlapped with the general visual field. The terminal 500 can provide an Augmented Reality (AR) in which a virtual image is superimposed on a real image or a background and displayed as a single image using the characteristics of the display.

The camera 521 is disposed adjacent to at least one of the left eye and the right eye, and is configured to photograph a forward image. Since the camera 521 is located adjacent to the eyes, the camera 521 can acquire a scene viewed by the user as an image.

Although the camera 521 is provided in the control module 580 in FIG. 5, it is not limited thereto. The camera 521 may be installed in the frame part, or may be provided in a plurality of ways to acquire a stereoscopic image.

The glass-type terminal 500 may have user inputs 523a and 523b operated to receive control commands. The user input units 523a and 523b can be employed in any manner as long as the user operates in a tactile manner such as touch, push, or the like with a tactile impression. 5 illustrates that the frame unit and the control module 580 are provided with user input units 523a and 523b of a push and touch input method, respectively.

In addition, the glass-type terminal 500 may be provided with a microphone (not shown) for receiving sound and processing it as electrical voice data and an acoustic output module 552 for outputting sound. The sound output module 552 may be configured to transmit sound in a general sound output mode or a bone conduction mode. When the sound output module 552 is implemented in a bone conduction manner, when the user wears the terminal 500, the sound output module 552 closely contacts the head and vibrates the skull to transmit sound.

Next, a communication system that can be implemented through a terminal according to the present invention will be described.

First, the communication system may use different wireless interfaces and / or physical layers. For example, wireless interfaces that can be used by a communication system include Frequency Division Multiple Access (FDMA), Time Division Multiple Access (TDMA), Code Division Multiple Access (CDMA) , Universal Mobile Telecommunications Systems (UMTS) (especially LTE, LTE-A), Global System for Mobile Communications (GSM), and the like.

Hereinafter, for the convenience of explanation, the description will be limited to CDMA. However, it is apparent that the present invention can be applied to all communication systems including an OFDM (Orthogonal Frequency Division Multiplexing) wireless communication system as well as a CDMA wireless communication system.

A CDMA wireless communication system includes at least one terminal 100, at least one base station (BS) (also referred to as a Node B or Evolved Node B), at least one Base Station Controllers (BSCs) , And a Mobile Switching Center (MSC). The MSC is configured to be coupled to a Public Switched Telephone Network (PSTN) and BSCs. The BSCs may be paired with the BS via a backhaul line. The backhaul line may be provided according to at least one of E1 / T1, ATM, IP, PPP, Frame Relay, HDSL, ADSL or xDSL. Thus, a plurality of BSCs may be included in a CDMA wireless communication system.

Each of the plurality of BSs may comprise at least one sector, and each sector may comprise an omnidirectional antenna or an antenna pointing to a particular direction of radial from the BS. In addition, each sector may include two or more antennas of various types. Each BS may be configured to support a plurality of frequency assignments, and a plurality of frequency assignments may each have a specific spectrum (e.g., 1.25 MHz, 5 MHz, etc.).

The intersection of sector and frequency assignment may be referred to as a CDMA channel. The BS may be referred to as a base station transceiver subsystem (BTSs). In this case, a combination of one BSC and at least one BS may be referred to as a "base station ". The base station may also indicate a "cell site ". Alternatively, each of the plurality of sectors for a particular BS may be referred to as a plurality of cell sites.

A broadcast transmission unit (BT) transmits a broadcast signal to terminals 100 operating in the system. The broadcast receiving module 111 shown in FIG. 1A is provided in the terminal 100 to receive a broadcast signal transmitted by the BT.

In addition, the CDMA wireless communication system may be associated with a GPS for identifying the position of the terminal 100. The satellite aids in locating the terminal 100. Useful location information may be obtained by two or more satellites. Here, the position of the terminal 100 can be tracked using all the techniques capable of tracking the location as well as the GPS tracking technology. Also, at least one of the GPS satellites may optionally or additionally be responsible for satellite DMB transmission.

The location information module 115 provided in the terminal is for detecting, computing, or identifying the location of the terminal, and may include a GPS module and a Wi-Fi module as typical examples. Optionally, the location information module 115 may perform any of the other functions of the wireless communication unit 110 to obtain replacement or additionally data regarding the location of the terminal.

The GPS module 115 calculates distance information and accurate time information from three or more satellites and then applies trigonometry to the calculated information to accurately calculate three-dimensional current position information according to latitude, longitude, and altitude can do. At present, a method of calculating position and time information using three satellites and correcting an error of the calculated position and time information using another satellite is widely used. In addition, the GPS module 115 can calculate speed information by continuously calculating the current position in real time. However, it is difficult to accurately measure the position of a terminal using a GPS module in a shadow area of a satellite signal, such as in a room. Accordingly, a Wi-Fi location tracking system (WPS) can be utilized to compensate for GPS-based positioning.

The Wi-Fi location tracking system (WPS) tracks the location of the terminal 100 using a Wi-Fi module included in the terminal 100 and a wireless AP transmitting or receiving a wireless signal with the Wi-Fi module Technology refers to a WLAN-based location positioning technique using Wi-Fi.

The Wi-Fi location tracking system may include a Wi-Fi location server, a terminal 100, a wireless AP connected to the terminal 100, and a database in which certain wireless AP information is stored.

The terminal 100 connected to the wireless AP can transmit a location information request message to the Wi-Fi location server.

The Wi-Fi location server extracts information of the wireless AP connected to the terminal 100 based on the location information request message (or signal) of the terminal 100. [ The information of the wireless AP connected to the terminal 100 may be transmitted to the Wi-Fi positioning server through the terminal 100 or may be transmitted from the wireless AP to the Wi-Fi positioning server.

The information of the wireless AP to be extracted based on the location information request message of the MS 100 includes a MAC address, an SSID (Service Set IDentification), a Received Signal Strength Indicator (RSSI), a Reference Signal Received Power (RSRP) Signal Received Quality), channel information, Privacy, Network Type, Signal Strength, and Noise Strength.

As described above, the Wi-Fi location server can receive the information of the wireless AP connected to the terminal 100 and extract the wireless AP information corresponding to the wireless AP to which the terminal is connected from the pre-established database. The information of any wireless APs stored in the database includes at least one of MAC address, SSID, channel information, privacy, network type, radius coordinates of the wireless AP, building name, Available), the address of the AP owner, and the telephone number. In this case, in order to remove the wireless AP provided using the mobile AP or the illegal MAC address in the positioning process, the Wi-Fi location server may extract only a predetermined number of wireless AP information in order of RSSI.

Then, the Wi-Fi location server can extract (or analyze) the location information of the terminal 100 using at least one wireless AP information extracted from the database. And compares the received information with the received wireless AP information to extract (or analyze) the location information of the terminal 100.

As a method for extracting (or analyzing) the position information of the terminal 100, a Cell-ID method, a fingerprint method, a triangulation method, and a landmark method can be utilized.

The Cell-ID method is a method for determining the location of the wireless AP having the strongest signal strength among the neighboring wireless AP information collected by the terminal as the location of the terminal. Although the implementation is simple, it does not cost extra and it can acquire location information quickly, but there is a disadvantage that positioning accuracy is lowered when the installation density of the wireless AP is low.

The fingerprint method collects signal strength information by selecting a reference position in a service area, and estimates the position based on the signal strength information transmitted from the mobile terminal based on the collected information. In order to use the fingerprint method, it is necessary to previously convert the propagation characteristics into a database.

The triangulation method is a method of calculating the position of the terminal based on the coordinates of at least three wireless APs and the distance between the terminals. In order to measure the distance between the terminal and the wireless AP, the signal intensity is converted into distance information, the time of arrival (ToA) of the wireless signal, the time difference of arrival (TDoA) Angle of Arrival (AoA) or the like can be used.

The landmark method is a method of measuring the position of a terminal using a landmark transmitter that knows the location.

Various algorithms can be utilized as a method for extracting (or analyzing) the location information of the terminal.

The extracted location information of the terminal 100 is transmitted to the terminal 100 through the Wi-Fi location server, so that the terminal 100 can acquire the location information.

The terminal 100 may be connected to at least one wireless AP to obtain location information. At this time, the number of wireless APs required to acquire the location information of the terminal 100 can be variously changed according to the wireless communication environment in which the terminal 100 is located.

1A, short-range communication technologies such as Bluetooth, RFID, IrDA, UWB, ZigBee, NFC (Near Field Communication), and Wireless USB may be applied to the terminal according to the present invention. have.

Among them, the NFC module provided in the terminal supports non-contact type short-range wireless communication at a distance of about 10 cm. The NFC module may operate in either a card mode, a reader mode, or a P2P mode. In order for the NFC module to operate in the card mode, the terminal 100 may further include a security module for storing card information. Here, the security module may be a physical medium such as a UICC (Universal Integrated Circuit Card) (for example, SIM or USIM), Secure micro SD and a sticker, or a logical medium (e.g., embedded SE It is possible. Data exchange based on SWP (Single Wire Protocol) can be made between NFC module and security module.

When the NFC module is operated in the card mode, the terminal can transmit the stored card information to the outside such as a conventional IC card. Specifically, when a terminal storing card information of a payment card such as a credit card or a bus card is brought close to the fare payment machine, the mobile local payment can be processed, and the terminal storing the card information of the access card can be processed , The approval process for access can be started. Cards such as credit cards, transportation cards, and access cards are mounted on the security module in the form of an applet, and the security module can store card information on the mounted card. Here, the card information of the payment card may be at least one of a card number, balance, and usage details, and the card information of the access card may include at least one of a name, a number (e.g., It can be one.

When the NFC module is operated in the reader mode, the terminal can read data from an external tag. At this time, the data received from the mobile terminal by the tag may be coded into a data exchange format (NFC Data Exchange Format) defined by the NFC Forum. In addition, the NFC Forum defines four record types. Specifically, the NFC Forum defines four RTDs (Record Type Definitions) including Smart Poster, Text, Uniform Resource Identifier (URI), and General Control. If the data received from the tag is a smart poster type, the control unit executes a browser (e.g., an Internet browser), and if the data received from the tag is a text type, the control unit can execute the text viewer. If the data received from the tag is a URI type, the control unit executes the browser or makes a telephone call, and if the data received from the tag is of the general control type, appropriate operation can be executed according to the control contents.

When the NFC module operates in the peer-to-peer mode, the terminal can perform P2P communication with another terminal. At this time, LLCP (Logical Link Control Protocol) may be applied to P2P communication. For P2P communication, a connection may be created between the terminal and another terminal. At this time, the generated connection can be divided into a connectionless mode in which one packet is exchanged and terminated, and a connection-oriented mode in which packets are exchanged consecutively. Through peer-to-peer communications, data such as business cards in an electronic form, contact information, digital photos, URLs, and Bluetooth and set-up parameters for Wi-Fi connectivity can be exchanged. However, since the usable distance of NFC communication is short, the P2P mode can be effectively used to exchange small-sized data.

Hereinafter, embodiments related to a control method that can be implemented in a terminal configured as described above will be described with reference to the accompanying drawings. It will be apparent to those skilled in the art that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof.

Conventionally, the terminal is limited to detecting a fall of a specific moment and transmitting an alert. However, in the present invention, the risk of falling is predicted by data measured in everyday life, and control is accordingly performed. In other words, in the present invention, the fall detection rate and the falloff detection are lowered by using context awareness rather than motion fall detection alone, and attention is paid prior to the emergency moment to help the user to take measures before an emergency situation That is the purpose. Thus, according to the present invention, it is possible to estimate and guide the risk factors, and to link the training based on the guidance.

In connection with recognizing a fall or an emergency, it is one way to see the image remotely and the caregiver to directly judge the fall or emergency situation. However, the above method is different from a technical problem, and a problem such as a privacy invasion of an individual is emerging. Therefore, even if the image is automatically recognized, the user can feel the rejection feeling by using the image. Accordingly, the present invention refers to a process of judging a parent by looking at an image, and detects an emergency through a sensor other than the sensor for the image, and utilizes the detected data.

7 is a flowchart illustrating a data processing method of a terminal according to an embodiment of the present invention.

The terminal senses and collects the acceleration data through the sensor (S702).

Based on the acceleration data thus collected, the terminal first determines whether the user is shocked (S704). The terminal determines the user's posture based on the collected acceleration data (S706). Here, for example, if it is determined in step S704 that the user has been impacted, the posture is for judging the posture such as the user being knocked down or wobbled due to the received impact.

Next, the terminal determines whether there is a movement of the user based on the collected acceleration data (S708). Here, in step S708, it is determined whether or not the user moves based on at least one of steps S704 and S706.

For example, the steps S704 to S706 may be sequentially determined as shown in FIG. 7, and only one of the steps may be determined, or may be determined in a different order from that shown in FIG. The steps S704 to S706 may be referred to as an acceleration-based fall algorithm 720.

This acceleration-based fall algorithm 720 may be implemented by adding further steps as shown, or vice versa. In addition, the acceleration-based fall algorithm 720 may be implemented with completely different step (s) not shown. Alternatively, the fall algorithm 720 may be based on totally different data rather than on an acceleration data basis, or in combination with other data. However, for the sake of convenience, it is explained in FIG. 7 that the above steps are sequentially determined.

The terminal according to the present invention senses and collects context-aware data in addition to the determination made through the fall algorithm 720 (S710). Here, the context awareness data may be collected and generated through a sensor in the terminal, or may be generated by receiving data from sensors other than the terminal. The terminal re-verifies whether or not the user has fallen, based on the collected context-aware data, through the fall algorithm 720. In other words, even if the user determines that the user has fallen through the fall algorithm 720, the terminal determines again whether the fall of the determined user is correct based on the situation recognition data other than the acceleration data.

Hereinafter, with reference to FIGS. 8 and 9, the fall algorithm 720 will be described in more detail as follows.

In Fig. 8, a portion 810 describing fall motion and a data graph portion 820 of the fall motion are shown.

The fall motion description part 810 sequentially shows the motions according to falls from the posture A standing from left to right with respect to the horizontal axis and the fall posture D-F. Herein, the falling postures D, E, and F are all a user's lying or lying posture due to falls. If D or E is a falling posture, E or F can be regarded as representing that there is no movement after the fall D or E .

The data graph portion 820 represents the data according to the fall motion in graph form, and the data may be based on an acceleration sensor or a gyro sensor. However, the present invention is not limited thereto.

On the other hand, the data graph portion 820 can be divided into page 4 (Phase 1) 830 to page 4 (Phase 4) 860 referring to the fall motion and the data graph according to the fall motion.

Referring to the data graph, the page before the falling posture D completely falling from the standing posture A is referred to as page 1 (810). The fall position (D) is referred to as page 2 (840). The position before the fall position F after the fall position D is named as Phase 3 (850). Finally, after the fall position (F), we call it page 4.

Referring to falling motion, the impact on the user occurs on page two 840 and can be determined. For example, in page 2 (840), even if a user falls down due to falls, the impact can be re-activated if the impact is not enough to allow the user enough, such as being mental or injured Therefore, if such a degree of shock is reported to a manager or the like, there is a risk of malfunction due to erroneous detection. In such a case, it can be excluded from the fall. In other words, page 2 (840) determines whether the amount of impact due to the user's fall is below a predetermined threshold. Here, the threshold may be arbitrary depending on, for example, the age, health status, sex, and other settings of the user registered or having the terminal.

After the page 3 (850), the terminal can determine whether there is a movement of the user after the fall. This may affect the determination of the amount of impact due to the user's fall in the future, and may also affect the situation recognition data to be described later.

On page 3 (850), the user's movement after the fall is sensed. As a result of the sensing, if there is no movement of the user, the page 3 850 is switched to page 4 (860), and an alarm, a control command, or the like may be transmitted to the manager or his terminal at the switching point.

Referring to FIG. 9, this invention relates to a fall algorithm, and more particularly to attitude measurement.

Referring to Fig. 9, for example, when the sensor is attached to the body, the lying posture can be accurately recognized.

9A is a graph of sensing data for the X, Y, and Z axes. 9B shows the Y axis and the Z axis when the user is standing, and FIG. 9C shows the Y axis and Z axis when the user is lying.

9A, a graph for an X axis 910, a Y axis 920, a Z axis 930, and a vector sum 940 is shown based on the horizontal axis samples and the vertical axis values .

In FIG. 9A, Box 1 shows no impact, Box 2 shows an impact, Box 3 shows no motion, and Box 4 shows an initial state.

In Fig. 9, the gravity axis changes from Y to Z, and the slope can be judged at a very low frequency.

FIG. 10 is a view for explaining a place as one of the parameters of the context-aware data according to an embodiment of the present invention.

The location is described as one of the parameters of various context awareness data according to the present invention. Here, the place may be referred to as various names such as a position and a position.

For example, even if a user who has a terminal primarily includes a fall, there may be a difference in degree. In other words, even if the user hurts, the data on the fall does not necessarily warn the administrator or warn the user of the fall. That is, the user's fall is sufficiently dangerous to the user, and such a situation should be assumed, otherwise unnecessary time may be wasted both for the user and for the administrator.

Accordingly, in the present invention, an attempt is made to provide an accurate and efficient management system by adopting context-aware data in order to more accurately determine whether or not the primary terminal owner has fallen a warning or caution to a manager.

In this regard, context awareness data can be defined in various ways, one of which is to disclose and explain the place first.

If the user is taking a lying position similar to a fall, for example, the conventional sensor may determine that the fall is a fall and notify the manager of the fall. In practice, however, the user may simply be lying comfortably at rest. In other words, even if the user takes a lying posture, whether or not it is the meaning of the fall can be recognized differently depending on the place where the user is lying. For example, when a user is lying in a bed or lying in a kitchen or toilet that is not related to a bed or a rest, the user must interpret and perceive this differently and perform subsequent processing.

With respect to the location parameter, which is one of the context-aware parameters, the user's location is important. That is, whether the current position of the user is indoors, outdoors, an indoor restroom, a bedroom, a living room, or a kitchen may be an important factor in determining the status of the user.

In addition, the current user is in a state in which there is no movement, and its position may be an important factor in determining the state of the user depending on a kitchen, a toilet, a bedroom, a living room, and the like. For example, in the case where the user is located in the toilet room where the user has previously fallen in the state of no movement, a weight can be given to the user in comparison with other places. Or, in the case of a place where many users usually fall, the weight may be given to the other places.

Hereinafter, a terminal location recognition method will be described.

In the case of confirming a place with an image through a camera or the like, when the user is in a lying position or in a motionless position, the user can judge that the user is in an abnormal state immediately if the user is in the kitchen, toilet floor or living room floor. Alternatively, when the indoor and the outdoor are distinguished, it can be known through periodic GPS on or off. For example, it is difficult to acquire sensing data even when the GPS is turned off or turned on in the room. Alternatively, to reduce battery consumption, indoor and outdoor can be divided into different levels of illumination using an illumination sensor. That is, if there is a large amount of illuminance sensed by the illuminance sensor, it is possible to judge that the room is indoors if the outside is reversed.

In addition, it may be possible to confirm the location by turning on the GPS sensor, checking the indoor and outdoor areas, and then turning off the GPS sensor only when there is a difference in the amount of the sensing sensed by the ambient light sensor of the terminal.

On the other hand, the spatial recognition in the room through the terminal can recognize a space by using a received signal strength indicator (RSSI) or a modulated signal using an LED. This will be described later with reference to FIG.

Also, it is possible to use a method of initially calibrating the earth magnetic field value with respect to the space by using a magnetic sensor of the terminal.

Fig. 10 particularly describes the recognition of space with an LED lamp base, in relation to said place.

If light is irradiated from a lighting device or a light emitting device (LED) installed in a room, the ID is modulated by turning on / off very quickly even though it is difficult to recognize by the human eye. 10A, a modulation ID ('101111') of light emitted from a light emitting element provided on a ceiling or the like is irradiated from the light emitting element through the illuminance sensor of the terminal held by the user And obtains a demodulated ID ('101101') from the ID of the modulated light emitting device. Then, the user can recognize the space where the user is located through the ID thus demodulated. 10A to 10C are diagrams illustrating a case where the ID is '101111' in FIG. 10A, the ID is '101100' in FIG. 10B and the ID Quot; 101101 ". The other IDs are respectively mapped to a specific place or space. For example, the IDs in FIG. 10A may be a toilet, the IDs in FIG. 10B may be a room, and the IDs in FIG. 10C may mean a living room.

As described above, according to the present invention, space and ID are mapped in advance as shown in FIG. 10, and the position of the terminal of the user or the user can be recognized through this mapping. On the other hand, in the mapping, not only the space and the ID are mapped, but the spaces are related to the fall place, so the weight for the space can be defined in advance. Meanwhile, in relation to the context-awareness parameter of the present invention, it is possible to detect abnormality status through life pattern analysis or to detect false negative activity (step, distance, calorie), position recognition, ambient ), Etc., can be used to analyze the life pattern and to predict the risk of falling, etc. In the above, a false negative means, for example, a case where a fall is mistakenly regarded as not falling, whereas a false positive The present invention is not limited to this, but may be applied to a display device in a space in which a light is incident by notifying a danger signal to the outside of a fall, a risk of falling, You may be aware of the danger.

FIG. 11 is a diagram for explaining an everyday life pattern and an abnormality as parameters of context-aware data according to an embodiment of the present invention.

In Fig. 11, one of the parameters of the context recognition data will be described with reference to the parameters such as the location and position in Fig. 10 described above.

Most users have a certain life pattern in daily life. Particularly, in the case of an elderly person, a child, a person who is uncomfortable in the body, etc. who need to be aware of the situation, as in the present invention, this life pattern can be a more important factor.

For example, a certain pattern change can be a dangerous factor in the case of the above-mentioned users.

In the case of such a life pattern, it may be set in advance in the terminal, but it may be determined by appropriately combining and updating the life patterns of the same or similar age as the measurement of the periodic life pattern and the average user of the environment.

Accordingly, the terminal can determine beforehand the occurrence of an event such as an accident before the occurrence of an event such as an accident, and prevent or remove factors such as a danger according to the event through attention, observation, etc. can do.

With regard to the life pattern, the following individual elements or elements may be referred to. A change in the total number of steps per day or week of the user, a time spent by the user in each room of the room, a ratio of time spent by the user in the room in the room, a ratio of the user staying indoor / The time spent lying on the bed of the user, the total sleeping time, the user's sleeping time, the time the user moved inside the room, the number of times the user collided with surrounding objects, the difference between the average value of the user's bio- Etc. may be considered as an element of the life pattern parameter. Here, each of the elements or a combination of at least two elements is considered as the life pattern parameter.

Meanwhile, if the terminal collects data on a predetermined time or period basis and then averages them, the terminal determines that it is an abnormal event and generates a notify signal if a difference between the average pattern of the user exceeds a predetermined threshold value It can be delivered to the manager or the like.

In addition, the administrator can manage the user who has the terminal by dividing each terminal into colors in a faster and more convenient manner. The color may also be applied to each parameter collected in the terminal. For example, assuming that many of the above-mentioned elements are considered as life pattern parameters, not all elements are at all dangerous when comparing results. Therefore, the color can be set differently on a per element basis according to the state comparison result of each element. Assuming one element, it is displayed in green color if the element is currently stable at the time of comparison, and displayed in yellow if it is determined to be in a dangerous state, a state requiring attention or observation, and a red Flashing when necessary), so that the management convenience of the manager can be improved. In addition, the color of the terminal may be displayed differently according to the status of the user even in the terminal owned by the user, so that a risk prevention function can be performed by allowing the other terminal to view the terminal of the user so that the status of the user can be determined. For example, when the average value of the specific elements collected through the terminal differs by 10% or more from the predetermined average value at the time of the comparison, the yellow color is changed to the yellow color at the risk prediction step. If the difference is more than 20% . ≪ / RTI > In addition, the terminal may set itself to broadcast an event occurrence to an administrator or a preset server or a terminal at the time of the user's state change or the risk occurrence phase.

On the other hand, even if one of the above-mentioned elements exceeds a preset threshold value or each element does not exceed a predetermined threshold value, it can be determined that an event has occurred even if the total of the elements exceeds a predetermined threshold value. In the latter case, consideration is given to the aspect of the prediction, which can be arbitrarily set. Here, an event may include not only a risk occurrence but also a risk occurrence prediction step.

If the above-described contents are selected on the administrator terminal 1110 of FIG. 11, the current state information 1120 of the selected user is provided.

12 is a diagram for explaining the head direction of a user as one of the parameters of the context-aware data according to an embodiment of the present invention.

Figure 12 illustrates directional data for the user as yet another of the context-aware parameters.

In particular, in the case of FIG. 12, when there is no movement of the user, it is possible to recognize the current direction of the user, specifically the head direction, and recognize the state of the user.

The direction of the user in the normal space, that is, the head direction, may be fixed or fixed in consideration of, for example, the characteristics and purpose of the space. For example, the head direction of the user lying in the bed can be determined according to the bed.

However, if the head direction of the user is not in the existing life pattern after the user has no movement or a large impact, it may be recognized as a risk factor or situation. In particular, the head direction and the like can be sensed through, for example, a geomagnetic sensor or an acceleration sensor provided in the terminal or installed in the corresponding space.

In other words, the terminal can measure the position and direction of the head of the user, the direction of the user's gaze, etc., and use it as the situation recognition data, especially when the user stays in a specific space in the daily life pattern.

Referring to FIG. 12, it is possible to use the sensing of the posture on the space of the user to recognize the situation.

The manager can determine the current state of the user by sensing the user's lying posture and whether the user is moving.

12, for example, the original normal position is lying on the bed and the head direction should also be east on a plane basis. However, if the user is lying not in the bed position and the head direction is not on the east side It can be determined that the state is not normal (Abnormal). On the other hand, in this case, if the user position is not a bed, or if the head direction is not the east direction, or if the head direction is close to the east, it can be displayed as the observation demand state (Observer).

This can be applied to the kitchen, the living room, the bathroom, the entrance, etc., in the same or similar manner, as shown in Fig.

In other words, it may be difficult for a user to accurately measure when falling from a bed, but if the head is different from the existing sleeping posture and is in a different posture and there is no movement, it is transmitted as a control command Lt; / RTI >

In this case, if there is another sensor capable of acquiring an image in the surrounding state in the state determined as described above, it may be more accurate to confirm one more time through the sensor.

In addition, in the case of the intensity of the impact as one of the determining factors of falling, it is difficult for the user to immediately judge it as a fall if the impact strength is less than the threshold value. Accordingly, in the conventional terminal, if it is judged that the fall is caused only by the intensity of the impact, it may be a problem. However, even if the impact intensity for the fall determination is less than the threshold value, It is possible to re-verify or re-determine the accuracy of the fall determination. This makes it possible to prevent falloff detection when the impact strength is high, thereby improving system efficiency.

13 is a diagram for explaining voice or audio as one of parameters of context-aware data according to an embodiment of the present invention.

Next, voice or audio (hereinafter audio) parameters will be described as another of the context-aware parameters.

It is usually difficult to attach the sensor to all parts of the human body. Therefore, when an impact is applied to a portion where the sensor is not attached, the accuracy of impact detection may be deteriorated such that the strength of the impact can not be precisely sensed. In relation to the present invention, falls are particularly important for users who need to monitor, manage, and the like, such as children and the elderly. Such a shock detection or emergency situation can have a great impact even once, You can lower your confidence.

As one solution to the situation described above, an audio sensor can be used. For example, the normal impact is very likely to accompany audio generation. Here, the audio generation may be a sound for the impact itself, but also audio such as moaning sound of the user due to impact.

Accordingly, the terminal analyzes the audio data sensed by the user, and if the level difference is different from the normal voice or the audio level (e.g., db) of the normal user, that is, the average audio level, So that the subsequent processing according to the present invention can be started.

FIG. 13A shows a graph of voice data of a user, and FIG. 13B shows a data graph obtained by analyzing the graph.

Referring to FIGS. 13A and 13B, it can be seen that the user's audio level is significantly different at the first point 1310 and the second point 1320. [ 13A and 13B, the terminal can determine the state of the user, such as a shock or a dangerous situation, from the audio level sensing data of the user, and can accurately sense the state of the user by using it as a context recognition parameter.

13A and 13B, the first point 1310 can be known as an impact point, and the second point 1320 can also be determined as an impact point. Alternatively, an impact may be applied at the first point 1320, and the second point 1320 may be data to re-verify whether the first point 1310 is impacted. In the latter case, after the first impact 1310 is applied, if the impact exceeds the threshold value, the user makes a moan sound 1320 due to the reaction to the impact, One point 1310 may also function as an ancillary data function to recognize that the impact is a real situation.

In particular, in FIG. 13A, the impact sound or the like records a high intensity in the high frequency range and can be measured in the entire frequency band like an impulse signal in FIG. 13B.

On the other hand, regarding the recognition of the situation, whether the sensor is worn can also be recognized as one parameter. Hereinafter, the fall sensor or device (hereinafter, fall-away device) may refer to one of the devices shown in Fig.

For example, if the fall device has not been used by the terminal or the user, it is impossible to measure whether or not the user falls. In connection with this, it is also important to monitor the non-use status of the fall device, and if the fall algorithm operates when there is no use, many false detections may occur.

Accordingly, in connection with the present invention, it is possible to trigger a recognition of wear device wearing through the proximity sensor and the acceleration sensor. For example, when the user wears the fall-down device, movement occurs. Therefore, if the acceleration signal is greater than a specific value, the proximity sensor is turned on and monitored for a predetermined time. On the other hand, in case of charging, it can be confirmed through connection with the cradle. On the other hand, when the fall-down device is worn, the proximity sensor becomes true and movement occurs. However, It can be judged that there is no movement in True City. In particular, it can be recognized when connected to the proximity sensor on the ground.

On the other hand, the fall device can be used as a touch sensor when it is worn directly on the skin. For example, such a fall device can distinguish wear and non-wear due to differences in capacitance between objects and skin. Therefore, the touch sensor has a true value when worn, and the touch sensor can have a false value when not worn.

The fall-down device may also be in the form of a necklace. In this case, the fall device is capable of sensing through a value sensed using a weight sensor, for example. For example, when the specific axis of the sensor is operated as a gravity axis, it can be regarded as a case where the falling device is worn. If the switch is turned off or the specific axis of the sensor is not the gravity axis,

On the other hand, although not shown, status check data can be used as one of the context-aware parameters.

For example, in the physical response 1, it is possible to confirm whether or not the user moves after the stimulation such as sound or vibration through the fall device (or terminal), and gradually increase the intensity of the stimulus. Here, the stimulus may include various contents such as electric stimulation, blood glucose measurement, and the like. Alternatively, it may be requested to the surrounding people via audio or image output.

Next, the physiological state can be judged. For example, a user's basic bio-signal can be an emergency reference material. In this case, it can be used as emergency information according to the quality of the signal, and it can be used as a reference value. In addition, if the user is a chronic disease patient, it is very important to measure the biometric information of the disease as a result of the user's disease record or the like. For example, a vital sign (blood pressure, pulse, body temperature, SpO2, respiration, blood glucose?) Can be measured and sent to the guardian / manager.

Even if the sensor is not added, breathing, heart rate, etc. can be measured with the acceleration sensor used in the fall-down device as shown in Fig.

15A and 15B are graphs of respiration data, and FIGS. 15C to 15E are graphs of heart rate data.

Therefore, referring to FIG. 15, it is possible to measure respiration, heart rate, etc. at the left, right, and center of the chest when the user is lying down, and in this case, It is possible to receive and use signals as much as possible. However, the signal may be weak depending on the characteristics of the floor in case of the elderly with weak heartbeat or lying down.

In summary, when the user (patient) likely to have an abnormality is seen, it is generally grasped whether it is shaking or moving with physical stimulation, and the physical stimulation can be gradually increased to prevent the user from being intrusive at the time of erroneous detection.

FIG. 14 is a diagram for explaining a user's location search as one of the parameters of the context-aware data according to an embodiment of the present invention.

Generally, outdoor use uses GPS to locate a patient, and indoor use wi-fi or RSSI (Received Signal Strength Indicator). However, there may be no beacon in the room, and it is difficult to find the number of floors in a tall building.

Therefore, according to the present invention, as shown in FIG. 14, rather than grasping the absolute position, the manager can communicate the environmental difference between the manager and the user to the manager, and can estimate the position by reporting the relative signals.

First, in the case of the altitude difference, for example, when the first floor is reached, the manager can receive the user's pressure value and temperature value, compare with the value of the first floor, and estimate the approximate altitude value with the altitude conversion formula . Using this estimated altitude value, it is possible to roughly determine which floor the user is located in, so that quick action is possible.

Next, in the case of the temperature difference, if the temperature difference is large, the reliability of the difference in altitude difference of the conversion formula is lowered, so that it can be used as a reliability index. For example, a specific space, such as a basement, may be estimated by temperature alone.

In addition, the position of the user can be estimated by adjusting the ground sensing value, indoor or outdoor classification through the illuminance sensor, and the like.

Alternatively, when the number of users is not known accurately in a high-rise building, the GPS altitude can not be measured, and Bluetooth can be used, but it is possible to sense valid data only at a close range. In addition, although the altitude is outputted through the pressure sensor, the altitude data obtained from the data of only the pressure sensor can not be seen as an absolute value because it can be influenced by the temperature.

Therefore, as shown in FIG. 14, the relative value should be received and compared with the elevation value of the manager, and the temperature difference should be considered. For example, when the temperature inside the building is similar, the approximate position, that is, the number of floors can be calculated through the relative value in the same building by comparing the manager's altitude with the user's altitude.

On the other hand, when the internal temperature of the building is referred to, if the difference between the manager's environment temperature and the user's environment temperature is large, the user's pressure and altitude value can be received and compared with the manager's data. In this case, sensing may be possible even in a different building.

On the other hand, in the case of an outdoor patient having the same number of floors, after the temperature data of the manager and the temperature data of the user are compared with each other after the GPS data has been used up to the neighborhood, the room and the outside are checked with the illuminance data, So that the detailed position can be monitored.

Next, the conventional fall detection algorithm can only detect a fall situation, that is, if it is not a fall, it can not be prevented from being detected by preventing it from falling, so that it is possible to lower the anxiety composition and reliability of the user, According to the present invention, an emergency situation can be detected and prevented in advance by detecting and managing a dangerous area, and services can be always provided even in daily life.

Therefore, according to the present invention, it is possible not only to accurately detect the fall situation similarly to the prior art, but also to detect in advance by caution / observation when there is a risk factor. Further, according to the present invention, even if it is not a dangerous situation, it is possible to continuously detect a related situation, accumulate and learn data to learn the event type, event type (e.g., It is possible to provide services of system efficiency and high reliability.

16 and 17, as described above, unlike the conventional fall algorithm, the steady state, the attention / observation state, and the emergency state can be separately defined, and the detection, processing, and the like can be performed accordingly. Further, as described above, such data can be accumulated and used for learning and more accurate detection and processing.

In particular, FIG. 17 allows for more accurate user state detection and processing through context awareness parameters described above or below, by placing the abnormal state and attention state at attention / observation step 1710. On the other hand, the conventional fall algorithm only ends when it judges only whether or not the fall occurs, and reports it directly to the manager, and there is a problem of protection and trust of the user.

In connection with this fall algorithm, it is very difficult to develop due to consideration of various parameters and various patterns of events.

Therefore, even if any data is sensed, many factors must be taken into account in determining whether or not a fall has occurred. Therefore, the probability of false detection of fall or not, determined from the actually detected data, is very high, .

In addition, fall detection may result in many or serious consequences for the user, and it is therefore necessary to increase the detection accuracy of the fall detection. For example, in the case of non-detection compared to false detection, a worse result can be generated.

In this regard, in the present invention, the simulated data can be applied to the real world to determine the state of the user.

For example, the activity daily living (ADL) evaluation item of the patient can be used to adjust the activity data of the user for one week by measuring the user's activity level, motion information, and sleeping position. On the other hand, during this period, re-calibration may be necessary in case of an emergency, without detecting an emergency situation. In addition, the sensitivity can be applied differently based on the measured data. For example, when there is little activity in the room, the sensitivity may be applied to a higher level, and the adjustment may be set by the administrator at any time.

On the other hand, the patient status questionnaire evaluation can also be used as one data, and the patient's sensitivity can be applied differently through questionnaire evaluation. For example, it may be possible to identify cases that should be more sensitive than the default state. For example, whether you live alone, your age, how many times you have fallen in the past, how many times you have had medical help from a past fall, how much medical help you need on a day-to-day basis, How often a person needs help or hurts while moving, and what medical help is needed for the user in an emergency situation.

In connection with the present invention, a method of informing a person in the room, for example, an administrator, of an external emergency situation will be described as follows.

Conventionally, there has been no way to notify a person in the room passively other than a speaker. For example, they had to listen to the radio mainly, actively check it by telephone, or directly check the information through the TV / Internet. This was particularly the case with management facilities.

On the other hand, in the case of elderly persons, evacuation in the case of external danger may be more difficult, but may not be sensitive to sound, and other notification signals may be required.

Therefore, according to the present invention, in such a situation, it is possible for the user to immediately recognize the notification from the management facility through the vibration of the terminal, for example, the wearable device, as shown in Fig. Alternatively, it may be possible to allow the user to recognize the situation more easily and conveniently by changing the illumination so as to turn on red light through the terminal or to output color or image data of a specific pattern. In addition, if it is recognized that a device equipped with a display such as a TV is located around the user's terminal, information on a warning message, a evacuation method, and the like may be output on the display of the device so that the user can recognize the device. In addition, if a power failure due to a fire or the like occurs, the user is guided to operate sufficiently through the battery, and when an administrator selects a place where the user can evacuate through indoor navigation for evacuation, It is possible. In addition, even in the event of an emergency, a music application such as a music or a classical music that the user likes is executed or music is outputted, thereby inducing the user's mental and physical stability and inducing the user to recognize it through the IoT or peripheral device .

As described above, the fall device according to the present invention can be implemented in the form of a neckband, as shown in FIG. 2, through which a lifemate function of the user can be performed.

Here, the life-mate function is, for example, different from a conventional wrist or necklace shape, and has a neckband shape, is attached to the body center, is advantageous for posture estimation, and is installed and fixed so as to come into contact with the user's body. , Body temperature, etc. of the user can be measured immediately, and falling can also be detected.

For example, a general fall system uses an accelerometer to measure impact and posture to detect falls. In this case, a fall system must be attached to the center of the body to use the posture. However, when implemented in the form of a neckband as in the present invention, the attitude itself can be measured, and the fall can be accurately detected by combining with an accelerometer, an altitude (pressure) sensor, a geomagnetic sensor or the like.

On the other hand, according to the fall device according to the present invention, it is also possible to detect whether or not the user is moving. For example, if the user wearing or wearing the fall-down device has not been in motion for a long time, it can recognize the dangerous state and send a warning to the administrator.

In the case of the installed type fall detection system, a warning is issued when there is no movement for a maximum of 8 hours due to difficulty in distinguishing from the sleep when there is no movement. However, according to the fall device according to the present invention, And can be detected separately. At this time, in the present invention, it is possible to analyze the acceleration data of the sleep experiment and to detect the difference by analyzing the difference from the basic motions in a dangerous situation. A more detailed description thereof will be described later.

Further, according to the fall device according to the present invention, it is possible to detect a behavior that is impacted by a specific value or more even if it is not a fall. In this case, for example, the acceleration device can use the acceleration sensor. For example, when the user feels a large impact through the fall device, the state of the user can be directly detected in the fall-down device, thereby inducing the user's mental and physical stability. In addition, such data can be used cumulatively. For example, a user having a large external impact, such as an elderly person, has a high risk of falling, so that cumulative management can more accurately determine whether the current state or fall is classified.

In addition, with respect to the context-awareness parameter of the present invention, the habit data of daily life is repetitive data, so that the user located in a non-ordinary space can be quickly identified and a behavior with a high risk of falling can be determined in advance . For example, the user may be identified as staying or lying at a place where the user has not stayed in the home, and it is highly likely that the user is in an abnormal state. Therefore, the position and position of the user are monitored at the user's home, Can be detected quickly.

On the other hand, the spatial position can be obtained in various ways, but it is difficult to estimate the detailed posture because the posture must be measured by one sensor.

Therefore, the falling device according to the present invention can measure the head direction and can estimate the posture by measuring the absolute coordinate orientation with the sensing value of the geomagnetic sensor. For example, since the direction in which the user lays down on the bed is constant, it can be determined that if the user lays the bed in the other direction, the risk is high. A more detailed description thereof will be described later.

The habitual life pattern is obtained from the activity-based measurement values such as the activity amount, the indoor / outdoor activity time, the sleeping time, the lying-on time, etc. and detects a case where the boundary exceeds the boundary at a certain point, It can also be used as element data.

Next, with respect to the gait, when the user has (senile) disease, it is often affected by walking. For example, at least one of a walking speed, a stride distance / time, a stride balance, etc. of a user may be measured and an aspect that is reduced or increased through comparison with an average value or a threshold value may be detected to determine whether the user has fallen or abnormality .

The attitude is also the same. It is also possible to measure the degree to which the user bends the waist when the user walks through the neckband-type fall device. For example, in the case of the elderly, the status of the user can be periodically managed and guided by monitoring the change of the waist.

On the other hand, biometric information can generally be used as medical information by measuring a biological signal in a wearable device or a wearable system. However, since the bio-signal varies instantaneously, the measured bio-signal may not have a significant meaning without information on the point of time / action, so that only the biomedical information at a specific time point is measured by the fall- It is possible to detect and determine the state of the user. The specific time may refer to at least one of, for example, immediately after sleeping, maximum / minimum in sleep, immediately after the wake, morning / afternoon, maximum / minimum, and almost no movement. For example, if the value of the heartbeat immediately after the wakeup is higher than that in the normal case, there may be a problem in the physiological condition, and the strong activity should be limited during the day, and the continuous increase may mean a decrease in the physiological function.

It is also possible to predict fall risk, for example, the cause of major falls is caused by cognitive disorders, physiological disorders, and walking disorders. It is possible to prevent the fall risk by monitoring the increase of the factors causing the fall.

Next, FIG. 18 is a diagram for explaining whether or not the user exists in one of the context-aware parameters according to the present invention.

That is, in FIG. 18, when the user generally does not move, it can be seen that the value of the acceleration sensor usually has no change or has a specific value (for example, zero).

18A to 18C are graphs of data obtained by analyzing acceleration data during sleeping, for example, through the wrist.

In the fall-down device according to the present invention, it is possible to sense motion due to breathing and heart vibration because it is worn on the heart part, thereby detecting a case where the breathing and the heart vibration are larger than the maximum value and there is no motion. Also, in the case of sleeping, it is preferable not to detect it as abnormal data because it is in a state of no motion, but it is not a dangerous situation. However, it is different from the fact that the sleeping data can be used as comparison data for detecting an abnormal condition or used for other data, as described later, and can be accumulated and used.

Generally, if there is no movement of the user, it means that there is no movement for a predetermined period of time (for example, 8 hours). If the user does not detect the movement for a predetermined period of time in a real dangerous situation, , Which may affect device or system reliability.

Therefore, it is necessary to analyze the pattern when the sleeping surface is worn by wearing the acceleration sensor, to set a threshold value different from the case where there is no motion, and to detect the sleeping state separately from the case where there is no movement earlier than the time.

FIG. 19 is a flowchart illustrating a fall detection algorithm according to an embodiment of the present invention, and FIG. 20 is a graph of data relating to acceleration and buffer data with reference to FIG.

19 and 20 refer to the page status of FIG. 8 described above.

Referring to FIG. 19, the terminal senses various data (S1902). At this time, the terminal continuously stores the sensed data in a buffer through various sensors such as an acceleration sensor and an altitude sensor. This step may correspond to page 1.

Next, the following process can be performed on page 2. The terminal determines whether the user is a small shock based on the sensed data (S1904). At this time, in order to determine whether the small shock exists, the terminal may perform sampling at 100 Hz. Here, the signal frequency of the fall impact is 100 Hz or more, but if the sampling is increased, the battery consumption is large. If the frequency is less than 100 Hz, the accurate shock value may not be measured. However, the signal frequency is one example, and the present invention is not limited to the signal frequency. On the other hand, the small shock refers to a shock intensity that does not calculate a fall feature when the motion has a value of 2 g or less, for example. This may be for example to reduce battery consumption. On the other hand, the reference of the small shock is not limited to the above-mentioned numerical values in order to facilitate understanding of the present invention and for convenience of explanation.

For example, if the terminal encounters a value exceeding the small shock, the terminal continuously stores the data until the maximum value reaches the center of the buffer. This may be, for example, to extract or detect the best fall feature.

If the shock value is not a small shock, that is, if the shock exceeds a threshold value, it is determined whether the shock value is greater than or equal to a dangerous critical value, that is, a big shock (S1906). If it is determined in step S1906 that a big shock has occurred, the terminal generates a big shock event and warns it (S1908).

As described above, if the user's shock is smaller than the small shock but smaller than the big shock, the terminal calculates to detect the fall feature (S1910). For such fall feature detection, the acceleration features may be used, such as max (VS), max (diff (VS)), high frequency feature max (Z2)

The body posture is estimated from the fall feature value thus calculated (S1912). It is determined whether the estimated body posture corresponds to fall page 3 (S1914).

The body posture estimate may be continuously measured, for example, at a predefined time in the page 3 section. For example, if the posture is calculated as an average value for a predefined time period of an axis perpendicular to the paper plane of the acceleration triaxial axis and the value is, for example, less than a predefined value (for example, 0.6 g) . In this case, if the average value does not move quickly, the average is calculated to be about 0 to 1 g, and if it lies completely horizontally on the ground, it can be 0, and if standing vertically,

The falling page 3 step determination may be made such that the previously calculated falling features are above a threshold value and the ratio of lying and motionless is not less than a predetermined threshold value (e.g., 70%) and the altitude value changed during that period is a threshold value / Key cm).

That is, with reference to page 3, the terminal computes a rate without motion (S1916) and counts the duration of the page 3 step (S1918). Then, the terminal determines whether the motionless rate exceeds a first threshold (S1920).

If the motionless ratio exceeds the first threshold value, it is determined whether the duration of the page 3 step exceeds the second threshold value (S1922).

The terminal then computes the altitude change, which is calculated from the time immediately before the page 2 step to the end of the page 3 step. This can be calculated as the difference between the maximum altitude value and the minimum altitude value (S1924). Since the noise of the altitude sensor is large and the filtering is performed, the response speed is slow and several drops may occur.

It is determined whether the calculated altitude change exceeds a predetermined third threshold value (S1926). If the calculated altitude change value satisfies the above condition, it is detected as fall (S1928). Then, the terminal vibrates itself or through another terminal, and receives confirmation from the user whether it is an actual fall (S1930). At this time, the user can cancel by pressing the button (S1932). Otherwise, if the user is unconscious, the fall warning is automatically provided to the administrator (for example, the care-giver) after a predetermined time elapses (S1934). This step corresponds to page 4.

In FIG. 20, the first line 2010 denotes a small shock reference level, and the intervals 2020, 2030, 2040, and 2050 each indicate a two second buffer. The interval 2060 denotes a fall feature calculation buffer interval, and the interval 2070 denotes a body posture calculation buffer interval. At this time, the three-axis acceleration vector thumb (VS) may be an average value of the square powers of the sensing values of the respective axes.

FIG. 21 is a flowchart illustrating a motion determination algorithm according to an embodiment of the present invention, and FIG. 22 is a diagram illustrating an example of a heartbeat-related data graph associated with FIG.

21 is a flowchart illustrating a concrete determination algorithm for determining no motion.

Normally, falls can be seen as a dangerous situation accompanied by an impact. However, in the case of an elderly person or a child, a dangerous situation can occur even if there is no impact or a very small impact.

When such a dangerous situation occurs, it is possible to notify the surrounding or the manager of the dangerous situation through the wearable device, that is, the emergency call provided in the terminal according to the present invention, but in the case of the user who has lost the child or unconsciousness, none. Therefore, in such a case, a means is required for the terminal to detect the dangerous situation by itself and to automatically notify the dangerous situation to the surrounding area or the like.

Non-wearable sensors, on the other hand, can not distinguish an emergency from sleeping, and can only send a warning to a dangerous situation if there is no movement for longer than the average sleeping time (for example, 8 hours) .

However, the time is very long and can be very dangerous in an emergency situation. Therefore, in such a case, it is necessary to detect and distinguish sleeping and dangerous situations in the terminal according to the present invention, and to perform automatic processing.

In this regard, the terminal according to the present invention can always be attached to or held on a human body, and thus can be judged and processed faster than the installed type. For example, the terminal according to the present invention can be attached to a human body and can distinguish a sleeping state, a motionless state, a falling state, etc. of a user directly from a biorhythm or a biological signal. In the case of the distinction from the sleep state, it can be distinguished from the sensed value by comparison with the user's usual sleep pattern.

In addition, the distinction between sleeping and non-moving is required by analyzing the acceleration (vibration) data of the water surface and the biometric data, and the reference can be set. For example, the terminal obtains the difference between the maximum value (max) and the minimum value (min) of a two second buffer every two seconds for this classification. Based on the values thus obtained, it is possible to judge whether there is a motionless state or a sleep state. For example, a terminal worn on the chest area may have vital vibration (breathing, heartbeat) even if there is no movement of the user, and if the set value of motion is set too small, an alarm may sound even though there is no actual movement. Therefore, when the terminal is worn on the chest area, it is preferable to set it smaller than the set value Thr1 which is larger than the living body vibration and does not move.

On the other hand, when the terminal (or sensor) is attached to the chest area, the maximum value of the bio-vibrations can be obtained through experiments and set to the first threshold value, and the distinction criterion between the no-motion state and the sleep state can be used.

In addition, when there is no movement, it may occur not only in the case of a fall, but also in a sleep state.

In the case of no motion during sleep and in the case of no movement of an emergency situation, when there is a motionless case corresponding to the above condition, the corresponding time is counted.

If the above condition is not satisfied, the motionless time can be reset. If the mobile station has accumulated a continuous motionless time period longer than a second threshold value (for example, 110 minutes), it can determine that the motionless motionless state is anomalous and broadcast a no motion warning.

Here, the second threshold value can be obtained through user's sleep data or objective sleep experiments, and it can be seen that the time without movement of the whole sleeping area is much shorter than the sleeping time due to the periodical change of the sleeping phase. This is described in more detail below in FIG.

When a no-motion event occurs, the terminal provides an alarm, such as a vibration or a buzzer, to the terminal itself or to another terminal, induces a change of the user's consciousness recovery or motionless state, If so, the alarm is canceled, otherwise, an emergency call or alarm is provided to the administrator.

In summary, the terminal performs acceleration sensing (S2102), and calculates the difference between the maximum VS and the minimum VS (S2104). (S2106), and determines whether the difference between the maximum VS and the minimum VS exceeds a first threshold (S2108).

If it is determined in step S2108 that the difference between the maximum VS and the minimum VS exceeds the first threshold value, the motionless time is counted (S2110), and it is determined again whether the counted motionless time exceeds the second threshold value (S2112 ).

If it is determined in step S2112 that the counted motionless time exceeds the second threshold, a no-motion event is generated and a warning is generated (S2114). Thereafter, the maximum VS and minimum VS difference values are re-calculated (S2116), and the maximum VS and minimum VS difference values are determined to be greater than the third threshold (S2118). If it is determined in step S2118 that the event has not been exceeded, the no-motion event is canceled (S2120). Otherwise, the generated event is notified to the administrator (S2122).

FIGS. 22A and 22B show the case where the heart rate is measured at the chest region and the case where the heart rate is not measured, respectively. In each figure, the threshold value, the maximum VS, the minimum VS, and the difference value are shown.

FIG. 23 is a data graph illustrating an algorithm for distinguishing between sleeping and non-sleeping according to an embodiment of the present invention.

FIG. 23 shows an algorithm for distinguishing the sleep time from the motionless interval.

FIG. 23A shows a sleep cycle in a graph form, and FIG. 23B shows actual sleep data.

Referring to FIG. 23, it is very difficult to set a threshold to the motion level in the sleep state. Because people have different sleep patterns.

However, the maximum value can be found by analyzing the no-motion cycle from the water surface.

In this case, in a specific case, the maximum value found through the above-mentioned no-motion period analysis may be exceeded. However, if it is judged that the normal sleep surface is left alone, an abnormal sleep state exceeding the maximum value may become a risk factor for a specific user.

Referring to FIG. 23A, the sleep (REM) section is composed of four steps in total, and the above four steps are repeatedly performed during sleep. Here, in particular, with respect to the present invention, the steps that can be regarded as a motion-free section among the total of four sleep sections may be, for example, a third section and a fourth section.

Therefore, when the third section and the fourth section are continuously maintained or repeated for a long time, it can be determined that the third section and the fourth section are abnormal when judging based on a usual sleep section or step, and in this case, .

Referring to FIG. 23B, a motion-free sleep interval is obtained from a data graph sensed by an acceleration sensor, and the time is accumulated to warn when the continuous duration of the motionless sleep interval exceeds a threshold value.

Therefore, if all the sleeping times are left without movement, a warning will sound during the sleeping at every threshold time, but this state can be judged to be abnormal even in the sleeping side.

24 to 26 are flowcharts illustrating an algorithm for identifying a user in an abnormal state according to an embodiment of the present invention.

As described above, it can be defined variously that one of the context-aware parameters or the status judged through them is abnormal.

Normally, people can repeat certain behaviors in daily life and form a certain pattern. Therefore, if the pattern is formed out of the habitual behavior, there is a high possibility of causing a dangerous situation to a specific user. Therefore, in this case, before the occurrence of an unwanted event, the user or the manager can recognize the situation in advance through caution, warning, etc., and prevent the occurrence of the event through subsequent management or processing.

An example of a user lying down pattern is as follows. The user can generally measure the habituated pattern of the position / orientation when lying down in everyday life and give an abnormal event if lying in a pattern that is out of this. For example, it is possible to detect that the user is lying in a place where the user is not lying, or that the user is lying in a direction that is not lying in the past, and can warn that a careful observation is necessary.

If more abnormal habits or behaviors are found and used for the management of specific users, the prevention effect can be excellent.

On the other hand, the user's lying posture can be determined by sensing through the acceleration sensor, and the user's lying position can be measured in conjunction with the above-mentioned IOT sensor (situation recognition: position). Here, it is also possible to distinguish the user's lying-up / standing by an AC coupled signal of the acceleration sensor.

In addition, the orientation of the user can be estimated by finding an azimuth using a geomagnetic sensor.

In addition, data can be collected to classify patterns based on acceleration after removing abnormal cases that night, and periodic adjustments can be made to specific postures.

24 is a flowchart illustrating the adjustment of setting data at the time of initial setting of a terminal according to an embodiment of the present invention.

The terminal senses the data of the acceleration sensor (S2402) and estimates the body posture (S2404).

Based on the estimated body posture, the terminal determines whether the user is lying and whether there is motion (S2410). After the determination, the terminal generates data on whether the user lie down or not, and the terminal senses the current position of the user through the internal sensor or the external sensor or device and acquires the sensed data (S2412). Then, the sensed data is stored as a normal value (S2414).

Meanwhile, the terminal senses data using the geomagnetism sensor (S2406), and co-ordinates the azimuth angle value with the sensed geomagnetic sensor data (S2408). The covert sensor data is stored as a normal value (S2414).

Through this operation, standard data such as sensors for abnormal determination of the terminal can be collected and accumulated to form a steady value. The process may be performed several times based on the first time or the cumulative data, and the normal value may be extracted using the average value. For example, referring to FIG. 24, the stored value may be the same as the bedroom, the posture lying, and the azimuth angle equal to 50 degrees.

On the other hand, FIG. 25 is a flowchart illustrating the operation mode after FIG. 24, and FIG. 26 is a flowchart illustrating the detailed operation of FIG.

25, the basic steps S2502 to S2514 are similar to or similar to steps S2402 to S2414 of Fig. 24 described above.

25, the operation mode is not the initial setting and adjustment mode but the operation mode after that. In this case, the data obtained and stored in step S2514 of FIG. 25 is compared with the normal data stored in step S2414 of FIG. That is, the data obtained in step S2514 of FIG. 25 is compared with the normal data stored in step S2414 of FIG. 24, and it is determined whether the angle difference or the distance difference exceeds the threshold value (S2516) It is determined that the abnormal event is not currently occurring, that is, an abnormal event has occurred, and the alert can be transmitted (S2518).

The abnormality determination may be made with reference to FIG.

The terminal calculates the difference value between the maximum VS and the minimum VS (S2602), determines whether the difference between the calculated maximum VS and the minimum VS exceeds the threshold (S2604), and finally determines whether an abnormal event has occurred . For example, if the difference between the maximum VS and the minimum VS exceeds the threshold value, it is determined that the current state is an abnormal state or no motion and the alarm is transmitted to the terminal itself or the manager (S2606). In the opposite case, Or the motionless state is canceled or canceled (S2608).

FIGS. 27 to 29 are diagrams for explaining a correlation between a lying posture and an abnormality determination algorithm according to an embodiment of the present invention.

FIGS. 27 to 29 illustrate a method for determining whether or not the user is abnormal based on the user's lying posture data.

Referring to FIG. 27, the lying posture of the user, that is, the body posture, can be mainly measured in the absence or small motion. For example, if an acceleration signal larger than the small shock is present in the buffer, the body posture can be measured only when all the values of the buffer are equal to or smaller than the value of the small shock, without performing a lie calculation. At this time, the data about the position where the user is lying can be received through another installable sensor or the like.

27A shows a standing posture, in which a sensor 2712 is attached to a user's body, and X-axis, Y-axis, and Z-axis data related to the body posture can be sensed. On the other hand, FIG. 27B shows a lying posture, in which a sensor 2714 is attached to the body of the user, so that X-axis, Y-axis, and Z-axis data related to the body posture can be sensed. Here, the sensor of FIG. 27B is the same as the sensor of FIG. 27A, and the values of the X, Y, and Z axes measured based on FIG. 27A are different from those of FIG. 27B. From this, it is possible to sufficiently judge whether the user is lying or standing. On the other hand, when FIG. 27B is compared with FIG. 27A, it can be seen that the directions of the respective axes are different. Therefore, it can be known in advance that the data to be sensed on the basis of each axis are different from each other.

In FIG. 27C, data graphs for x-axis data, y-axis data, and vector thumb are shown from the bottom. FIG. 27D is another representation of the data graph of FIG. 27C.

Referring to FIGS. 27C and 27D, the body posture can be determined to be louder if the absolute value of the average value of the 2-second buffer is greater than 0.35. However, the present invention is not limited to the above numerical values. On the other hand, the threshold value is the value considered to judge that the obliquely leaning condition is also lying elsewhere. On the other hand, the data sensed by the geomagnetic sensor can be distinguished whether it is lying or standing at a stationary place. For example, it can be inferred from the change of the X-axis data. However, it is preferable to estimate the posture through the acceleration because the above value may change even in the case of horizontal movement.

On the other hand, in FIG. 27C, A is the X, Y axis and vector thumb data for the standing state, and B is the X, Y axis and vector thumb data for the lying state. 27C and 27D, it can be seen that the data change between the standing posture and the lying posture is extremely severe, and the posture change can be predicted from this. It can also be used to infer some degree of shock or abnormal condition.

Figs. 28A and 28B show a state in which the user is lying in the other direction, Figs. 27A and 27B, and Figs. 28A and 28B, respectively.

Figs. 28D and 28E correspond to Figs. 27C and 27D, respectively, and show graphs of sensing data for Figs. 28A to 28C.

Referring to FIG. 28D, A is a standing posture in FIG. 28A, B is a posture in a first direction in FIG. 28B, and C is a data graph in a posture in a second direction in FIG. 28C. At the time of each posture change, a sudden change in the data graph can be detected, and event occurrence can be predicted therefrom. Figs. 28E and 28F are graphs of data expressing Fig. 28D in a different manner from the data for Figs. 28A to 28C.

On the other hand, the following equation can be used as a method of estimating the head direction in the user's lying position. Here, the head direction estimation may be performed by a combination of a geomagnetic sensor and an acceleration sensor, for example.

The roll and pitch are obtained through the acceleration sensor (aX, aY, aZ acceleration sensor values) and the rest are obtained via the geomagnetic sensor (mX, mY, mZ geomagnetic sensor values). This can be referred to the following.

roll = atan (aY / sqrt (aX. ^ 2 + aZ. ^ 2));

pitch = atan (sqrt (aY. ^ 2 + aZ. ^ 2) / aX);

Xh = mXcos (roll) + mY sin (pitch) sin (roll) -mZ cos (pitch)

Yh = mYcos (pitch) + mZsin (pitch) (mX, mY, mZ magnetic values)

On the other hand, the head direction can be finally calculated through the following.

Heading = assign (Yh / Xh)

Fit to 0 to 360 degrees for one turn.

if (Xh == 0 && Yh < 0) Yaw = 90;

if (Xh == 0 && Yh > 0) Yaw = 270;

if (Xh < 0) Yaw = 180-assigned (Yh / Xh) * 180 / PI;

if (Xh > 0 && Yh < 0) Yaw = -atan (Yh / Xh) * 180 / PI;

if (Xh > 0 && Yh > 0) Yaw = 360-assigned (Yh / Xh) * 180 / PI;

29 is a diagram related to the azimuth angle in relation to the abnormal judgment.

29A and 29B, it can be determined whether or not the user is lying at the azimuth angle. In particular, based on the data 2910 of the first section in FIG. 29A, it can be seen that the user lies in the corresponding section.

More specifically, the azimuth angle has a value between 0 and 360 degrees. However, due to the nature of the sensor, there is noise, and the value may fluctuate. If the angle is near 360 degrees or 0 degrees, as shown in FIG. 29B, 0 degrees and 360 degrees may be repeated.

Here, for example, there is only a difference of 350 degrees and 10 degrees between 20 degrees, but it can be seen that a difference of 340 degrees is obtained in the case of the bearing.

Also, as shown in FIG. 29A or 29B, if the noise is about 10 degrees at 0 degree, noise of about 350 degrees may be generated by repeating between 355 and 5 degrees.

In order to solve this problem, it is possible to calculate the distance (cos (Azimuth), sin (Azimuth)) by changing the coordinate value. However, in this case, there are many associations. Related contents are as follows.

Sqrt ((Cos (d1) - cos (d2)) ^ 2 + (sin (d1)

Alternatively, when subtracting two values, determine whether to subtract clockwise or counterclockwise, and subtract 360 by subtracting the difference between the subtraction values when the value exceeds the interval '0'. Related contents are as follows.

d1d2 = abs (d1-d2)

if (d1d2 > 180)

d1d2 = abs (360-d1d2)

When the difference between the stored point and the measured point is 1 (d1) or 2 (d2) and the difference exceeds the first threshold value, it can be judged to be abnormal.

An alarm is given to the device after the abnormal state to wake up the user once, and if there is a movement exceeding the second threshold, the motion is canceled, and if not, the administrator can be warned.

FIG. 30 is a flowchart illustrating an abnormality determination method according to another embodiment of the present invention, and FIG. 31 is a data graph related to the abnormality determination of FIG.

Hereinafter, the above-described contents of FIGS. 24 to 29 are continuously accumulated on the basis of the big data, and it is desired to pursue more accuracy.

Referring to FIG. 30, the user's position data and azimuth data are accumulated and stored (S3002-1, S3002-2). 31B, histogram data (Histogram data) is calculated (S3004) on the basis of the accumulated user's position data and azimuth data, and a distribution ratio (DR) is calculated based on the calculated histogram data (S3006).

Then, it is determined whether the distribution diagram DR exceeds the first threshold (S3008). As a result of the determination, if the distribution map DR exceeds the first threshold value, it is determined again whether the difference between the previous normal value and the current normal value is less than the second threshold value (S3010).

According to the determination result, if the difference between the previous normal value and the current normal value exceeds the second threshold, the normal value is updated and readjusted (S3012).

In other words, if there is no abnormality after the data is stored continuously, it is adjusted as data of the normal case.

The data value stored in step S3002 may be different from the initial calibration. In other words, it is a process of finding a value which varies very slowly from the originally adjusted value, which can be referred to as a change in ambient flux or a change in a user's minute habit.

For this purpose, for example, the present invention stores a position, body posture, azimuth data, and the like for one or more days. And stores the normal case of each position and each posture. In this case, for example, in the case of lying down, a cumulative value mode, that is, a median value having the largest cumulative number in the histogram of FIG. 31B is regarded as a periodic adjustment reference value, and in the case of an instantaneous abnormality 3120, do.

As shown in FIG. 31A, periodic adjustment may not be performed in data where the azimuth angle of the lying-down is not concentrated at one angle but distributed at various angles. Particularly, when the distribution diagram DR is smaller than the first threshold value, adjustment may not be performed.

On the other hand, as shown in the histogram of FIG. 31B, the greater the distance from the maximum mode value, the higher the probability of abnormality is.

30 and 31, for example, in the case of a bedroom, the normal case may be seen as lying on the bed and lying head-to-head in the direction of the bed head, and the abnormal case may be in a state It can include a state in which there is no motion, a state in which it is similar to the existing stored pattern, and another state. Or, in the case of a kitchen, an abnormal state would not normally lie in the kitchen, so all lying postures can be viewed as abnormal. On the other hand, if the toilet is lying in the bathtub, it can be judged as normal, but if the bathtub is laid with a predetermined angle in the direction of the bathtub or if it maintains the posture for a predetermined time or more in a direction in which the direction is oblique, it can be regarded as abnormal. Finally, in the case of a living room, it can be seen as a normal state when lying on a sofa or lying on the floor and taking a sleep, but it may be regarded abnormal if it is laid in a direction different from the direction of sitting or lying on the sofa and kept for a predetermined time or more. For example, when watching a TV mainly on a sofa, if the direction is viewed in the normal direction or in the normal direction, it may be seen as abnormal when lying in the opposite direction.

According to at least one of the various embodiments of the present invention described above, the input signal may be recognized and the recognized input signal may be analyzed based on artificial intelligence to correspond to the input signal, Even if the signal is not audio, it can be processed easily and quickly, and an adaptive response is provided based on an event such as a situation of a speaker (user) by using a display of another adjacent device, thereby improving the convenience .

The present invention described above can be embodied as computer-readable codes on a medium on which a program is recorded. The computer readable medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of the computer readable medium include a hard disk drive (HDD), a solid state disk (SSD), a silicon disk drive (SDD), a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, , And may also be implemented in the form of a carrier wave (e.g., transmission over the Internet). Also, the computer may include a control unit 180 of the terminal. Accordingly, the above description should not be construed in a limiting sense in all respects and should be considered illustrative. The scope of the present invention should be determined by rational interpretation of the appended claims, and all changes within the scope of equivalents of the present invention are included in the scope of the present invention.

110: Smart control ring device
610: Signal receiving section 620:
622: Analysis section 624: AI processing section
630: Signal processor 640: Memory
650: Signal output section

Claims (15)

A first sensor;
A second sensor for sensing context aware data; And
Extracting first data from the sensing data of the first sensor to determine whether the user is in a first state, extracting second data from the sensing data of the second sensor if the user is in the first state, And if the user is in the second state, generates a control command and transmits the control command to the external terminal. The method according to claim 1,
Wherein the first data comprises:
Shock data. 3. The method of claim 2,
The first state determination may include:
And determines that the impact data exceeds a first threshold value. The method of claim 3,
Wherein the second data comprises:
The at least one of the position data, the movement data, the illumination data, the GPS data, the position data, the temperature data, the geomagnetism data, the pressure data, the altitude data and the biometric data. 5. The method of claim 4,
The second state determination may include:
And when the second threshold is exceeded based on at least one of the first data and the second data, falls. 5. The method of claim 4,
The second state determination may include:
And judges the fall only when the impact data exceeds the first threshold as a result of the first state determination and the second data exceeds the second threshold. The method according to claim 1,
The second state may comprise:
Wherein the falling state is divided into a plurality of page states,
The first page state is when the first state is not satisfied,
The second page state is determined based on the lying position of the second data for the second state determination and satisfying the first state,
The third page state is a case where the no-motion ratio in the second page state exceeds a third threshold,
And the fourth page state is determined to be a case in which the duration of no motion in the third page state exceeds a fourth threshold value. 8. The method of claim 7,
Wherein,
And determines the third or fourth page state as a fall state when determining the second state. 8. The method of claim 7,
The rate of motionlessness and the duration of no-motion-
Wherein the determination is made based on the acceleration data sensed by the acceleration sensor, which is one of the first and second sensors. The method according to claim 6,
Wherein the second data comprises:
Further comprising sleep data and azimuth data. 11. The method of claim 10,
Wherein,
Upon determining the second state,
The user accumulates and averages the surface data and the azimuth data of the user sensed through the terminal,
Compares the sleeping data with motion data to determine whether or not the terminal user is sleeping,
And compares the azimuth data with the motion data to determine whether the user is sleeping and whether the user is in a falling state. 12. The method of claim 11,
The azimuth data may include:
Wherein the terminal is related to sensing data for the head direction of the terminal user. The method according to claim 1,
The terminal comprises:
A voice recognition function, or an artificial intelligence function,
Wherein the terminal is implemented in any one of a neckband, a brooch, and a necklace including at least one pad attachable to a human body of a terminal user. The method according to claim 1,
Wherein the first sensor comprises:
An acceleration sensor, a gyro sensor, an altitude sensor, and a pressure sensor. The method according to claim 1,
Wherein the second sensor comprises:
A geomagnetic sensor, an illuminance sensor, and a GPS sensor.

Images