KR101689258B1 - System and method of identifying person based on motion recognition - Google Patents

Abstract

An operation signal acquisition device for extracting and transmitting an operation signal from the operation of the user and an individual identification device for generating the individual identification information by signal processing the operation signal transmitted from the operation signal acquisition device, A feature information extracting unit for extracting feature information from the operation signal, an operation feature information storing unit for storing operation feature information of a plurality of users including the user, And an individual identification information generating unit for generating the individual identification information of the user by comparing the feature information with the feature information.

According to the present invention, it is possible to perform personal identification that is robust to environmental noise, even when the user does not need to cooperate with the individual using the individual characteristics of the user.

Motion characteristics, feature vectors, learning, personally identifiable information

Description

[0001] SYSTEM AND METHOD OF IDENTIFYING PERSON BASED ON MOTION RECOGNITION [0002]

The present invention relates to an operation-based personal identification system and method, and more particularly, to an operation-based personal identification system and method using an operation characteristic of a user to perform personal identification that is robust against environmental noise, Based personal identification system and method.

Personal identification has been widely used for the purpose of providing security or personalization services in various applications. In the case where the personal identification is applied, an authentication process of inputting a password for using a personal portable device or a computer, or a process of inputting a face or voice to identify a user to utilize a robot or an agent service .

The conventional personal identification method can be largely classified into a password input method and a biometric authentication method. In the biometrics authentication method, characteristic information such as fingerprint, iris, face, and voice is detected by using various sensors, and then recognized Thereby identifying the user.

The above-described personal identification method is suitable for personal authentication for security. That is, it is mainly applied to perform specific actions or pre-verification steps to receive a specific service. In the preliminary verification step, the user must cooperatively perform a process for identifying a person, such as inputting information for personal identification, sticking a finger to the sensor for fingerprint recognition, or placing the eye on the recognition sensor for iris recognition.

In situations where personal authentication for security is the goal, it is natural for the user to recognize and cooperate with the process for personal identification. However, it may be inconvenient and unnatural for a user to recognize and cooperate with the process of personal identification in the case of providing personalized services.

On the other hand, face recognition or voice-based speaker recognition is a method in which a user can identify an individual even when the user does not recognize the individual identification process. For example, it is possible to identify the user by recognizing the user by sensing the user's face or voice signal in a normal situation. However, the conventional method is also affected by environmental noise. For example, facial recognition has a limitation in that the recognition performance is greatly deteriorated according to the illumination and the background condition, and the facial face is seen in the camera and it is necessary to stop for a while to properly recognize the face. There is a limit to apply to real life.

It is an object of the present invention to provide an operation-based personal identification system and a method using an operation characteristic of a user to perform an individual identification that is robust to environmental noise while not requiring the user to cooperate with the individual.

Another object of the present invention is to provide a computer-readable recording medium on which a program for realizing each step of the operation-based personal identification method is recorded.

According to an aspect of the present invention, there is provided an apparatus and method for controlling an operation of a mobile terminal, the mobile terminal including an operation signal acquisition device for extracting and transmitting an operation signal from a user operation, Wherein the personal identification apparatus comprises: a feature information extracting unit that extracts feature information from the operation signal; an operation feature information storage unit that stores operation feature information of a plurality of users including the user; And an individual identification information generation unit for generating individual identification information of the user by comparing the feature information with the operation characteristic information of the plurality of users stored in advance.

In the operation-based personal identification system according to the present invention, the operation signal acquisition device may include an inertial sensor for extracting the operation signal.

In the operation-based personal identification system according to the present invention, the inertial sensor may include an acceleration sensor or an angular velocity sensor.

Further, in the operation-based personal identification system according to the present invention, the personal identification apparatus may further include a preprocessing unit for receiving the operation signal and performing preprocessing, wherein the feature information extraction unit extracts, from the operation signal subjected to the preprocessing, The feature information can be extracted.

In the operation-based personal identification system according to the present invention, the preprocessor may perform the preprocessing by distinguishing the operation signal from a section in which the operation of the user is actually performed and a section in which the operation of the user is not performed .

In the operation-based personal identification system according to the present invention, the feature information extraction unit may extract the feature information from the operation signal in the form of a feature vector.

Also, in the operation-based personal identification system according to the present invention, the operation feature information storage unit may store the operation feature information of the plurality of users as feature vectors.

Further, in the operation-based personal identification system according to the present invention, the personal identification apparatus may further include a user identification information obtaining unit for obtaining the identification information of the user, and the operation feature information storing unit may store, And store the feature information as the action feature information for the user.

In the operation-based personal identification system according to the present invention, the feature information is a feature vector type, and the operation feature storage unit stores the feature vector or information obtained by learning the feature vector as the operation feature information for the user have.

Further, in the operation-based personal identification system according to the present invention, the identification information of the user may further include additional information including at least one of the age, sex, and taste of the user.

In the operation-based personal identification system according to the present invention, the user identification information obtaining unit may receive the identification information of the user through pre-designated input means, or may read the identification information of the user stored in advance.

The present invention also provides an operation-based personal identification method comprising the steps of: (a) receiving an operation signal extracted from an operation of a user; (b) extracting feature information from the operation signal; (c) And generating personal identification information of the user by comparing the user identification information with operation feature information of a plurality of users including the user stored in advance.

(B-1) performing the preprocessing by receiving the operation signal; (b-2) repeating the pre-processing of the pre-processed operation signal And extracting the feature information from the feature information.

In the operation-based personal identification method according to the present invention, the step (b-2) distinguishes the operation signal from the section in which the operation of the user is actually performed and the section in which the operation of the user is not performed, And performing the steps of:

In the operation-based personal identification method according to the present invention, the step (b) includes the step (b-3) extracting the feature information from the operation signal in the form of a feature vector, (C-1) comparing the feature vector with a feature vector for the plurality of user's action feature information.

The method of claim 1, further comprising: (d) acquiring identification information of the user, wherein after step (b): (e) And storing it as operation feature information.

In the operation-based personal identification method according to the present invention, the step (e) may include storing the feature vector or information obtained by learning the feature vector as the action feature information for the user.

Further, in the operation-based personal identification method according to the present invention, the identification information of the user may further include additional information including at least one of age, sex, and taste of the user.

Further, in the operation-based personal identification method according to the present invention, the method may further include: (f) requesting to input a predetermined series of operations before the step (a).

In the operation-based personal identification method according to the present invention, the feature information includes feature information on a plurality of operations, and the step (c) includes: (c-2) (C-3) generating personal identification information of the user on the basis of the provisional individual identification information.

Further, in the operation-based personal identification method according to the present invention, each of the plurality of user's operation feature information may include a user identification feature and an operation identification feature.

The present invention also provides a computer-readable recording medium on which a program for realizing each step of the above-described motion-based personal identification method is recorded.

According to the present invention, it is possible to perform personal identification that is robust against environmental noise, without the need for the user to cooperate with the individual, using the user's personal characteristics.

Hereinafter, an operation-based personal identification system and method according to the present invention and a computer-readable recording medium on which a program for realizing the operation-based personal identification system and method are recorded will be described in detail with reference to the accompanying drawings.

1 is an exemplary block diagram of an operation-based personal identification system in accordance with the present invention.

Referring to FIG. 1, an operation-based personal identification system according to the present invention includes an individual identification device 100 and an operation signal acquisition device 200.

The operation signal acquisition device 200 extracts an operation signal from the operation of the user and transmits the extracted operation signal. The motion signal acquisition device 200 may include an inertial sensor 210, for example. The inertial sensor 210 includes, for example, an acceleration sensor or an angular velocity sensor, and converts the operation of the user into an electric signal, that is, an operation signal.

The user interacts with the personal identification device 100 using the action signal acquisition device 200. [ For example, the user can send an instruction or transmit information to the personal identification device 100 by performing a series of operations using the operation signal acquisition device 200. [

The transmission method between the personal identification device 100 and the operation signal obtaining device 200 may be, for example, a wired means such as USB, or a wireless means such as Zigbee, wireless LAN or Bluetooth.

The personal identification device 100 generates personal identification information by signal processing an operation signal transmitted from the operation signal acquisition device 200. The personal identification device 100 may be embedded in a device such as a personal computer, a home appliance such as a TV, a home or industrial robot, and the like.

Referring to FIG. 1, the personal identification apparatus 100 includes a feature information extracting unit 110, a personal identification information generating unit 130, and an operation feature information storing unit 150. The personal identification apparatus 100 may further include a user identification information obtaining unit 170 and a preprocessing unit 190.

The feature information extraction unit 110 extracts feature information from an operation signal transmitted from the operation signal acquisition device 200.

Meanwhile, the personal identification apparatus 100 may further include a preprocessing unit 190.

The preprocessing unit 190 receives the operation signal and performs preprocessing. For example, when the acceleration signal generated by using the acceleration sensor is received as the operation signal, the preprocessor 190 receives the acceleration signal and attenuates the fine noise of the signal over the entire frequency band through the Gaussian filter. It is also possible to perform a preprocess such as applying a high-pass filter to remove the gravitational acceleration component.

The feature information extracting unit 110 extracts feature information from an operation signal that has been pre-processed by the preprocessing unit 190.

Meanwhile, there may be a section in which the user performs the actual operation and an interval in which the operation is not performed. Therefore, the preprocessing unit 190 can increase the efficiency of the preprocessing by separating the operation signal into the section in which the user's operation is actually performed and the section in which the user's operation is not performed.

The feature information is a feature value inherent in the operation signal. The feature information extracted by the feature information extracting unit 110 may have, for example, a series of feature vectors. The extracted feature information may be transmitted to the personal identification information generating unit 130 and used to generate personal identification information of the user. Also, the extracted feature information may be transmitted to the operation feature storage unit 150 and stored as operation feature information for the user, or may be used to update operation feature information for the user through learning.

The motion feature information storage unit 150 may store motion feature information for a user in the form of a result of learning feature vectors using a recognizer such as a neural network or SVM.

The operation characteristic information storage unit 150 stores operation characteristic information of a plurality of users.

That is, the action feature information storage unit 150 previously stores the action feature information of a plurality of users, for example, in the form of a feature vector, and provides the generated feature information in order to generate user's personal identification information.

As described above, the motion feature information storage unit 150 may update motion feature information of a plurality of users by using the feature information extracted by the feature information extraction unit 110. [

The personal identification apparatus 100 may further include a user identification information obtaining unit 170 for updating or generating the operation feature information of the plurality of users.

When the feature information extracted by the feature information extraction unit 110 is transmitted to the operation feature storage unit 150 and is stored as operation feature information for the user, the identification information of the user is information about the user's operation Can be used to distinguish between the two.

The user identification information obtaining unit 170 obtains the identification information of the user.

The identification information of the user can be inputted directly by a user through an input means such as a keyboard and a mouse, or can be read out if necessary after being stored in advance.

The identification information of the user may include not only simple identification information such as a user ID but also additional information such as age, sex, taste, etc. of the user.

The personal identification information generation unit 130 generates the personal identification information of the user by comparing the feature information extracted by the feature information extraction unit 110 with the operation feature information of a plurality of users stored in the operation feature information storage unit 150 . The personal identification information is information that the user is identified by the personal identification device 100 through the operation signal and is used for user confirmation when command input or information provision is performed in the personal identification device 100. [

The personal identification information generating unit 130 compares the feature information extracted by the feature information extracting unit 110, e.g., a feature vector, with the motion feature information of a plurality of users stored in the operation feature storage unit 150, .

For example, when the motion feature information stored in the motion feature storage unit 150 takes the form of a classifier such as an SVM or a neural network, the personal identification information generation unit 130 generates a feature vector, To the corresponding recognizer, and by looking at the output value output, it is possible to identify the user and generate the personal identification information.

Or if the motion feature information stored in the motion feature storage unit 150 stores the feature vector as it is, the personal identification information generation unit 130 generates a feature of the motion feature information stored in the motion feature storage unit 150 in real time Vectors and the feature vectors extracted by the feature information extraction unit 110 are learned or compared to identify the user and generate the individual identification information.

Figures 2 to 4 are exemplary flowcharts of an action-based personal identification method in accordance with the present invention.

FIG. 2 is an exemplary flowchart of an operation-based personal identification method according to the present invention, in which the user's operation feature information is stored in advance and is performed in the personal identification apparatus 100 of FIG.

First, the personal identification apparatus 100 acquires user identification information (S110).

That is, in the personal identification apparatus 100, a user registration is performed to perform an operation-based personal identification method according to the present invention. The identification information of the user may further include additional information including at least one of the age, sex, and taste of the user.

The personal identification apparatus 100 then requests the user to input a series of operations (S130). In step S130, the personal identification device 100 may request the user to input a specific operation, but may request the user to arbitrarily input a desired operation.

In this case, the action characteristic information of the user in the future may include operation sequence information for identifying the user.

When the user inputs and transmits an operation signal through the operation signal acquisition device 200 (Fig. 1), the personal identification device 100 receives the operation signal (S150).

Then, the personal identification device 100 extracts feature information from the operation signal (S170).

On the other hand, in step S170, the operation signal is received, the preprocessing is performed, and the feature information is extracted from the preprocessed operation signal.

In step S170, the feature information may be generated by extracting the user identification feature and the operation identification feature from the operation signal, and then combining the two.

The characteristic of user identification is mainly information that can reflect operation characteristic which is unusual for individual. For example, in the case of an acceleration signal, it means various features such as the number of peaks in the operation section extracted from the acceleration signal itself, the difference value between the maximum peak and the minimum peak, the operation length, and the traveling direction of the peak.

The motion identification feature is used to identify an operation that is not a user, and is information that can distinguish an operation performed by an actual user.

For example, when a character is input as an operation, it is advantageous to identify the operation by utilizing the locus information extracted from the acceleration signal.

Since the types of features favorable to each purpose are different according to the user identification or the operation identification, the user identification feature and the operation identification feature are extracted from the operation signal. It is assumed that a recognizer for motion recognition is provided in advance in order to extract the motion recognition feature.

Then, the personal identification apparatus 100 stores the feature information extracted in step S170 as operation feature information for the user by matching the user identification information acquired in step S110 (S190).

That is, the personal identification apparatus 100 stores, as pairs of the operation feature information of the user including the user identification feature and the operation identification feature, as the operation feature information for the user based on the identification information of the user.

The process of storing the user's action feature information in advance may be performed a plurality of times. For example, the action feature information of the user may be stored for each action.

The feature information extracted in step S170 may be in the form of a feature vector. In this case, step S190 may store the feature vector or information obtained by learning the feature vector as operation feature information for the user.

The step of previously storing the behavior feature information of the user is performed in advance for a plurality of users.

FIG. 3 is an exemplary flowchart of a method for identifying a non-cooperative user in an operation-based personal identification method according to the present invention.

First, the personal identification apparatus 100 receives an operation signal extracted from the operation of the user (S210).

Then, the personal identification apparatus 100 extracts feature information from the operation signal (S230).

In step S210, the above-described preprocessing can be performed.

For example, when the acceleration signal is received as the operation signal, the acceleration signal is received and the fine noise of the signal is attenuated over the entire frequency band through the Gaussian filter. It is also possible to perform a preprocess such as applying a high-pass filter to remove the gravitational acceleration component.

Meanwhile, there may be a section in which the user performs the actual operation and an interval in which the operation is not performed. Therefore, it is possible to improve the efficiency of the preprocessing by distinguishing the operation signal from the section in which the user's operation is actually performed and the section in which the user's operation is not performed.

When the preprocessing is performed, the user identification feature and the operation identification feature are extracted, and the operation identification characteristic is used to confirm the user's operation.

The feature information can be extracted, for example, as a feature vector form.

Thereafter, the feature information extracted in step S210 is compared with the operation feature information of a plurality of users stored in advance to generate personal identification information of the user (S250).

As described above, each of the plurality of user's operation feature information includes a user identification feature and an operation identification feature. In addition, each of the plurality of user's action feature information may be a feature vector or a type of feature vector learned.

Accordingly, the personal identification information of the user can be generated by comparing the user identification feature and the motion identification feature in the feature information extracted in step S230 with the operation feature information of a plurality of users.

In this case, the feature information may be, for example, a feature vector form. In this case, the feature vector or the feature vector of the plurality of users' operation feature information is learned and compared with the feature vector of the feature information. .

The non-cooperative user's personal identification method according to FIG. 3 may be repeatedly used to generate more accurate personal identification information.

For example, when a user inputs a plurality of operations, the above-described feature information may include feature information on a plurality of operations.

In step S250, the individual personal identification information may be generated based on the provisional individual identification information after generating the provisional individual identification information by comparing the plurality of operations with the plurality of pieces of the user's operation characteristic information.

For example, if {{user 1, 'A'), (user 2, 'B'), (user 1) are generated by collecting the same operation four times or collecting four different operations four times, , 'D'), (user 3, 'R')}, if the user's personal identification information is finally generated, the current user identifies 'user 1'.

4 is an exemplary flow diagram of a method for identifying a collaborative user in an action-based person identification method in accordance with the present invention.

The embodiment of FIG. 4 is similar to the embodiment of FIG. 3 except that the operation of the user is designated in advance by the personal identification device 100, and the user inputs the corresponding operation in cooperation with the user. Therefore, in the embodiment with reference to FIG. 4, the step of requesting input of a series of operations, that is, step S310 is included, and there is no need to extract the feature for operation identification in step S350 of extracting the feature information, The difference is that the feature for motion identification is not used.

As described above, according to the present invention, in contrast to a conventional voice or image processing method in a non-cooperative situation, it is robust against environmental changes and the user can be identified even in a situation where the user is not cooperative.

In addition, the present invention not only provides less rejection than the conventional biometric method, but also provides strong security. For example, even if another person finds out an operation registered with a password and inputs the same operation, the user can be confirmed based on the difference in the individual operation characteristics.

The present invention also provides a computer-readable recording medium on which a program for realizing each step of the operation-based personal identification method according to the present invention described above is recorded.

The computer-readable recording medium refers to any kind of recording apparatus in which data, that is, data in the form of a code or a program, is stored so as to be readable by a computer system. Such a computer-readable recording medium is, for example, a memory such as a ROM and a RAM, a storage medium such as CD-ROM and DVD-ROM, a magnetic storage medium such as a magnetic tape and a floppy disk, The present invention is not limited thereto. Such a computer-readable recording medium may also be distributed over a networked computer system so that computer readable data can be stored and executed in a distributed manner.

However, detailed description of such a computer-readable recording medium is omitted because it is redundant with the operation-based personal identification method according to the present invention described with reference to FIGS. 2 to 4.

Although the present invention has been described in detail, it should be understood that the present invention is not limited thereto. Those skilled in the art will appreciate that various modifications may be made without departing from the essential characteristics of the present invention. Will be possible.

Therefore, the embodiments disclosed in the present specification are intended to illustrate rather than limit the present invention, and the scope and spirit of the present invention are not limited by these embodiments. The scope of the present invention should be construed according to the following claims, and all the techniques within the scope of the same should be interpreted as being included in the scope of the present invention.

1 is an exemplary block diagram of an action-based personal identification system in accordance with the present invention;

FIG. 2 is an exemplary flow chart of an operation-based personal identification method according to the present invention, the step of storing the user's action feature information in advance.

3 is an exemplary flow chart of a method for personal identification of a non-cooperative user in an action-based personal identification method according to the present invention.

4 is an exemplary flow chart of a method for personal identification of a collaborative user in an action-based person identification method in accordance with the present invention.

Description of the Related Art

100: Personal identification device 110: Feature information extraction unit

130: Personal identification information generation unit 150: Operation characteristic information storage unit

170: user identification information acquiring unit 190: preprocessing unit

200: Operation signal acquisition device 210: Inertial sensor

Claims (10)

An operation signal acquisition device for extracting and transmitting an operation signal from the user's operation, And an individual identification device for processing the operation signal transmitted from the operation signal acquisition device to generate individual identification information, Wherein the personal identification device comprises: A feature information extracting unit for extracting a feature for user identification and a feature for operation identification from the operation signal and extracting feature information by combining them, An operation feature information storage unit for storing operation feature information of a plurality of users including the user; A personal identification information generating unit for generating personal identification information of the user by comparing the feature information with the operation characteristic information of the plurality of users stored in advance, Based on the identification information. The method according to claim 1, The personal identification apparatus may further include a preprocessing unit for receiving the operation signal and performing preprocessing, and the feature information extracting unit may extract the feature information from the preprocessed operation signal Based personal identification system. 3. The method of claim 2, The preprocessor may perform the preprocessing by distinguishing the operation signal from a section in which the operation of the user is actually performed and a section in which the operation of the user is not performed Based personal identification system. The method according to claim 1, Wherein the personal identification apparatus further comprises a user identification information obtaining unit for obtaining the identification information of the user and the operation feature information storing unit stores the feature information as the operation feature information for the user based on the identification information of the user To do Based personal identification system. 5. The method of claim 4, Wherein the feature information is a feature vector type and the operation feature information storage unit stores the feature vector or information obtained by learning the feature vector as operation feature information for the user Based personal identification system. (a) receiving an operation signal extracted from an operation of a user; (b) extracting feature information from the operation signal by extracting a user identification feature and an operation identification feature, respectively, and combining them; And (c) generating personal identification information of the user by comparing the feature information with operation characteristic information of a plurality of users including the user stored in advance Based on the identification information. The method according to claim 6, (B-1) performing the pre-processing by receiving the operation signal; (b-2) extracting the feature information from the pre-processed operation signal; and (b- 3) extracting the feature information from the operation signal in a feature vector form Based personal identification method. 8. The method of claim 7, (C) comparing the feature vector with a feature vector of the plurality of user's action feature information; and (c) 2) generating temporary personal identification information for each of the plurality of operations by comparing with the action characteristic information of the plurality of users; and (c-3) generating personal identification information of the user based on the temporary personal identification information Including the step of creating Based personal identification method. 8. The method of claim 7, (d) acquiring identification information of the user, and (b) after the step (e), storing the feature information as the action feature information for the user Based personal identification method. 10. The method of claim 9, Wherein the step (e) includes storing the feature vector or information obtained by learning the feature vector as the action feature information for the user Based personal identification method.

Images