KR20210061211A - System for recognizing face and motion of clinical subject using video capture member - Google Patents

Abstract

The present invention provides a facial and motion recognition system of a clinical subject using an imaging unit. The facial and motion recognition system comprises: an imaging unit that images a clinical subject; a user terminal unit that extracts feature point information about the facial information and motion information of the clinical subject imaged by the imaging unit; and an authentication server unit that transmits an authentication result for user authentication and behavior recognition authentication to the user terminal unit on the basis of the feature point information received from the user terminal unit. The authentication server unit uses deep learning techniques to recognize the face and motion of the clinical subject, and the user terminal unit outputs any one of authentication information and non-authentication information, which are results of the user authentication and the behavior recognition authentication.

Description

System for recognizing face and motion of clinical subject using video capture member}

The present invention relates to a facial and motion recognition system of a clinical subject using an imaging unit, and more particularly, to verify whether or not a person is authenticated based on feature point information extracted from the facial information and motion information of the clinical subject photographed by the imaging unit. It relates to a facial and motion recognition system of a clinical subject using an imaging unit.

In general, pharmaceutical companies are conducting many clinical trials to develop new drugs, and for this purpose, they recruit clinical subjects to participate in clinical trials.

However, the current clinical trials are expensive, and it is difficult to recruit clinical subjects, and there is a problem in that the subject must visit the hospital in a cumbersome manner.

In order to solve the above problems, remote clinical services that conduct clinical trials remotely are being introduced. However, these remote clinical services do not ensure that subjects participating in clinical trials remotely are faithfully participating in the trial according to the requirements of the clinical trial. There was a problem that it was difficult to check.

In addition, there is a problem that it is difficult to confirm whether a subject who intends to participate in a clinical trial is actually participating in a clinical trial remotely (login, measuring and transmitting biometric information, taking medicine, measuring and transmitting behavior information, inputting a questionnaire, etc.).

For clinical trial tests conducted at locations other than clinical hospitals, clinical trials are conducted with clinically related organizations (Ministry of Food and Drug Safety, CRO, sponsors) reliability issues (subject identification issues, reliability issues of clinical trial data, clinical data quality issues) and clinical data quality issues. You may be in doubt about the quality of your data.

In order to check the transparent information of the subject, one method may be to take a video of all actions of the subject and upload it in real time.

However, the above method does not protect personal information and may invade the privacy of the clinical subject, and storage and network costs according to the capacity of the video cannot be ignored.

Therefore, it is time to need a technology that can authenticate users and identify targets for actions while preventing invasion of privacy and protecting personal information while reducing storage and network costs.

(Patent Document 1) Registered Patent Publication No. 10-0757167 (2007.09.03.)

(Patent Document 2) Unexamined Patent Publication No. 10-2015-0011046 (2015.01.30.)

An object of the present invention for solving the above problems is an image capture unit that provides authentication information or non-authentication information after extracting feature point information through a user terminal based on facial information and motion information of a clinical subject photographed by an image capturing unit. It is to provide the facial and motion recognition system of the used clinical subject.

In addition, an object of the present invention for solving the above problems is to provide a facial and motion recognition system of a clinical subject using an image capture unit that provides motion feature point information to a display unit based on motion information.

In addition, an object of the present invention for solving the above problems is to provide a facial and motion recognition system of a clinical subject using an image capture unit that processes information by extracting feature point information based on facial information and motion information.

The technical problem to be achieved by the present invention is not limited to the technical problems mentioned above, and other technical problems that are not mentioned can be clearly understood by those of ordinary skill in the technical field to which the present invention belongs from the following description. There will be.

The configuration of the present invention for achieving the above object is an imaging unit for photographing a clinical subject; A user terminal unit for extracting feature point information for the facial information and motion information of the clinical subject photographed by the image capturing unit; And an authentication server configured to transmit authentication results for user authentication and behavior recognition authentication to the user terminal based on the feature point information transmitted from the user terminal unit, wherein the authentication server unit uses a deep learning technique. A face and motion recognition system of a clinical subject using an imaging unit that recognizes the face and motion of the clinical subject, and the user terminal unit outputs any one of authentication information and non-authentication information that is a result of the user authentication and the behavior recognition authentication. Provides.

In an embodiment of the present invention, the user terminal unit comprises: an information storage unit for storing the facial information and the motion information; And And an information processing unit that extracts feature point information for the facial information and the motion information transmitted from the information storage unit, wherein the feature point information may include facial feature point information and motion feature point information of the clinical subject.

In an embodiment of the present invention, the information storage unit comprises: a first storage unit for storing facial information of the clinical subject photographed by the image capturing unit; And an information processing unit for extracting the facial feature point information from the facial information of the clinical subject transmitted from the first storage unit through a facial algorithm, wherein the information processing unit stores preset facial information and facial information of the clinical subject. By comparison, the facial feature point information may be extracted.

In an embodiment of the present invention, the information processing unit includes: an information storage unit including a second storage unit for storing motion information of the clinical subject photographed by the image capturing unit; And an information processing unit that extracts motion feature point information of the clinical subject from motion information of the clinical subject transmitted from the second storage unit through a motion algorithm, wherein the information processing unit includes preset motion information and the clinical subject's motion information. The motion feature point information may be extracted by comparing motion information.

In an embodiment of the present invention, the motion feature point information may include at least one of confirmation information on whether a drug is present in the cup of the clinical subject and motion information of the clinical subject drinking the drug.

In an embodiment of the present invention, the information storage unit further includes an authentication information storage unit for storing authentication results for user authentication and behavior recognition authentication transmitted from the authentication server unit, wherein the user terminal unit stores the authentication information. It may further include a display unit for outputting the authentication result for user authentication and behavior recognition authentication transmitted from the unit; may further include.

In an embodiment of the present invention, the display unit may output at least one of skeletal information, muscle information, log-in information, biometric information, and information on measurement and transmission of behavioral information, drug use information, and medical examination information of the clinical subject. I can.

In an embodiment of the present invention, the authentication server unit determines whether the feature point information transmitted from the user terminal unit matches the feature point information preset by the deep learning technique, and the feature point information and the preset feature point information are matched. If matched, the authentication server unit transmits the authentication information to the user terminal unit, and when the feature point information and the preset feature point information match, the authentication server unit may transmit the non-authentication information to the user terminal unit.

In an embodiment of the present invention, the image capture unit may be any one of a video camera, an infrared camera, and a 3D measuring device.

In an embodiment of the present invention, the infrared camera recognizes hot spots generated by blood flow through arteries and blood to generate the facial information and the motion information, and the user terminal uses the hot spot to generate the Feature point information can be extracted.

In an embodiment of the present invention, the 3D measuring device measures at least one of the angle and distance between the eyes, mouth, nose, and jaw, and the degree of bone protrusion, which are the measurement information of the clinical subject, and calculates the facial information and the motion information. And, the user terminal may extract feature point information of the clinical subject by using the measurement information.

The effect of the present invention according to the configuration as described above is that the clinical subject is provided with authentication information or non-authentication information after extracting feature point information through the user terminal unit based on the facial information and motion information of the clinical subject photographed by the imaging unit. You can check whether you are actually participating in the clinical trial.

In addition, the effect of the present invention according to the configuration as described above is that motion feature point information is provided to the display unit based on motion information, so that it is possible to confirm whether the clinical subject is faithfully participating in the clinical trial condition.

In addition, the effect of the present invention according to the above configuration can reduce the burden on storage space and reduce network traffic and network cost as the information is processed by extracting feature point information based on facial information and motion information.

The effects of the present invention are not limited to the above effects, and should be understood to include all effects that can be inferred from the configuration of the invention described in the detailed description or claims of the present invention.

1 is a block diagram showing a face and motion recognition system of a clinical subject using an image capture unit according to an embodiment of the present invention.
2 is a schematic diagram showing a process for facial feature point information and user authentication in a facial and motion recognition system of a clinical subject using an image capture unit according to an embodiment of the present invention.
3 is a schematic diagram showing a process for motion feature point information and behavior recognition authentication in a facial and motion recognition system of a clinical subject using an image capture unit according to an embodiment of the present invention.

Hereinafter, the present invention will be described with reference to the accompanying drawings. However, the present invention may be implemented in various different forms, and therefore is not limited to the embodiments described herein. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present invention, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a part is said to be "connected (connected, contacted, bonded)" with another part, it is not only "directly connected", but also "indirectly connected" with another member in the middle. "Including the case. In addition, when a part "includes" a certain component, this means that other components may be further provided, not excluding other components, unless specifically stated to the contrary.

The terms used in the present specification are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In the present specification, terms such as "comprise" or "have" are intended to designate the presence of features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, but one or more other features. It is to be understood that the presence or addition of elements or numbers, steps, actions, components, parts, or combinations thereof does not preclude in advance.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1. Description of the facial and motion recognition system of the clinical subject using the imaging unit

1 is a block diagram showing a face and motion recognition system of a clinical subject using an image capture unit according to an embodiment of the present invention.

A facial and motion recognition system of a clinical subject using an image taking unit according to an embodiment of the present invention includes an image taking unit 100, a user terminal unit 200, and an authentication server unit 300.

The image capturing unit 100 is a device for photographing or moving pictures of a clinical subject, and may be, for example, a video camera. Basically, the image capturing unit 100 captures a picture or video of a clinical subject, and then transmits the photographed picture and video to the user terminal unit 200.

That is, the photograph or video of the clinical subject acquired by the imaging unit 100 is facial information and motion information of the clinical subject, and is used as basic data for extracting feature point information.

In the present invention, the image taking unit 100 is described as being a video camera, but is not limited thereto. For example, the image taking unit 100 may be any one of a video camera, an infrared camera, and a 3D measuring device.

Here, the infrared camera recognizes hot spots generated by blood flow through arteries and blood to generate facial information and motion information, and accordingly, the user terminal unit 200 extracts feature point information of a clinical subject using hot spots.

On the other hand, the 3D measuring device generates facial information and motion information by measuring at least one of the angle and distance between the eyes, mouth, nose, and jaw, which are measurement information of the clinical subject, and the degree of bone protrusion, and accordingly, the user terminal unit 200 ) Extracts the feature point information of the clinical subject using the measurement information.

2 is a schematic diagram showing a process for facial feature point information and user authentication in a facial and motion recognition system of a clinical subject using an image capture unit according to an embodiment of the present invention.

The user terminal unit 200 extracts feature point information for facial information and motion information of a clinical subject photographed by the image capturing unit 100.

In addition, the user terminal unit 200 outputs an authentication result for user authentication and behavior recognition authentication based on the authentication information transmitted from the authentication server unit 300.

Exemplarily, when the user terminal unit 200 receives authentication information from the authentication server unit 300, the feature point information of the clinical subject photographed by the image capturing unit 100 is authenticated in accordance with the preset feature point information of the clinical subject. Outputs information (user authentication, behavior recognition authentication).

On the other hand, when the user terminal unit 200 receives non-authentication information from the authentication server unit 300, the feature point information of the clinical subject photographed by the image capturing unit 100 is non-authentication information inconsistent with the preset feature point information of the clinical subject. (User non-authentication, behavior recognition non-authentication) is output.

The user terminal unit 200 includes an information storage unit 210, an information processing unit 220, a control unit 230 and a display unit 240.

The information storage unit 210 stores facial information and motion information transmitted from the image capturing unit 100. The information storage unit 210 includes a first storage unit 211, a second storage unit 212, and an authentication information storage unit 213.

The first storage unit 211 stores facial information of a clinical subject photographed by the image capturing unit 100.

The second storage unit 212 stores motion information of a clinical subject photographed by the image capturing unit 100.

The authentication information storage unit 213 stores authentication results for user authentication and behavior recognition authentication transmitted from the authentication server unit 300. Specifically, the authentication information storage unit 213 stores either authentication information or non-authentication information according to the authentication result processed by the authentication server unit 300.

The information processing unit 220 extracts feature point information for facial information and motion information transmitted from the information storage unit 210. Here, the feature point information includes facial feature point information and motion feature point information of a clinical subject.

The facial feature point information is significantly different compared to the preset facial average value obtained by finding the average value for each part of the face from the collected information on the faces (eyes, nose, mouth, chin, forehead, facial bone protrusion, etc.) of a large number of people. Is the extracted value.

In addition, the motion feature point information is a value that is significantly different compared to the preset facial average value obtained by finding the average value for each activity from the collected information about various and many people's actions (habitual behavior, drug-taking behavior, etc.).

In addition, the information processing unit 220 divides and processes facial feature point information and motion feature point information transmitted from the information storage unit 210.

Specifically, the information processing unit 220 extracts facial feature point information from the facial information of the clinical subject transmitted from the first storage unit 211 through a facial algorithm. At this time, the information processing unit 220 extracts facial feature point information by comparing the preset facial information with the facial information of the clinical subject.

Next, the information processing unit 220 extracts motion feature point information of the clinical subject from motion information of the clinical subject transmitted from the second storage unit 212 through a motion algorithm. At this time, the information processing unit 220 extracts motion feature point information by comparing preset motion information with motion information of a clinical subject.

Here, the motion feature point information includes at least one of confirmation information on whether the medicine is present in the cup of the clinical subject and motion information on which the clinical subject drinks the medicine.

The control unit 230 controls the operation of the image capturing unit 100, the information storage unit 210, the information processing unit 230, the display unit 240, and the authentication server unit 300.

Specifically, the control unit 230 controls the image capturing unit 100 so that the facial information and motion information acquired by the image capturing unit 100 are transmitted to the first storage unit 211 and the second storage unit 212, respectively. .

In addition, the control unit 230 controls the information processing unit 230 to extract facial feature point information and motion feature point information from the facial information transmitted from the first storage unit 211 and the motion information transmitted from the second storage unit 212 do.

In addition, the control unit 230 controls the authentication server unit 300 so that authentication information or non-authentication information transmitted from the authentication server unit 300 is transmitted to the authentication information storage unit 213.

In addition, the control unit 230 controls the display unit 240 to output authentication information or non-authentication information transmitted from the authentication information storage unit 213.

The display unit 240 outputs authentication results for user authentication and behavior recognition authentication transmitted from the authentication information storage unit 213. The authentication result may be any one of authentication information confirming that the clinical subject is the subject and non-authentication information confirming that the clinical subject is not the subject.

In addition, the display unit 240 may output at least one of skeletal information, muscle information, log-in information, biometric information, and information on measurement and transmission of behavioral information of a clinical subject, drug use information, and medical examination information.

3 is a schematic diagram showing a process for motion feature point information and behavior recognition authentication in a facial and motion recognition system of a clinical subject using an image capture unit according to an embodiment of the present invention.

The authentication server unit 300 transmits an authentication result for user authentication and behavior recognition authentication to the user terminal unit 200 based on the feature point information transmitted from the user terminal unit.

At this time, the authentication server unit 300 recognizes the face and motion of the clinical subject using a deep learning technique.

Specifically, the authentication server unit 300 determines whether the feature point information transmitted from the user terminal unit matches the feature point information preset by the deep learning technique.

If the feature point information and the preset feature point information match, the authentication server unit 300 transmits the authentication information to the user terminal unit 200.

On the other hand, if the feature point information and the preset feature point information do not match, the authentication server unit 300 transmits the non-authentication information to the user terminal unit 200.

2. Description of the operation of the clinical subject's facial and motion recognition system using the imaging unit

Hereinafter, the operation of the face and motion recognition system of a clinical subject using the image capture unit of the present invention will be described with reference to FIGS. 1 to 3.

First, after obtaining the facial information and image information of the clinical subject from the imaging unit 100, the facial information and the image information are transmitted to the user terminal unit 200.

Next, the facial information is transmitted to the information processing unit 230 through the first storage unit 211, and the motion information is transmitted to the information processing unit 230 through the second storage unit 212.

Next, the information processing unit 230 extracts facial feature point information (face feature point information) and motion feature point information (behavior, behavior, muscle, skeleton, etc.) from the facial information and motion information, and transmits it to the authentication server 300.

Next, the authentication server unit 300 checks whether the preset facial feature point information and the extracted facial feature point information, and the preset motion feature point information and the extracted motion feature point information match using a deep learning technique, and then, according to the authentication result, the authentication information and Any one of the non-authentication information is transmitted to the user terminal unit 200.

Next, any one of authentication information and non-authentication information is stored in the authentication information storage unit 213 and then output through the display unit 240.

According to the present invention as described above, the clinical subject actually participates in the clinical trial by providing authentication information or non-authentication information after extracting the feature point information through the user terminal unit based on the facial information and motion information of the clinical subject photographed by the imaging unit. You can check whether you are the right person.

In addition, the present invention provides motion feature point information to the display unit based on the motion information, so that it is possible to confirm whether the clinical subject is faithfully participating in the condition of the clinical trial.

In addition, according to the present invention, as the feature point information is extracted and processed based on facial information and motion information, a burden on a storage space may be reduced, and network traffic and network cost may be reduced.

The above description of the present invention is for illustrative purposes only, and those of ordinary skill in the art to which the present invention pertains will be able to understand that other specific forms can be easily modified without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative and non-limiting in all respects. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as being distributed may also be implemented in a combined form.

The scope of the present invention is indicated by the claims to be described later, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be construed as being included in the scope of the present invention.

100: image recording unit
200: user terminal
210: information storage unit
211: first storage unit
212: second storage unit
213: authentication information storage unit
220: information processing unit
230: control unit
240: display unit
300: certificate server

Claims (11)

An imaging unit for photographing a clinical subject;
A user terminal unit for extracting feature point information for the facial information and motion information of the clinical subject photographed by the image capturing unit; And
Including; an authentication server for transmitting an authentication result for user authentication and behavior recognition authentication to the user terminal unit based on the feature point information transmitted from the user terminal unit,
The authentication server recognizes the face and motion of the clinical subject using a deep learning technique,
The user terminal unit outputs any one of authentication information and non-authentication information that is a result of the user authentication and the behavior recognition authentication,
Facial and motion recognition system of clinical subjects using an imaging unit.
The method of claim 1,
The user terminal unit,
An information storage unit for storing the facial information and the motion information; And
Including; an information processing unit for extracting feature point information for the facial information and the motion information transmitted from the information storage unit,
The feature point information includes facial feature point information and motion feature point information of the clinical subject,
Facial and motion recognition system of clinical subjects using an imaging unit.
The method of claim 2,
The information storage unit,
A first storage unit for storing facial information of the clinical subject photographed by the imaging unit; And
Including; an information processing unit for extracting the facial feature point information from the facial information of the clinical subject transmitted from the first storage unit through a facial algorithm,
The information processing unit extracts the facial feature point information by comparing preset facial information and facial information of the clinical subject,
Facial and motion recognition system of clinical subjects using an imaging unit.
The method of claim 2,
The information processing unit,
An information storage unit including a second storage unit for storing motion information of the clinical subject photographed by the image capturing unit; And
Including; an information processing unit for extracting the motion feature point information of the clinical subject from the motion information of the clinical subject transmitted from the second storage unit through a motion algorithm,
The information processing unit extracts the motion feature point information by comparing preset motion information with motion information of the clinical subject,
Facial and motion recognition system of clinical subjects using an imaging unit.
The method of claim 3,
The motion feature point information includes at least one of confirmation information on whether a drug is present in the cup of the clinical subject and motion information of the clinical subject drinking the drug,
Facial and motion recognition system of clinical subjects using an imaging unit.
The method of claim 1,
The information storage unit further includes an authentication information storage unit for storing an authentication result for user authentication and behavior recognition authentication transmitted from the authentication server unit,
The user terminal unit further comprises a display unit for outputting the authentication result for user authentication and behavior recognition authentication transmitted from the authentication information storage unit;
Facial and motion recognition system of clinical subjects using an imaging unit.
The method of claim 6,
The display unit outputs at least one of skeletal information, muscle information, log-in information, biometric information, and information on measurement and transmission of behavioral information, drug use information, and medical examination information of the clinical subject,
Facial and motion recognition system of clinical subjects using an imaging unit.
The method of claim 1,
The authentication server unit determines whether the feature point information transmitted from the user terminal unit matches the feature point information preset by the deep learning technique,
If the feature point information and the preset feature point information match,
The authentication server unit transmits the authentication information to the user terminal unit,
If the feature point information and the preset feature point information match,
The authentication server unit transmits the non-authentication information to the user terminal unit,
Facial and motion recognition system of clinical subjects using an imaging unit.
The method of claim 1,
The image capture unit is characterized in that any one of a video camera, an infrared camera and a three-dimensional measuring device,
Facial and motion recognition system of clinical subjects using an imaging unit.
The method of claim 9,
The infrared camera recognizes hot spots generated by blood flow through arteries and blood to generate the facial information and the motion information,
The user terminal unit extracts feature point information of the clinical subject using the hot point,
Facial and motion recognition system of clinical subjects using an imaging unit.
The method of claim 9,
The 3D measuring device generates the facial information and the motion information by measuring at least one of the angle and distance between the eyes, the mouth, the nose, and the jaw, and the degree of bone protrusion, which are measurement information of the clinical subject,
The user terminal unit extracts feature point information of the clinical subject using the measurement information,
Facial and motion recognition system of clinical subjects using an imaging unit.

Images