TW201918216A - Non-contact living body identification method - Google Patents

Abstract

A non-contact living body identification method comprises steps of selecting a first skin region and a different second skin region from an image of a subject; extracting a first physiological signal from the first skin region, and a second physiological signal from the second skin region by using a microprocessor; performing a cross correlation calculation on the first physiological signal and the second physiological signal to produce a resultant waveform; using the resultant waveform to quantify the degree of similarity of the signal periods between the first physiological signal and the second physiological signal; and determined whether the subject is a living body based on the degree of similarity after quantification. The above method can be realized on the smart phone, so that the medical staff can be simple, fast and correct to confirm whether a patient remains alive by using the smart phone, and as soon as possible to rescue the patient.

Description

Non-contact living body identification method

The present invention relates to a method of identifying a living state of a living body, and more particularly to a method of identifying a living body using a non-contact vital sign detection using a portable device.

Emergency treatment of patients with "Out Of Hospital Cardiac Arrest (OHCA)" is one of the major problems facing public health security today. For patients with OHCA, if they can be rescued as soon as possible and their vital signs can be monitored at any time, the survival rate of patients with OHCA can be effectively improved.

The common method of measuring vital signs is to measure the human body's electrocardiography (ECG) and photoplethysmography (PPG) by professional medical instruments. However, conventional medical instruments are not easily available outdoors except for hospitals. Not only that, even if they have these medical instruments outdoors, it is not easy for ordinary people without professional training to operate. Moreover, conventional medical instruments need to be connected to the human body by wires during use, and in addition to being subject to environmental and space constraints during measurement, it takes a lot of time for the device to be worn.

In real life, without the aid of the above-mentioned medical instruments, the carotid artery of the patient is generally touched by hand to distinguish whether the pulsation has stopped. However, the general public uses this resolution method to identify less than 50%; even professional medical personnel cannot answer the judgment quickly and confidently.

Therefore, it is necessary to develop a non-contact detection method that can be used by the public. Conventional non-contact living body identification techniques include the Republic of China Patent Publication No. 200741559, No. 201033907, and No. 201445456. Patent Publication No. 200741559 adopts a background comparison method, and uses an empty background as a reference image in the scene to compare it with the film to be tested to check whether an unnecessary background appears in the film to be tested, thereby identifying the film to be tested. The object in the object is a living body or a non-living body. Patent Publication No. 201033907 is a biometric identification method that needs to detect whether an object to be tested has a specific action, such as eye movement and mouth movement, to identify that the object in the film to be tested is a living body or a non-living body. Patent Publication No. 201445456 is a living body face recognition and customs clearance mechanism. After the face image is normalized and calculated, it is determined whether the object to be tested is a living body by a predetermined threshold.

However, Patent Publication No. 200741559 requires pre-recording of an empty background to perform living object recognition. Patent Publication No. 201033907 needs to conform to a specified action to be recognized, and has many limitations in practical applications, resulting in inconvenience in use. Patent Publication No. 201445456 is similar to the methods of other related research documents, but the computational complexity of this method is relatively high and it is difficult to practically apply it in general daily life.

In view of this, the inventor of the present invention hopes to develop a convenient and effective non-contact living body identification method, and realizes the method in a smart phone, which not only provides a universal vital sign detector that can be carried and used in the general public. It can also quickly and correctly determine the survival status of the critically ill and increase the accuracy rate of the high-level ambulance dispatch.

An object of the present invention is to provide a non-contact living body identification method which can be realized by a smart phone, can quickly and correctly determine whether a subject is a living body or a non-living body, and is suitable for remote monitoring.

In order to achieve the above object, the present invention provides a non-contact living body identification method, comprising: providing a photographing device for capturing an image of a subject; selecting a different first skin region and a second skin from the image a first physiological signal is captured from the first skin region by a microprocessor, and a second physiological signal is captured from the second skin region, wherein the first physiological signal and the second physiological signal are light volumes The change tracing signal includes a skin color information and a pulse signal; the microprocessor performs a cross-correlation calculation on the first physiological signal and the second physiological signal to generate a result waveform: using the result waveform to the first physiological signal and The degree of similarity of the periodic changes of the second physiological signals is quantified; and whether the subject is a living body is determined according to the degree of similarity after the quantification.

In an embodiment, the method further includes: determining, according to the skin color information contained in the first physiological signal and the second physiological signal, the skin color of the subject; and determining whether to provide an additional color according to the judgment result of the skin color depth The light source illuminates the subject.

In an embodiment, the method further includes: passing the first physiological signal and the second physiological signal through a band pass filter to isolate signals outside the pulse frequency range before performing the cross correlation calculation; and passing the band pass filter Then, a green-red difference algorithm is performed on the first physiological signal and the second physiological signal to improve the saliency of the signal in the pulse frequency range; and the green-red difference algorithm is processed by a Kalman filter. The first physiological signal and the second physiological signal are smoothed.

In one embodiment, the resulting waveform intersects a zero axis at a complex zero crossing, and each of the two adjacent zero crossings has a time interval, wherein the quantizing step includes: providing a live signal threshold; a standard deviation of the time interval; and comparing the standard deviation to the living signal threshold to obtain a recognition result.

In one embodiment, the method is installed in a portable device in a software form, and the step further comprises: measuring a hand shaking signal of one of the holders of the portable device; setting a degree about the degree of hand shaking Threshold value; compares the size of the hand shaking signal and the threshold value; if the hand shaking signal exceeds the critical value, a warning window is popped up and the living body identification is required again.

The present invention quantifies the degree of similarity of the cyclic changes of the same subject by measuring the two physiological signals of the same subject but taking two different skin regions as the basis for identifying the living body. The method of the present invention can be implemented on a smart phone, so that the medical staff can easily, quickly and correctly identify whether a subject is in a viable state, and can perform the rescue as soon as possible.

The above and other technical contents, features and advantages of the present invention will be apparent from the following detailed description of the preferred embodiments. The directional terms mentioned in the following embodiments, such as upper, lower, left, right, front or rear, etc., are only used to refer to the directions of the accompanying drawings. Therefore, the directional terms are used for illustration only and are not intended to limit the invention.

FIG. 1 is a schematic flow chart of a non-contact living body identification method according to an embodiment of the present invention. The non-contact living body identification method is implemented in a smart portable device, and the steps include: capturing an image (step S10), selecting a signal capturing area (step S20), signal processing (step S30), and a living body signal identification algorithm. (Step S40), hand shake degree determination (step S50), and output of the recognition result (step S60) and the like, which will be described in detail below.

The step S10 of capturing an image is to capture an image by using a commercially available photographic device or a built-in photographic device of the mobile phone to obtain the light and dark changes caused by absorption and reflection of the human skin as the heart beats under the illumination of the ambient light source. Signal.

The step S20 of selecting the signal capturing area is to select a face position from the image by the photographing device (S21); and selecting a different first skin area and a second skin area as the signal capturing area from the face position (S22) ). In a preferred embodiment, the first skin area and the second skin area can be selected from the sides of the cheek or the nose, etc., because these areas are less affected by hair banging, eyebrows shaking, blinking, mouth talking, etc. The noise generated by the action interferes with the signal, so that a better quality signal can be obtained.

The signal processing step S30 includes a signal processing flow such as signal capture (step S31) and noise removal (steps S32 to S34). First, a first physiological signal S1 is extracted from the first skin area by a microprocessor, and a second physiological signal S2 is extracted from the second skin area (step S31). The first physiological signal S1 and the second physiological signal S2 may be two different photoplethysmography (PPG) or two different remote photoplethysmography (rPPG) signals. Taking the rPPG signal as an example, it is mainly to capture the physiological signal calculated by the change of the light source absorbed or reflected by the skin or other surrounding tissues on the human body through the camera. Therefore, the rPPG signal contains skin color information and pulse signal. For convenience of explanation, in the following, the first physiological signal S1 is also referred to as "rPPG signal S1", and the second physiological signal S2 is also referred to as "rPPG signal S2".

Since the human body's rPPG signals S1 and S2 are very small and susceptible to noise pollution, it is necessary to isolate the noise to obtain an effective pulse signal. These noises may include skin color effects, light source changes, or moving noise generated by body shakes, and the noises to be overcome in this embodiment are mainly skin color effects and moving noise.

In terms of skin color effects, if the subject's skin color is darker, it will directly affect the strength and stability of rPPG signals S1 and S2. In order to improve and stabilize the quality of the rPPG signals S1 and S2, in this embodiment, after the rPPG signals S1 and S2 are extracted (step S31), a skin color classification process is provided to determine whether an additional light source needs to be provided to illuminate the subject (steps). S32) to reduce the effect of skin tone on subsequent steps.

The detailed steps of the skin color classification process are shown in Figure 2. First, the two rPPG signals S1 and S2 (step S321) are respectively input into the microprocessor that executes the skin color classification process. Then, the skin color information in the two rPPG signals S1 and S2 is classified according to the color depth (step S322). In an embodiment, step S322 may employ "A. Treesirichod et al., "Digital Photographic RGB Scores used for the Evaluation of Skin Color," Indian Journal of Clinical and Experimental Dermatology, vol. 1, pp. 17-20, The skin color classification method proposed in 2015. Next, the skin color classification result Sc is compared with a set color threshold Th1 (step S323). When the skin color classification result Sc is smaller than the color threshold Th1, it means that the subject is judged to be a dark skin color, and an additional light source is provided at this time (step S324). The subject is illuminated to enhance the intensity of the skin absorption/reflection signal. Further, stable rPPG signals S1 and S2 are obtained. In an embodiment, the additional light source can be provided by controlling the flash in the smart phone.

In the mobile noise, as in step S33 of FIG. 1, in this embodiment, the two rPPG signals S1 and S2 are respectively passed through a band pass filter to isolate signals outside the pulse frequency range. Then, a green-red difference algorithm is respectively performed on the two rPPG signals S1' and S2' outputted by the band-pass filter to improve the saliency of the pulse signal in the rPPG signals S1' and S2'. For a detailed description of the green-red difference algorithm, see "L. Feng et al, "Motion-resistant remote imaging photoplethysmography based on the optical properties of skin," IEEE trans. on Circuits and System for Video Technology, vol. 25, No. 5, pp. 879-891, 2015.".

Finally, the rPPG signals S1' and S2' output by the green-red difference algorithm are smoothed using a Kalman filter (step S34). Thus, not only the stability of the rPPG signals S1' and S2' can be improved, but also the analysis of the subsequent live signal recognition algorithm (step S40).

Please refer to FIG. 3, which is a detailed flowchart of the living signal recognition algorithm (step S40). The smoothed rPPG signals S1" and S2" are input to the microprocessor that executes the live signal identification algorithm (step S41), and a cross-correlation is performed on the rPPG signals S1" and S2" to generate a cross-correlation. The resulting waveform (step S42). The two result waveforms W1 and W2 that may be obtained after the cross-correlation calculation are shown in FIGS. 3A and 3B, respectively. Then, a zero-crossing detection is performed on the resultant waveform W1 (or W2) to obtain a complex zero-crossing point P (step S43), and then the time interval T1 between each two adjacent zero-crossing points P is calculated (or T2), and calculate the standard deviation of these time intervals T1 (or T2) (Zero-crossing standard deviation), then this standard deviation It is compared with a live signal threshold Th2 to obtain a discrimination result R1 or R2 (step S44).

The above-described cross-correlation calculation step S42, the zero-crossing rate detection step S43, and the comparison step S44 are the core steps of the live signal recognition algorithm (step S40). Through these core steps S42 to S44, the resulting waveform W1 (or W2) can be used to quantify the similarity or degree of difference between the cyclic changes of the rPPG signals S1" and S2". Further, it is judged whether or not the subject is a living body based on the degree of similarity or degree of difference after the quantization, and the identification result R1 or R2 is output (step S60). These core steps S42 to S44 are respectively detailed as follows.

Since the periodic changes of the whole body pulse signal of the same living subject are consistent in a short time, the present invention takes the rPPG signals S1" and S2" in two different skin regions of the same living subject. A cross correlation calculation is performed (step S42), and a correlation between the two rPPG signals S1" and S2" is obtained.

Referring to FIG. 3A and FIG. 3B, if the two rPPG signals S1" and S2" are from the same living subject, the waveforms W1 of the two rPPG signals S1" and S2" after cross-correlation calculation (step S42) are presented as follows. The specific period of variation of Figure 3A. On the other hand, if it is from a non-living subject, such as a dummy or a face image, the rPG signals S1" and S2" of different skin regions are subjected to cross-correlation calculation, and the resulting waveform W2 exhibits the noise characteristics as shown in FIG. 3B. . It is apparent from Figs. 3A and 3B that there is a significant difference in physiological signals obtained from living and non-living bodies. It can be clearly seen from FIG. 3A and FIG. 3B that the result waveform W1 of FIG. 3A has a relatively regular change, and the time interval T1 between two adjacent zero-crossing points P is relatively close; however, FIG. 3B is compared with the signal of FIG. 3A. In other words, the change of the waveform W2 appears to be rather messy, and the time interval T2 between the two adjacent zero-crossing points P is different.

After the cross-correlation calculation (step S42), the correlation between the two rPPG signals S1" and S2" can be effectively known, and then the result of the zero-crossing rate detection (step S43) is used to bring the result waveform W1 (or W2). The time interval T1 (or T2) between every two adjacent zero-crossing points P is quantified to determine whether the resulting waveform W1 (or W2) has a periodic pulsation existing in the living body, thereby realizing a living/non- The judgment of the living signal. The zero-crossing rate detection is a feature that the calculation result waveform W1 (or W2) changes on the zero-axis Z, and the main steps include: the search result waveform W1 (or W2) is rotated from positive to negative and negative to positive on the zero-axis Z. Position; then, these positions are marked to obtain a basic zero-crossing point P, and the time interval T1 (or T2) between every two adjacent zero-crossing points P is calculated.

In step S44, measuring a physiological signal of a normal person, and using the cross-correlation calculation and the zero-crossing rate detection to extract the rPPG signal characteristics of the living tissue of the normal person to obtain training data, and then pass the training data. The waveform characteristics are summarized as a live signal threshold Th2. In this way, the identification of the living skin signal can be accurately performed. Then, the microprocessor performs cross-correlation calculation and zero-crossing rate detection, and calculates the standard deviation of the time interval T1 (or T2) between every two adjacent zero-crossing points P in the result waveform W1 (or W2). , the standard deviation will be calculated Compare with the set living signal threshold Th2 to obtain a recognition result R1 or R2. The identification result R2 in Figure 3 represents the standard deviation When the threshold value Th2 of the living body signal is less than the set value, the degree of similarity between the cyclic changes of the rPPG signals S1 and S2 is high, so that the two rPPG signals S1 and S2 are determined to be living signals. On the contrary, the identification result R1 indicates that the two rPPG signals S1 and S2 are recognized as non-living signals. If the two rPPG signals S1 and S2 are in vivo signals (identification result R2), calculation of a heart rate value is performed (step S45).

In step S50 of FIG. 1, in an embodiment, in order to ensure the effectiveness of the living body identification method, the method of the present invention can increase the judgment process of the hand shake degree, and automatically perform the hand on the operator who currently uses the application. The shaking detection, the detailed steps are shown in Figure 4. Firstly, a gravity sensor (step S51), such as a built-in gravity sensor (G-Sensor) in the smart phone, is used, and the gravity sensor is used to measure the hand of the holder of the smart phone. The signal STr (Tremor signal) is shaken (step S52). Next, a threshold value Th3 related to the degree of hand shake is set, and the hand shake signal STr is compared with the threshold value Th3 (step S53). During the identification process, if the hand shaking signal STr exceeds the threshold value Th3, it means that the hand shaking is too frequent and abnormal, and a warning window is jumped out in the operation interface and the living body identification is requested again (step S54) to ensure that Identify the accuracy of the results. On the other hand, if the hand shake signal STr does not exceed the threshold value Th3 during the identification process, it can be determined that the hand shake level of the current measurement is relatively stable and normal, and the identification result can be directly displayed in the operation interface (step S60).

In another embodiment, the hand shake degree determination flow (step S50) is not limited to be followed by step S40. This one-hand sway degree determination process can be introduced at any stage of the non-contact living body identification method.

In a more simplified embodiment, even if the skin color depth judgment (step S32) and the hand shake degree judgment (step S50) shown in Fig. 1 are omitted, the basic function of recognizing the living body can be achieved.

Compared with the prior art, the main difference point is that in addition to the live signal recognition algorithm, there is also a method for capturing the rPPG signal. In the aspect of the living body signal recognition algorithm, since the biggest difference between the living signal and the non-living signal is the degree of similarity of the rPPG signal period change, the present invention measures the same subject but captures two different skin areas. After the rPPG signal is processed by the algorithm, the similarity degree of the cyclic changes of the two is quantified as the basis for identifying the living body. In terms of the extraction of rPPG signals, it is worth noting that the present invention requires at least two different regions of interest (ROI) to extract rPPG signals for subsequent judgment and analysis. The method of the present invention can be implemented on a smart phone, so that the medical staff can easily, quickly and correctly identify whether a subject is in a living state, and can perform the rescue as soon as possible.

The above is only the preferred embodiment of the present invention, and the scope of the invention is not limited thereto, that is, the simple equivalent changes and modifications made by the scope of the invention and the description of the invention are All remain within the scope of the invention patent. In addition, any of the objects or advantages or features of the present invention are not required to be achieved by any embodiment or application of the invention. In addition, the abstract sections and headings are only used to assist in the search of patent documents and are not intended to limit the scope of the invention.

Steps for S10~S60‧‧‧ Non-contact living body identification method

S21~S22‧‧‧Steps for selecting the signal capture area

Steps for S31~S34‧‧‧ signal processing

S321~S324‧‧‧Steps of skin color classification process

Steps for S41~S45‧‧‧ Live Signal Identification Algorithm

S51~S54‧‧‧Hand shake determination step

P‧‧‧ zero crossing

R1, R2‧‧‧ identification results

S1 (rPPG signal S1) ‧‧‧ first physiological signal

S2 (rPPG signal S2) ‧ ‧ second physiological signal

S1', S2'‧‧‧ bandpass filter and rPPG signal output by the green-red difference algorithm

S1”, S2”‧‧‧ smoothed rPPG signal

Sc‧‧‧ skin color classification results

STr‧‧‧Hand shaking signal

T1, T2‧‧‧ time interval between each two adjacent zero crossings

Th1‧‧‧ color threshold

Th2‧‧‧ live signal threshold

Th3‧‧‧ threshold for hand sway

W1, W2‧‧‧ result waveform after cross-correlation calculation

Z‧‧‧zero axis

‧‧‧Standard deviation of the time interval between each two adjacent zero crossings

1 is a schematic flow chart of a non-contact living body identification method according to an embodiment of the present invention.

2 is a schematic flow chart of a skin color classification process according to an embodiment of the present invention.

FIG. 3 is a schematic flow chart of a living body signal recognition algorithm according to an embodiment of the present invention.

FIG. 3A is a waveform diagram showing the result of cross-correlation calculation of physiological signals of a living subject according to an embodiment of the present invention. FIG.

FIG. 3B is a waveform diagram showing the result of cross-correlation calculation of physiological signals of a non-live subject according to an embodiment of the present invention. FIG.

Fig. 4 is a flow chart showing the determination of the degree of hand shake in an embodiment of the present invention.

no

no

Claims (6)

A non-contact living body identification method includes: providing a photographing device for capturing an image of a subject; and selecting, by the photographing device, a different first skin region and a second skin region from the image; a microprocessor extracts a first physiological signal from the first skin area, and extracts a second physiological signal from the second skin area; the first physiological signal and the second Performing a cross-correlation calculation on the physiological signal to generate a result waveform: using the result waveform to quantify the degree of similarity between the first physiological signal and the second physiological signal; and determining the degree of similarity based on the quantization Whether the subject is a living body. The non-contact biometric identification method of claim 1, wherein the first physiological signal and the second physiological signal both include skin color information, the method further comprising: the first physiological signal and the second physiological The skin color information contained in the signal is classified according to the color depth to judge the skin color of the subject; and based on the judgment result of the skin color depth, it is determined whether an additional light source needs to be provided to illuminate the subject. The non-contact living body identification method of claim 1, further comprising: passing the first physiological signal and the second physiological signal through a band pass filter before performing the cross correlation calculation; After the filter, performing a green-red difference algorithm on the first physiological signal and the second physiological signal; and the first physiological signal processed by the green-red difference algorithm by a Kalman filter And smoothing the second physiological signal. The non-contact living body identification method according to claim 1, wherein the result waveform intersects a zero-axis at a complex zero-crossing point, and each of the two adjacent zero-crossing points has a time interval, wherein the quantizing step comprises: Providing a living signal threshold; calculating a standard deviation of the time intervals by the microprocessor; and comparing the standard deviation to the living signal threshold to obtain a recognition result. The non-contact living body identification method according to claim 1 is installed in a portable device in a software form, and the method further comprises: measuring a hand shaking signal of one of the holders; Setting a threshold value for the degree of hand shaking; comparing the hand shaking signal with the threshold value; and if the hand shaking signal exceeds the threshold, jumping out of a warning window and requesting a live identification again. The contactless living body identification method according to the first aspect of the invention, wherein the first physiological signal and the second physiological signal are light volume change trace signals.