TW201540258A - Fast image-based pulse detection method - Google Patents

Abstract

Disclosed herein is a fast image-based pulse detection method, which uses a camera to capture consecutive images of a human face for recognition of pulse signals, the method comprising: (1) using a camera to capture a human face image and perform the face detection; (2) obtaining several consecutive face images after a preset time, the averages of R, G, and B being calculated respectively for each face image, and three sets of color-time signal being obtained after the calculation is completed; (3) subjecting the three sets of color-time signal to a signal separation algorithm for producing three sets of noise-free time signal; (4) subjecting these three sets of noise-free time signal to an autocorrelation-based method for generating three sets of waveform; and (5) since a pulse value can be calculated using any one of these three sets of waveform, this invention proposing a calculation method to choose a correct waveform therefrom, so as to obtain a final pulse information.

Description

Fast image pulse detection method

The invention relates to an image pulse detection method, which uses a camera to capture a continuous face image to recognize a pulse signal, and belongs to the technology of pulse detection, and is particularly a technology for utilizing face image for pulse detection.

Image-based pulse detection is an emerging technology in recent years. The advantage of this technology is that pulse detection can be performed using a video camera that is very popular on the market, and this technology is non-contact, the face and sensor of the subject ( There is no direct contact between the cameras, unlike the traditional contact finger pulse detectors which have health concerns.

The current image-based pulse detection process is: (1) the camera captures the face image and performs face detection; (2) after a set time, several consecutive face images are acquired, and each face image is calculated. The average values of R, G, and B are respectively formed into three sets of color time signals after completion of the calculation; (3) the separation algorithm by a signal (J.-F. Cardoso, "High-order contrasts for independent component analysis," Neural Comput .11(1), pp.157-192, 1999.) The three sets of color time signals are processed, and the output produces three sets of time signals without noise; (4) the finger type is taken while the camera is photographing the face. The pulse detector detects the pulse and uses the generated time signal as a reference, which is similar to the similarity between the time signal of the reference and the three sets of time signals generated in step 3. The group time signal is most similar to the time signal of the finger type pulse detector; (4) the interval between the two peaks is calculated for the second group of time signals without noise by using the Fourier transform method, and the interval corresponds to the frequency of the pulse. This practice is disclosed in the U.S. Patent "Method and system for measurement of physiological parameters" (Patent Publication No.: US20110251493 A1).

In addition, as in the Chinese patent "Method and system for contact-free heart rate measurement" (Patent Publication No.: CN103040452 A), it is mentioned that the pulse value can be judged not only by the color change of the face, but also by the color change of the hand and the foot. To determine the pulse, the patent has proposed a skin color detection algorithm to automatically detect the position of the hand and the foot. Both of the above patents use Fourier transform to convert the color time signal to the frequency domain to find the highest energy frequency position, and then convert this frequency into a pulse value. The disadvantage of Fourier transform is that the measurement resolution is directly proportional to the analysis time. To improve the resolution, it must be achieved by increasing the analysis time. For example, if the image capture frequency is 30 fps (frames per second), the analysis time must be as long as 18 seconds, and the resolution of the pulse detection can be increased to 1.76 bpm (bits per minutc).

It can be seen that there are still many shortcomings in the above-mentioned methods of use, which is not a good designer, but needs to be improved.

In view of the shortcomings derived from the above-mentioned conventional methods, the inventor of the present invention has improved and innovated, and after years of painstaking research, he finally succeeded in researching and developing the rapid image pulse detection method.

The object of the present invention is to propose a self-correlation-based method, directly in the spatial domain analysis, and cooperate with a super-analytic method to overcome the Fourier transform. The problem of measuring resolution greatly shortens the analysis time and improves the practicality of image-based pulse detection.

A fast image pulse detection method for achieving the above object includes the steps of: taking a face image by a camera, setting a frequency of capturing the image of the camera, and performing face detection; and step 2: After the set image capturing time, a plurality of consecutive face images are obtained, and each of the face images calculates an average value of R, G, and B, and after the calculation is completed, each face image is Forming three sets of color time signals; step 3, processing the three sets of the color time signals by the signal separation algorithm, and outputting a new three sets of color time signal data; step four, for the new three sets of the color time The signal data is calculated by the autocorrelation coefficient method to generate three sets of waveforms; and in step five, the three sets of waveforms are used, and the correct waveform is selected to calculate the pulse value.

The fast image pulse detection method provided by the invention has the following advantages when compared with other conventional technologies:

1. The present invention differs from other disclosed image-based pulse detection methods in that the processing of the color time signal is performed in the spatial domain, and an auto-correlation method (super resolution) method is proposed. The detection time can be greatly shortened.

2. The present invention proposes to analyze three sets of color time signals by using an auto-correlation method and a super-analytic method, and generate three sets of waveforms. Since the three sets of waveforms can respectively calculate the pulse values, the present invention proposes a waveform selection (spectrum). Selection scheme) The correct waveform to get the final pulse information.

3. The color time signal of the face image is very stable. If the first sample analysis from the color time signal causes an error every time, the present invention proposes the same data shift scheme. The calculation method is corrected.

S101~S105‧‧‧ pulse detection method architecture flow

201‧‧‧Three sets of principal component time signals for removing noise

202‧‧‧Three correlation coefficients of three sets of principal component signals calculated by the autocorrelation method

203‧‧‧ Simulated Gaussian function

204‧‧‧Analysis of the simulated Gaussian waveform shifting to the left

205‧‧‧ Simulated Gaussian waveform shifting to the right

S300~S306‧‧‧ pulse detection method flow

The detailed description of the present invention and the accompanying drawings will be further understood. The technical content of the present invention and the purpose of the present invention will be further described. FIG. 1 is a flow chart of the fast image pulse detecting method of the present invention; FIG. The figure is a description of the waveform selection calculation method of the present invention; FIG. 3 is a flowchart of the fast image type pulse detection method of the present invention.

The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

The present invention will be further described with reference to the accompanying drawings. Please refer to FIG. 1 , which is a structural diagram of a fast image pulse detection method according to the present invention. First, a continuous camera image S101 is captured by a camera, where a camera capture is required. The frame per second (fps), for example, can set fps equal to 30; the face detection method is used to detect the face region of each image, and the color channel average value S102 of the face region R, G, B is calculated respectively. Since the face detection method is not the focus of this patent, the face detection function of the Intel open software OpenCV is directly used, but it is an inseparable process; several face images are continuously accumulated (for example, an image of 4 seconds can be set) After, R, G, B The average value can form three sets of time signals S103 respectively; the signal of three sets of time signals of R, G, and B is extracted by a signal separation calculation method, and three sets of main component time signals S104 having the same length as the original signal are outputted. The signal separation calculation method uses joint diagonalization of eigenmatrices (JADE).

Auto-correlation method with super resolution method (Y. Medan, E.Yair and D. Chazan, "Super Resolution Pitch Determination of Speech Signals," IEEE Transactions On Signal Processing, vol.39, issue .1, pp.40-48, 1991.) Calculate the autocorrelation coefficients of the three sets of principal component time signals respectively, and output three sets of autocorrelation coefficient waveforms S105. The purpose of the autocorrelation method is to find two of the principal component time signals. The integer sample interval of the peak, the detailed flow and mathematical symbols are defined as follows: Assume that the symbol h [1: L ]=( h ₁ , h ₂ ,..., h _L ) represents a discrete time signal with a sample number L ; The starting point i _{0 of} any sample, defining the symbols x _τ ( i ₀ ) and y _τ ( i ₀ ) is a time signal in which the number of samples in both segments is m , and the detailed formula is:

In other words, x _τ ( i ₀ ) is a time signal taken from h [1: L ] and the starting point is And y _τ ( i ₀ ) is also a time signal taken from h [1: L ] but the starting point is For a more concise symbol representation, write x _τ ( i ₀ ) as x _τ =( x ₁ , x ₂ ,..., x _m ), and write y _τ ( i ₀ ) as y _τ =( y ₁ , y ₂ ,..., y _m ), then the autocorrelation coefficients γ _τ ( x _τ , y _τ ) of the two time signals can be calculated by the following formula:

Where | x _τ | and | y _τ | are the vector lengths of x _τ and y _τ , respectively, and ( x _τ , y _τ ) represents the inner product of the two vectors x _τ and y _τ . The position τ with the highest autocorrelation coefficient is the integer sample interval of the two peaks in the time signal h [1: L ], and the mathematical formula is defined as follows:

Where R _min and R _max represent the minimum possible value and the maximum possible value of τ , respectively. For example, when the human pulse is assumed to be up to 200 times per minute and the lowest is 38 times, the calculation formula of the binary value is as follows:

However, in the real world, the signal is continuous, not discrete. Reference 2 considers that the true peak sample interval is not an integer R ₀ , but a floating point number R = R + α . The calculation method of this floating point number is called super resolution method, which can be overcome by this method. The problem of pulse detection error values. Wherein R is the integer part of the R, R [alpha] is the fractional part, and 0 ≦ α <1. The physical meaning of the alpha value can be explained by a linearly interpolated formula, as follows:

If the definition symbol α ^* is the best solution for α , then α ^* can be expressed as:

The formula derivation and theoretical proof of the calculation of α ^* value are described in Reference 2. It will not be repeated in the present invention. The formula is as follows:

The three sets of waveforms of S105 are calculated by using the autocorrelation method to respectively calculate the three sets of principal component time signals in the interval [ R _min , R _max ]. For example, when fps = 30, R _min = 9 and R _max = 48 in the formula (4).

The pulse waveform can be calculated for each of the three sets of waveforms of S105. The calculation method is to find the position R ₀ corresponding to the highest peak in the waveform, and the pulse can be calculated by pushing back according to formula (4). As for how to select the correct waveform, the present invention proposes a waveform selection algorithm.

Please refer to Figure 2 for the waveform selection calculation method. First, find a few testers to assist in the pulse detection test film. While shooting the film, use a traditional finger pulse detector to detect the pulse as a reference answer; observe each A test film uses the above calculation method to calculate the three sets of waveforms generated in S105 of Figure 1, and compares with the answer of the finger-type pulse detector; after observation, it is found that the correct answer corresponding to each test film is very similar to a Gaussian function. (Gaussian Function), which is the second waveform in the legend of Figure 1 S105. Therefore, the present invention proposes to perform a convolution with three waveforms by a Gaussian function generated by an automatic simulation, and the waveform with the highest number of rotations is the correct answer. Figure 201 is the three sets of principal component time signals for removing noise; 202 of Figure 2 is the three sets of autocorrelation coefficient waveforms calculated by the autocorrelation method for the three sets of principal component signals; Figure 203 is the simulated Gaussian The function, for example, can set the Gauss mean ( R _max - R _min )/2, and the deviation is ( R _max - R _min )/7; since the position corresponding to the vertex (peak) of the Gaussian waveform is The pulse value can be converted, so the peak of the Gaussian waveform does not appear in the center every time. The 204 in Fig. 2 is the case where the simulated Gaussian waveform shifts to the left, and the 205 in Fig. 2 is the case where the simulated Gaussian waveform shifts to the right. The waveform selection algorithm of the present invention can be expressed in mathematical form as follows:

Where G represents the simulated Gaussian function, C _k represents the kth autocorrelation coefficient waveform, so k is an integer and 1 ≦ k ≦ 3, z represents the average of the Gaussian function, and its range is [ Z _min , Z _max ], for example Z _min -( R _max - R _min )/4, Z _max =( R _max - R _min )*3/4 can be set. Thus the sign ( G * C _k )( z ) represents the convolution of G with C _k and the mean value of G is z . S ₀ represents the last selected waveform.

In addition, the real-world time signal is very stable (Stationary). If the first sample analysis from the time signal may result in an erroneous result each time, the present invention proposes a sample data shift algorithm for correction. The basic principle is to calculate the autocorrelation coefficient by changing the initial sample, and select the starting sample with the highest autocorrelation coefficient as the answer. The formula for combining the data sample displacement calculus with the waveform selection calculus method is as follows:

among them Represents the kth autocorrelation coefficient waveform, and the position of the data analysis start sample is i ₀ , L represents the total length of the time signal, for example, fps = 20 can be set, the analysis time is 4 seconds, then L = fps * analysis time ( sec) = 30 * 4 = 120, m is calculated from the length of time when the signal correlation, for example, when setting m = R _max = 48, then i is ₀ range ₀ ≦ i 0 <24. In addition, if the data shift calculation method of the present invention is not used, i ₀ has only one value, which is 0, and in this case, the analysis time is 3.2 seconds. Figure 1 S101~S105 is to find the peak sample interval R _{0 in the} time signal, and the correct waveform has been selected. At this time, the super-analytic method is used to calculate the floating-point number α ^* by interpolation. After adding the two, the correct pulse value is derived by the formula (4), so that the problem that the Fourier transform method is limited by the pulse detection resolution can be solved. (Note: The Fourier transform method must increase the number of sample analyses to improve pulse detection resolution).

Please refer to FIG. 3 , which is a flowchart of a fast image pulse detection method according to the present invention. The steps are as follows: Step 1: S300 camera captures facial image stream; Step 2: S301 uses face detection technology to detect accurately Face position Step 3: S302 calculates the average value of each face shadow item RGB to form three sets of color time signals; in step 4, S303 separates the three sets of principal component time signals by the signal separation algorithm; step 5, S304 is autocorrelation The method is combined with the data displacement calculation method to calculate the output three sets of autocorrelation coefficient waveforms; in step 6, S305 selects the correct waveform by the waveform selection algorithm; and step VII, S306 calculates the floating-point peak sample interval by super-analytic method, and converts into pulse output. .

According to the above steps, the face of the subject faces the camera, and the camera captures the face image stream; the face detection technology detects the accurate face position; and calculates the face area R, G, B of each image. The individual averages form three sets of color time signals after accumulating a certain number of images; the three sets of principal component time signals are output after the signal separation algorithm extracts the color time signals; the autocorrelation method and the data displacement calculation method respectively The three sets of principal component time signals are processed to calculate and output three sets of autocorrelation coefficient waveforms; since the three sets of autocorrelation coefficient waveforms can respectively calculate the pulse, the waveform selection algorithm must be used to select the correct waveform. Finally, the super-analytic method is used to find the floating-point sample interval of two peaks, which can be converted into pulse output.

The detailed description of the present invention is intended to be illustrative of a preferred embodiment of the invention, and is not intended to limit the scope of the invention. The patent scope of this case.

To sum up, this case is not only innovative in terms of technical thinking, but also has many of the above-mentioned functions that are not in the traditional methods of the past. It has fully complied with the statutory invention patent requirements of novelty and progressiveness, and applied for it according to law. You have approved this invention patent application, in order to invent invention, to the sense of virtue.

S101~S105‧‧‧ pulse detection method architecture flow

Claims (7)

A fast image pulse detection method is to capture a continuous facial image by a camera to recognize a pulse signal, and the steps include: Step 1: photographing a face image with a camera, and setting a frequency at which the camera captures the image, and then performing Face detection; Step 2: After a predetermined image capture time, a plurality of consecutive face images are acquired, and each of the face images calculates an average value of R, G, and B, and is calculated. After completion, each face image forms three sets of color time signals; in step 3, the three sets of the color time signals are processed by the signal separation algorithm, and a new three sets of color time signal data are generated; 4. For the new three sets of the color time signal data, the autocorrelation coefficient is used as the main method to calculate and generate three sets of waveforms; and step five, use the three sets of waveforms, and select the correct waveform to calculate the pulse value. . The fast image pulse detection method according to claim 1, wherein the face detection refers to detecting a location of a face region in a face image. The fast image pulse detection method according to claim 1, wherein the waveform refers to a waveform formed by the autocorrelation coefficient, and the calculating step of the waveform comprises: step one, calculating by a data displacement The method calculates a complex array autocorrelation coefficient for each set of time signals in the new three sets of color time signals generated by the signal separation algorithm; and second, selects the waveform having the highest autocorrelation coefficient; and step three The new three sets of color time signals are processed in the same way. There are still three sets of autocorrelation coefficient waveforms. For example, in the fast image pulse detection method described in claim 1, wherein the correct waveform is selected, the waveform selection calculation method is used. The rapid image pulse detection method according to claim 1, wherein the calculating the pulse value is performed by calculating a floating point number peak sample interval by a super-resolution calculation method, and then converting the pulse into a pulse, wherein the steps include: Step 1: Calculate the floating-point peak sample interval of the autocorrelation coefficient waveform selected by the waveform selection module by using the super-resolution calculation method; and convert to the pulse output by the second step. The fast image pulse detection method according to claim 3, wherein the data displacement calculation method is performed by calculating the autocorrelation coefficient of the time signal by shifting one sample at a time. For example, the fast image pulse detection method described in claim 4, wherein the step of the waveform selection calculation method comprises: step one: simulating a Gaussian waveform; and step two, respectively, respectively, three sets of autocorrelation The coefficient waveform is subjected to a convolution operation; and in step 3, the waveform having the highest number of rotations is selected.