CN114010193A

CN114010193A - Data acquisition and processing system

Info

Publication number: CN114010193A
Application number: CN202110287257.XA
Authority: CN
Inventors: 刘海峰; 刘民; 李榕; 吕正业; 华雯卿
Original assignee: Ecdata Information Technology Co ltd; Criminal Investigation Brigade Of Shanghai Public Security Bureau
Current assignee: Ecdata Information Technology Co ltd; Criminal Investigation Brigade Of Shanghai Public Security Bureau
Priority date: 2021-03-17
Filing date: 2021-03-17
Publication date: 2022-02-08

Abstract

A data acquisition and processing system comprises a physiological data acquisition unit, an audio and video data acquisition unit and a host, wherein a ballistocardiogram signal BCG of a target person is acquired through the physiological data acquisition unit, a photoplethysmography signal PPG of the target person is acquired through the audio and video data acquisition unit, the host calculates according to the ballistocardiogram signal BCG or the photoplethysmography signal PPG to obtain a physiological signal, and calculates according to the physiological signal to obtain a psychological abnormal index and a credibility index. In the action process, the invention acquires the heart rate, the breathing rate, the voice and other physiological data of the target person without sensing, extracts the spectrum characteristics, automatically detects and analyzes the physiological and psychological states of the target person, grasps the psychological abnormal fluctuation and the emotional change in real time, assists the related action persons to judge the credibility of the target person when answering the inquiry, helps the related action persons to determine the action direction and range, and plays an active role in strengthening action force, adjusting action strategy and the like.

Description

Data acquisition and processing system

Technical Field

The invention relates to a data acquisition and processing system.

Background

After an event occurs, the relevant action personnel will typically determine the direction and scope of the action based on information feedback obtained from the scene. However, there are many objects in this range and there is no sense that all objects need to be queried by the relevant personnel to further exclude the target personnel and reduce the range. Even if the target person can be locked, sometimes because of lack of key evidence, clues need to be mined from the inquiry process to determine the action direction. However, whether it is the target person, the victim or the witness, the data content may be made to the benefit of themselves, deviating from the fact or even going against it completely. Moreover, the data content is not proved by a third party or has auxiliary evidence, and is difficult to determine if the data content is not determined, which undoubtedly increases the difficulty of action.

Generally, in the above situations, a professional psychophysical testing expert needs to be mobilized, and by adopting a psychophysical detection technology to a target person, a suspicious part is found and inquired in a focused manner, so that the method is helpful for quickly searching out the target person, adjusting the action direction in time, breaking through a psychological defense line of a focused target, and promoting the target person to replace the fact. However, professional psychophysical testing experts and departments have scarce resources, the number of events that can be received and processed is very limited, event processing requires scheduling and waiting, and optimal action time is easily missed.

Traditional psychological tests are generally performed using a polysomnography tester. Various sensors are arranged on a target person, the target person is inquired according to a pre-designed problem, various physiological signals of the target person during the supply are observed, and the authenticity of the supply of the target person is judged according to the signal change condition.

The conventional multi-lead tester is applied in the testing process, and has the following obvious limitations:

1. the lack of "stealth" perception devices: when using a multi-lead tester, the target person needs to be introduced with the test principle and interpreted for the safety of the test before applying the sensors to the target person, who can connect the sensors of the test device after obtaining the consent of the target person. If the target person is not matched during the inquiry process, the inquiry is difficult to implement. In addition, the target person is easy to generate tension and interfere with the test result. How to complete the 'recessive' test under the condition that the target person does not sense is one of the urgent needs of technologies. Today, with the vigorous development of biomedical engineering technology, how to detect various physiological signals of a human body in a non-contact (or less-contact) manner is a very great challenge, which is also a hot research field in the world.

2. Lack of intelligent real-time analysis capability: the multi-lead tester only outputs original signals of various sensors, and psychophysical testing professionals analyze whether a target person lies or not from the fluctuation of the original signals according to personal experience, so that the interference of artificial subjective factors is easily introduced. In order to enable non-professional psychological testers to improve the working efficiency and the processing capacity by means of a psychological testing technology, the only way is to combine an artificial intelligence method to realize real-time intelligent analysis.

3. Lack of the integration of multi-mode psychological detection technology: when the target person lies, besides the fluctuation of physiological indexes, the voice and the expression also have slight changes, and the vanishing subtle changes just expose the real internal world of the target person. Also, the eyes are often referred to as windows for human soul. In real life, the observed object has a facial tiny expression, an eye spirit or behavior, and can sense whether deception exists in the real internal world of the observed object in a certain context scene. Meanwhile, psychological detection is a complex technology, has strong indirection and is easily influenced by various other error factors, so that a plurality of mode perception and identification technologies need to be comprehensively applied to judge target persons by mutual evidence, and the multimode psychological detection technology can be helpful for improving the test accuracy and the robustness under complex interference.

4. The lack of the combination of recent psychological and deep machine learning developments: just as the bottleneck of the development of artificial intelligence is how to make machines learn the emotional computing power of human beings, in this situation, how to combine the research results related to human psychology to promote the development of artificial intelligence is a world problem that is not solved by human beings. As psychologically believed, the cognitive burden on a person who is intentionally deceased can be significant. The latest research results suggest that a deceptive person lies, and the deceptive person tries to keep speaking and cannot contradict each other, and has a balance contradiction in the aspect of keeping the coordination of long-range and short-range memory.

5. The function of directly inquiring and pushing information according to data information is lacked: related departments have huge data resources, and besides the business systems of the related departments, a great amount of data from various functional departments, units, enterprises and even the Internet of the society are also available. In the action process, relevant information is retrieved in real time according to the data of the target person and is pushed out in real time, so that powerful material evidence can be provided for judging the authenticity of the data, and the data value is maximized.

Disclosure of Invention

The invention aims to provide a data acquisition and processing system, which can acquire the heart rate, the breathing rate, the voice and other physiological data of a target person in a non-sensible manner, extract frequency spectrum characteristics, automatically detect and analyze the physiological and psychological states of the target person, grasp the psychological abnormal fluctuation and emotional change of the target person in real time, assist related action persons to judge the credibility of the target person when answering an inquiry, help the related action persons to determine the action direction and range, and play a positive role in aspects of strengthening action strength, adjusting action strategies and the like.

In order to achieve the purpose, the invention provides a data acquisition and processing system which comprises a physiological data acquisition unit, an audio and video data acquisition unit and a host, wherein a ballistocardiogram signal BCG of a target person is acquired by the physiological data acquisition unit, a photoplethysmography signal PPG of the target person is acquired by the audio and video data acquisition unit, the host calculates to obtain a physiological signal according to the ballistocardiogram signal BCG or the photoplethysmography signal PPG, a psychological dyskinesia index is calculated according to the physiological signal based on a Support Vector Machine (SVM) machine learning model, and a reliability index is calculated according to the physiological signal based on a nonlinear dynamics characteristic HVG, so that related mobile persons are assisted in judging the reliability of the target person when answering an inquiry.

The physiological data acquisition device is internally provided with a ballistocardiogram signal sensor for acquiring ballistocardiogram signals BCG.

The physiological data collector has a Bluetooth transmission function and sends the ballistocardiogram signal BCG to the host in a Bluetooth wireless transmission mode.

The audio and video data acquisition device comprises a photoelectric volume information acquisition module which is used for acquiring a photoelectric volume tracing signal PPG.

The physiological signal comprises at least: a respiratory rate value RR, a heart rate value HR, a heart rate variability index and a sample entropy SampEn; the heart rate variability indicators include at least: total standard deviation SDNN, peak-to-peak separation, low frequency power LF, high frequency power HF.

In the action process, the invention acquires the heart rate, the breathing rate, the voice and other physiological data of the target person without sensing, extracts the spectrum characteristics, automatically detects and analyzes the physiological and psychological states of the target person, grasps the psychological abnormal fluctuation and the emotional change in real time, assists the related action persons to judge the credibility of the target person when answering the inquiry, helps the related action persons to determine the action direction and range, and plays an active role in strengthening action force, adjusting action strategy and the like.

Drawings

Fig. 1 is a schematic diagram of a data acquisition and processing system provided in an embodiment of the present invention.

Fig. 2 is a ballistocardiogram in an embodiment of the invention.

Fig. 3 is a schematic diagram of heart rate variability in an embodiment of the invention.

Fig. 4 is a flowchart of a data acquisition processing method provided in the embodiment of the present invention.

FIG. 5 is a graph of 50 consecutive IBI data profiles in an embodiment of the present invention.

Detailed Description

The preferred embodiment of the present invention will be described in detail below with reference to fig. 1 to 5.

The invention aims to provide a set of field auxiliary system suitable for basic level work for related action personnel by combining the technologies of the biopsychology, the artificial intelligence and the like, wherein the biopsychology is one of important theoretical bases.

One of the most representative indicators in biopsychology is emotion, which is the sum of a series of subjective cognitive experiences representing complex psychological and physical states resulting from a variety of feelings, thoughts and explicit behaviors. Fraud is typically an explicit act. In the course of deception, individuals are often accompanied by a stressful, anxious mood. Emotional arousal causes changes in peripheral physiological indicators. By monitoring the change of the physiological indexes, the specific emotion (stress, anxiety and fear) of the individual can be judged, and whether the individual has avoidance of certain matters or not can be judged. In addition, the autonomic nervous system generally refers to efferent parts of the peripheral nervous system that control visceral movement, and is divided into the sympathetic nervous system and the parasympathetic nervous system. They send nerve impulses to all cardiac muscle, smooth muscle, and glands, causing them to become excited and work. Such activities are generally not controlled by a person's consciousness or will and their activity may be detected by external physiological monitoring devices.

It is believed that the sympathetic and parasympathetic nervous systems exert a dual innervation of the visceral system, controlling and regulating physiological activity in an antagonistic manner under different emotional states. When an individual is in a stressed state (queried facts) and a stress reaction (stress) occurs, sympathetic nerve excitation causes an increase in heartbeat, an increase in blood flow, an increase in blood pressure, and a deepening of respiration, so that physiological functions are rapidly invoked; in contrast, parasympathetic nerves are dominant when the body is quiet, maintaining normal physiological balance.

According to the invention, the Heart Rate Variability (HRV) of the target person is calculated by collecting the heart rate and the pulse volume of the target person, and the psychological state of the target person in the moving process is monitored in real time by combining the respiration rate and the voice change.

As shown in fig. 1, the present invention provides a data acquisition and processing system, which comprises a physiological data acquisition unit 1, an audio/video data acquisition unit 2 and a host 3, wherein the physiological data acquisition unit 1 acquires physiological data of a target person, the audio/video data acquisition unit 2 acquires audio and video data of the target person, and the host 3 calculates in real time according to the physiological data and the audio and video data of the target person to obtain a psychological transaction index and a reliability index, so as to assist related actors in determining reliability when the target person answers an inquiry.

The physiological data acquisition unit 1 is internally provided with a heart impact signal sensor, the heart beats to spray blood into blood vessels of the whole body to cause body vibration, and an optical fiber detector in the heart impact signal sensor detects the weak vibration by sensing small light intensity change in a self-adaptive manner, so that information such as blood flow speed, respiration rate, heart rate variability and the like of a target person is extracted. The physiological data collector 1 has a Bluetooth transmission function and sends the physiological data of the target person to the host 3 in a Bluetooth wireless transmission mode.

The audio and video data acquisition device 2 comprises a photoelectric volume information acquisition module for extracting an interested area from a face image sequence acquired by a camera and calculating a remote photoelectric volume signal based on the interested area.

The host 3 processes the physiological data and the audio and video data of the target person, analyzes the fluctuation of Heart Rate Variability (HRV), the fluctuation of Heart Rate (HR), the physiological and psychological complexity and the speech emotion, monitors the change of the physiological, psychological and emotional states of the target person, synchronously feeds the result back to the related action persons, and provides various auxiliary testing tools.

The data acquisition and processing system acquires the non-sensory physiological data, the acquired physiological data comprise data such as a heart impact signal, a heart rate variability signal, a respiration rate, a blood volume pulse signal and the like of a target person, and meanwhile voice information and facial video information of the target person can be acquired in real time in the action process.

The data acquisition and processing system monitors the physiological state of the target person in the moving process in real time according to the acquired physiological data and the audio and video data of the target person and feeds the physiological state back to the system, and assists related moving persons to know the physiological state change of the target person in the moving process in time.

The data acquisition and processing system monitors the psychological state of the target person in real time, calculates and analyzes the psychological pressure change of the target person according to various physiological data changes of the target person in the action process, displays and divides abnormal degrees such as normal, fluctuation and abnormity in a graphic mode, and the higher the numerical value is, the higher the psychological pressure is, so that the relevant action person is assisted to determine the action direction and range. Through analyzing the voice data of the target personnel, the different pressure types, cognitive processes and emotional reactions of the target personnel in the action process are distinguished, 9 emotions such as excitation, pressure, suspicion, voice control and the like of the target personnel are judged in real time, the relevant action personnel are helped to master the emotional changes of the target personnel, the psychological pressure of the target personnel is increased, the psychological defense line of the target personnel is broken through, and the target personnel are prompted to replace the fact. On the basis of the physiological and psychological states and emotion analysis results of the target person, the credibility of each statement of the target person is researched and judged, and the related action persons are assisted to judge whether the statement of the target person is real or not. When abnormal psychological activities and emotional changes occur in the moving process of the target person, the system can immediately give out striking alarm reminding.

The data acquisition and processing system converts the audio data into characters in real time by means of a voice transcription function and forms an inquiry record. The data acquisition and processing system is networked with an external database, valuable clue information including personnel information, related events, accommodation information, railway ticket purchasing information and the like is searched according to keywords such as identification numbers and the like provided by target personnel in the process of operation, and the valuable clue information is pushed to related operation personnel in real time. The data acquisition and processing system can store audio and video records in the whole process and mark key time periods, has a playback function of test records, and can quickly locate the live condition of the key time periods. The data acquisition and processing system supports uploading of test records to a big data center, large data modeling analysis and machine learning of the test records are supported, and experience accumulation and action strategy optimization are facilitated.

In an embodiment of the present invention, the hardware model and the technical parameters of the data acquisition and processing system are set as follows:

1. the physiological data acquisition device comprises:

heart rate acquisition range: the frequency is updated for 1 time/second, wherein the frequency is 40-200 bpm, and the numerical value is stably output after the machine is rested for 15 seconds;

respiratory rate acquisition range: at 6-45 rpm, stably outputting the numerical value after resting for 20 seconds, and updating the frequency for 1 time/second;

HRV numerical monitoring: 0-100, stably outputting the numerical value after 2 minutes, and updating the frequency for 1 time/second;

emission power: 4 dBm;

the Bluetooth transmission rate: 2 Mbps;

the Bluetooth transmission frequency band: 2.4G;

working humidity: 10% -90%;

reference transmission distance: 10 meters (no more than 4 meters is recommended to ensure the stability of signal transmission);

a power supply: a 5V/2A mobile power supply;

ambient temperature range: -20 to 60 ℃;

2. an audio and video data acquisition device:

an image sensor: 1/2.8' 200 ten thousand pixels scan CMOS line by line;

maximum resolution: 1920 (H). times.1080 (V);

maximum frame rate: 60 frames;

image coding: h.265 high compression coding technique;

photographic characteristics: 120dB ultra-wide dynamic;

directional audio response frequency: 50 Hz-16 KHz (heart shape)/60 Hz-14 KHz (super heart shape);

directional audio sensitivity: -45dB/-30dB ± 3 dB;

directional pickup distance: the distance between the target person and the target person is 1-2 meters, and the maximum distance is not more than 2.5 meters;

directional pickup angle: plus or minus 30 degrees;

omnidirectional pickup response frequency: 20Hz to 20 kHz;

omnidirectional pickup sensitivity: -30 dB;

omnidirectional pickup signal-to-noise ratio: 80dB (1 meter 40dB source), 50dB (10 meter 40dB source);

the omnidirectional monitoring range: square meter 10 to 150 (no more than 30 is recommended for ensuring voice quality);

dynamic range: 104dB (1KHz at Max dB SPL);

output signal amplitude: 2.5Vpp/-25 db;

a power supply: a 12V/2A power adapter;

compatible with an operating system: win7, Win 10;

ambient temperature range: -20 to 60 ℃;

3. a host computer:

a CPU: intel i7 processors;

memory: 16G;

hard disk: an SSD 256G;

a data transmitter: 1.4GHz quad-core ARM processor, Bluetooth 4.2;

operating the system: win 7.

The invention also provides a data acquisition and processing method, which is shown in the result form of continuous numerical values and discrete labels and provides four types of index values in total, wherein the index values are respectively as follows: 1. basic physiological data; 2. psychologic abnormal movement; 3. emotion recognition data; 4. "reliability".

The "basic physiological data" covers the heart rate and the respiration rate, wherein the heart rate is the number of beats per minute of a human. The respiration rate is the number of breaths per minute of a person. Because the indexes are acquired in real time and calculated in real time, the dynamic change of the numerical value reflects the physiological change process and the change amplitude of the target person. See the table below for output index parameters of heart rate and respiratory rate.

TABLE 1 Heart Rate and respiration Rate output index

The psychological dyskinesia is calculated by using basic physiological data, and the Heart Rate Variability (HRV) provides a theoretical basis for the indexes. The two types are continuous numerical indexes, and the audio signals collected by the audio and video collector are processed into 9 types of classification label results of 'emotion recognition data' to be displayed in the system. Finally, "reliability" is the comprehensive analysis and judgment of the system by learning, representing, and inferring data using methods such as signal system dynamics (HVG), and is intended to represent a highly generalized numerical result.

And entering a real-time monitoring mode after the test is started, and dynamically displaying the dynamic change process of the psychological transaction, the speech emotion and the credibility of the target person within the last 1 minute.

The physiological data acquisition unit 1 comprises a heart impact signal sensor and is used for covertly acquiring physiological signals of a human body: ballistocardiogram (BCG) to perform real-time confidence analysis. The remote sensing heart rate is further estimated in a mode that an audio and video data acquisition unit 2 acquires a photoplethysmography (PPG) signal, and the two physiological measurement technologies are fused to form a quasi-contactless physiological measurement and credibility analysis system. When the observed object has small motion amplitude, the accuracy of collecting the heart rate and heart rate variation signals by using the BCG is higher, and when the observed object has large motion amplitude, the heart rate and heart rate variation signals are measured by switching to remote sensing PPG.

When a target person sits on the cushion of the physiological data collector, the optical fiber in the cushion can be slightly bent due to the compression of the body weight of the target person on the cushion, the light quantity received by the photoelectric detector is reduced due to the bending, the detected light quantity can be converted into current quantity, and further converted into a voltage signal through the amplifier. The motion of heart beat and lung respiration affects the pressure exerted by the body on the cushion, which is then represented by the periodic variation of the voltage signal, such as the curve of the time-dependent voltage signal shown in fig. 2 is the ballistocardiogram. As can be seen from fig. 2, the ballistocardiogram BCG is highly correlated with the well known electrocardiogram ECG. The interval between the R peak and the R peak in the ECG and the interval between the J peak and the J peak in the BCG can be used to calculate the heart rate of the measured person. In addition, the BCG can be used to record the respiration rate and the respiration amplitude of the person to be measured, in addition to the heart rate of the person to be measured. Generally, when a person is under stress, the heart rate tends to rise, while the breathing rate tends to fall, while the amplitude of breathing tends to deepen.

Heart Rate Variability (HRV) refers to the fluctuation between successive heart cycles, and is typically represented by the variation of the RR interval (the interval consisting of two adjacent R peaks of the heart cycle) collected from Electrocardiogram (ECG) data. Psychological and neuroscience studies have demonstrated that a sustained interaction between the sympathetic and parasympathetic nervous systems induces periodic changes in heart rate, which can be reflected by HRV measurements. Thus, HRV is considered an indicator of cardiac adaptation to environmental changes and is used as one of the major non-invasive methods of autonomic nerve function assessment. An increasing number of psychological studies demonstrate the link between HRV and emotional response. In the past 30 years, there have been numerous studies on HRV, and HRV analysis methods have been extensively developed. A Heart Rate Variability (HRV) indicator, an example of which is shown in fig. 3, may be used to guide the identification of an individual's mood. Examples of HRV series constructed from successive RR intervals (a) in the electrocardiogram signal, and HRV examples of healthy subjects under calm (B), fear (C), happiness (D), anger (E) and sadness (F).

Table 2 HRV characteristics in general

Generally, HRV-related features extracted in emotion recognition work are classified into time-domain features, frequency-domain features, and nonlinear features. The time domain features are used to quantify the amount of variability in successive heart beat intervals, which is typically expressed in raw time units, and sometimes in natural logarithms of the raw time units to make them more closely fit to a normal distribution. The frequency domain features are used to estimate the distribution of absolute or relative power in different frequency bands. The non-linear features are then used to quantify the unpredictability of the time series. The specific indexes and meanings thereof are shown in Table 2. These indices have been widely used as significant features for emotion recognition. For example: the ratio of low frequency power to high frequency power allows an estimate of the ratio between sympathetic and parasympathetic activity under controlled conditions, an important HRV feature in emotion recognition efforts. Based on these indicators, the emotion of a person can be identified using a machine learning model that supports a vector machine.

The data acquisition and processing method carries out psychological dyskinesia analysis and carries out classification training by using four indicators (RR interval standard deviation: SDNN, low-frequency power high-frequency power ratio: LF/HF, adjacent RR interval standard deviation: SD2 and sampling entropy: SampEn) of Heart Rate Variability (HRV). The four indexes respectively cover time domain, frequency domain and nonlinear characteristics of heart rate variability. To accurately measure these heart rate variability indicators, the components associated with the heart pulses are first extracted using a band pass filter with a frequency range of 0.5Hz to 4 Hz. In order to reduce the negative interference of the motion which is irrelevant to the pulse signal to the signal, the filtered signal is decomposed by using a maximum overlap discrete wavelet analysis method, and the J peak coordinate in the BCG is determined through the decomposed signal.

According to the relation between indexes such as heart rate, respiration rate and heart rate variability and individual emotion, a supervised learning model is established by the system, a support vector machine is used as a training model, the support vector machine calculates experience risks by using a hinge loss function, and the classifier has sparsity and robustness. And putting the data and the labels acquired through the experiment into a classifier for training to finally obtain an emotion detection model. The model is integrated in system function software, an integer numerical value of 0-100 is output for each section of supply of target personnel, and the higher the numerical value output by the model is, the higher the emotion abnormality degree is.

Referring to table 3, the output index parameters of the psychological dyskinesia show the psychological stress variation of the target person in a graph form, and the psychological dyskinesia is divided into three degrees of "normal", "fluctuation" and "abnormal". When the psychological abnormal state reaches a fluctuation state or an abnormal state, the target person receives or has a larger stimulus and response, and the related action person needs to pay close attention.

TABLE 3 psychological abnormal movement output index

The data acquisition and processing method is used for emotion recognition and analysis, and the emotion state of the target person during the time of supplying the words is an important index for judging the word supply reliability of the target person. The voice emotion analysis technology distinguishes different pressure types, cognitive processes and emotional reactions according to basic sample data and Bayes minimum error rate decision theory and other algorithms. The 129 audio parameters were used to accurately find and measure the involuntary changes in the sound waves and to create a key to indicate the emotional profile of the talker.

The human pronunciation mechanism is a very complex process, and to do so requires a significant amount of muscle and body organ involvement, and in some way synchronizes them in precise time. First, the brain understands a given situation and evaluates the impact due to speech. Then, if it is decided to speak, air is squeezed from the lungs up to the vocal cords, which causes them to vibrate at a particular frequency, producing sound. The vibrating air continues to flow to the brain-manipulated tongue, teeth, and lips to produce a sound stream that becomes a word or phrase understandable by humans. The brain closely monitors this process to ensure that the emitted sound expresses a unique intent, is understandable and can be heard by listeners. Due to this uninterrupted brain monitoring, every "event" of brain activity is reflected by the speech flow. The speech emotion analysis technique ignores what a subject says (i.e., specific content), and only focuses on brain activity reflected by speech.

The core of the speech emotion analysis technique stems from the fact that the information generation algorithm accurately monitors small changes from within the high frequency (RHFR) and lower frequency (RLFR). The sounds understood, heard, and analyzed by most humans are within both ranges.

Higher Frequency Range (RHFR): this value may reflect an emotional state where the excitement value is high or strong.

Lower Frequency Range (RLFR): this value may reflect stress status, thought level, and other cognitive processes (e.g., brain processing of images).

SPT: emotional Level (Emotional Level): about 100 to 300 for a normal male; about 200 to 400 for average women.

SPJ: cognitive Level (Cognitive Level): about 100 to 300 for both common men and women, cognition is the psychological process of understanding, thinking, learning, and judgment.

From the two original parameters of SPT and SPJ, it can be seen whether the tested object is a Person with perceptual thinking (Heart Person) or a Person with rational thinking (Head Person). The higher the SPT, the stronger the sensitivity, and the higher the SPJ, the stronger the rationality.

The SPT is associated with the level of EMO (emotional stress level), which is calculated based on the SPT.

SPJ is correlated with COG levels (cognitive stress levels), which are calculated based on SPJ.

Fmain: attention, tension and rejection (Associated With concentration and rejection): a very high Fmain value indicates concentration. Very low Fmain values indicate shame, embarrassment and reluctance to talk about the subject.

FX-index of "synchronous thinking" ("Parallel roads" -): with respect to the attention level, this index should be combined with the Fmain index. The higher the number the more dispersed the thought.

Fmain and FX are parameters for expressing the attention concentration degree of the testee, Fmain and FX are inversely proportional, and the higher the Fmain, the lower the FX, and vice versa. FX ═ 1 indicates that attention is focused on a major piece of thought.

Error rate (False rate): the overall emotional change from the baseline level is described.

Error probability (False probability): the likelihood of high risk suspicious content is shown as a percentage value.

The combination of the False rate and the False probability shows the possibility of existence of the concealed event, the normal value of the False rate is generally about 100, the percentage of the False probability ranges from 5% to 95%, and the higher the value, the higher the possibility of existence of the concealed event.

Please refer to table 4 for output index parameters of speech emotion.

TABLE 4 Speech Emotion output indicators

The 9 emotional states of the speech emotion monitoring target person are "normal", "excited", "stressed", "extremely stressed", "confused", "masquerading", "suspicious", "hesitation", and "highly suspicious", respectively.

The data acquisition and processing method carries out reliability analysis, tracks the change of the heart rate and the respiration rate according to the change conditions of the heart rate and the respiration rate in the cognitive test experiment, and outputs abnormal physiological signals when the change of the heart rate and the respiration rate accords with the change characteristics of the physiological indexes of a test object receiving relevant stimulation in the experiment. The system tracks the heart rate and the respiratory rate respectively, detects wave crests and wave troughs by using a peak detection technology, and calculates the relative distance between the wave crests and the wave troughs to judge the signal variation amplitude. The more the curve change pattern matches the curve change rule described in the experiment, the larger the index value given. On the other hand, the Horizontal Visual (HVG) is a non-linear method for describing the complexity of the system, which is computationally efficient, easy to analyze and process, and can reveal valid information of a real complex system from a time series. Based on the above, the system calculates an integer value of 0-100 per second, and negates the value according to the credibility concept, wherein the lower the value is, the higher the abnormality degree is.

See table 5 below for output index parameters for confidence analysis.

TABLE 5 confidence analysis output index

As shown in fig. 4, in an embodiment of the present invention, a data acquisition processing method is provided, which includes the following steps:

and step S1, collecting physiological signals.

The method comprises the steps of collecting a ballistocardiogram signal BCG of a target person through a physiological data collector, and collecting a photoplethysmography signal PPG of the target person through an audio and video data collector.

The method for acquiring ballistocardiogram signals through the physiological data acquisition unit comprises the following steps:

1. obtaining BCG original signal BCG_o；

2. Filtering through a band-pass filter;

bandpass filter f passing through two different frequency bands₁，f₂Separately obtaining signals S related to respiration_respirationAnd signals BCG related to heart pulses_h；

S_respiration＝f₁(BCG_o),f₁Frequency band: 0.1 Hz-0.5 Hz (1)

S_cardiac＝f₂(BCG_h),f₂Frequency band: 0.5 Hz-4 Hz (2)

The method for acquiring the photoplethysmographic signal of the target person through the audio and video data acquisition unit comprises the following steps:

1. face recognition algorithm: identifying and marking a target face for a first frame image in a face image sequence extracted by the audio and video data acquisition device by using a face identification library of opencv, and extracting input image characteristics;

2. the face tracking algorithm comprises the following steps: because the face is unique and obvious in the video signal, the CSRT algorithm is adopted to meet the face tracking requirement;

3. landmark-based ROI area labeling: labeling the face mark points with Dlib-64 point positions, and simultaneously performing Kalman filtering on the obtained face mark points to realize a motion correction effect;

4. extraction of PPG signal based on ROI chromaticity: the chromaticity is constructed by RGB three-color channel, the space average of the interested region is calculated, and the original RGB signal matrix is constructed

For the matrix

Normalization and projection processing are carried out to obtain a matrix

Wherein, P_pIs a projection matrix, and N is a normalized matrix;

order to

To S₁(t),S₂(t) carrying out alpha tuning to obtain a remote photoplethysmography signal PPG, wherein the calculation formula is as follows:

p(t)＝S₁(t)+α·S₂(t)with，

also using band-pass filters f₂Letter to letterThe signals are filtered to obtain a PPG signal related to the heart pulse_h；

S_cardiac＝f₂(PPG_o),f₂Frequency band: 0.5 Hz-4 Hz (3)

And step S2, preprocessing the signal to obtain the physiological signal index.

After the physiological signals are obtained, a signal preprocessing stage is carried out, namely the physiological signals obtained by various real-time monitoring are calculated, and a series of physiological indexes such as heart rate (heart rate, HR), Respiratory Rate (RR) and the like of a monitored person are obtained through calculation. From the practice of emotion recognition technology, physiological signals are more authentic and objective and are not controlled by subjective consciousness, can be better used for expressing the mapping rule of 'physiology-emotion psychology', and are necessary information for calculating 'emotion abnormal values'. Wherein the physiological signal index comprises: heart rate values, respiration rate values, sample entropy, heart rate variability (overall standard deviation SDNN, peak-to-peak separation, low frequency power LF, high frequency power HF), etc.

Decomposing the filtered signal using a maximum overlap discrete wavelet analysis based Method (MODWT);

for convenient representation, BCG is adopted_hAnd PPG_hIs uniformly expressed as a signal S related to the heart pulses_cardiacObtaining a heart pulse related signal S_cardiacThen a group of decomposed signals are obtained by maximum overlapping discrete wavelet decomposition

Where n is the sample size, and when n is 5, the sub-signal

The highest correlation to heart pulses;

determining J-peak coordinates from the decomposed signal using an automatic peak detector;

using an automatic peak detector f_AMPDFor respiration related signal S_respirationSignals with highest correlation to heart pulses

Carrying out peak detection to obtain the peak coordinates of the signal:

peak coordinate T obtained by automatic peak detector_res，T_carCan be used to calculate a physiological signal indicator associated with the human body. The specific calculation principle is as follows:

1. respiratory rate value RR

Suppose there are two adjacent respiratory peak coordinate values

The instantaneous respiration rate R can be calculated_rateComprises the following steps:

2. heart rate value HR

Similarly, assume that there are two adjacent heart pulse peak coordinate values

Then the instantaneous heart rate H_rateComprises the following steps:

3. index of heart rate variability

Defining peak-to-peak intervals based on the peak coordinate values of adjacent heart pulses

According to the calculated peak-to-peak intervals, the heart rate variability index of the individual can be calculated.

The overall standard deviation SDNN calculation formula is as follows:

wherein the content of the first and second substances,

both the sympathetic and parasympathetic systems contribute to the SDNN value, and the correlation of the different band powers with the SDNN is very high.

The calculation of the low frequency power LF and the high frequency power HF requires fast fourier transform of a time interval of peak-to-peak intervals IBI to obtain a frequency spectrum f (IBI).

The power over a certain frequency band is calculated as:

wherein i of low frequency power LF_L,i_HI of high frequency power HF of 0.04Hz, 0.15Hz respectively_L,i_H0.15Hz, 0.40Hz, respectively, N is the total number of sequences.

4. Sample entropy

The standard deviation of the adjacent RR interval differences may represent IBI for a segment of consecutive heart beat intervals by the index SD2_nThe following fig. 5 can be drawn on a two-dimensional plane with the nth heartbeat interval as the abscissa and the (n +1) th heartbeat interval as the ordinate. The distribution of these points can be approximated as an ellipse, the center of which is located at a (heart beat interval average ) determined coordinate point. The major and minor semi-axes of the ellipse are SD1 and SD2, respectively.

The formula for minor half axis SD2 is:

wherein x, y represent all horizontal axes, coordinate values of vertical axis, std (-) represents the calculated standard deviation.

The sample entropy SampEn is calculated as follows: for sequences consisting of N data IBI_nForming a group of vector sequences with dimension m according to the sequence numbers, X_m(1),...,X_m(N-m +1) wherein X_m(i)＝{IBI_i,IBI_i+1,...,IBI_i+m-1}。

Definition vector X_m(i) To X_m(j) Distance d [ X ] between_m(i),X_m(j)]Namely:

d[X_m(i),X_m(j)]＝max_{k＝0,...,m-1}(|IBI_i+k-IBI_j+k|) (11)

for a given X_m(i) Statistics of X_m(i) And X_m(j) The number of the distance between the two is less than or equal to r is marked as B_iFor 1. ltoreq. i.ltoreq.N-m, the definition:

definition B^(m)(r)：

Add dimension to m +1, calculate X_m+1(i) And X_m+1(j) The number of the distance between the two is less than or equal to r is marked as A_iFor 1. ltoreq. i.ltoreq.N-m, are defined

Definition A^(m)(r)：

The sample entropy is defined as:

and step S3, calculating the psychological abnormal change index.

And acquiring the psychological transaction based on a Support Vector Machine (SVM) machine learning model.

The psychological abnormal change is that all physiological indexes (heart rate HR, respiratory rate RR, SDNN, SD2, LF/HF, SampEn) in a period of time are input into a support vector machine in a vector form, the support vector machine obtains a numerical value with the output of 0-1 through training and learning collected label data, and the numerical value is multiplied by 100 to obtain an abnormal emotion score value y through rounding down:

the output y value is typically close to the

value

0 or 100, indicating that the trainer target person is in a critical state when the value is around 50.

The calculation of physiological outliers is mainly based on the instantaneous heart rate H_rateAnd the respiration rate R_rateMonitoring of (3). Whether the change of the physiological index is consistent with the conclusion of the cognitive test is determined by setting a plurality of valve values and labels. The tags a, b are initialized to 0. When the respiration rate decreases by 10% within 1s, label a is assigned 1; when the heart rate drops by more than 5%, the label b is assigned 1; if the heart rate then recovers and continues to drop by more than 5% over the next 10 seconds, label b is assigned a value of 2. The degree of heart rate and respiration rate decline corresponds to different coefficients w1, w 2. Specific physiological abnormality values are defined in table 6.

Table 6 physiological abnormal value output correspondence table

Wherein the content of the first and second substances,

where w1 and w2 are weights, and exp is the whole number mapped in the [ 0, 1 ] interval.

And step S4, calculating a reliability index.

The "confidence index" is calculated based on the nonlinear dynamics (HVG). The calculation of confidence level depends on the variation of the HVG value. Because of the differences between individuals, outputting psycho-outliers first requires determining the HVG baseline for each individual. Within 30 seconds after the system is started, the target person keeps a rest state, and the average HVG value in the period is the baseline HVG:

HVG_baseline＝mean(HVG(i)) (18)

thereafter, the HVG value of the signal is calculated every second, and the deviation value of the HVG is outputted as a psychological abnormality value. According to the real requirement, the 1.1 times of HVG baseline value is corresponding to the psychological abnormal value: 30, of a nitrogen-containing gas; a 1.2-fold HVG baseline value corresponds to psycho-outliers: 60. and (3) mapping by using an exponential function, and setting a constant coefficient in the formula through experiments to obtain a final empirical formula:

the target user is the related action personnel of each basic unit, and the related action personnel can conveniently master the psychological activities of the target personnel in real time in the action process. The installation and deployment environment needs to meet the following conditions:

1. providing 2 three-pin power interfaces;

2. providing 2 image network access ports and IP addresses;

3. the site has no strong electromagnetic interference, and no machine room site exists nearby;

4. in order to ensure stable data transmission, the straight-line distance between the physiological data collector and the host is recommended to be not more than 3 meters;

5. the audio and video data acquisition unit points to the direction of the target person, the arrangement distance of the audio and video data acquisition unit is within the range of 1.5-2.5 meters from the target person, and the closer the audio and video data acquisition unit is to the target person, the better the effect is. The collector is as far away as possible from the relevant action person and is adjusted to clearly display the face of the target person. In order to guarantee data accuracy, the face image of the front of the target person is collected as much as possible in the action process.

6. A computer with intelligent recording software installed is required to be deployed in the environment, and a voice transcription microphone is arranged. The system time of the computer must be consistent with the system time of the host.

The invention realizes the purpose of noninductive acquisition of the heart rate, the breathing rate, the voice and other physiological data of the target person, extraction of the frequency spectrum characteristics, automatic detection and analysis of the physiological and psychological states of the target person, real-time mastering of the psychological abnormal fluctuation and emotional change of the target person, assistance of the relevant action persons in judging the credibility of the target person in answering the inquiry, assistance of the relevant action persons in determining the action direction and range, and positive effects in aspects of strengthening action force, adjusting action strategies and the like.

Meanwhile, the voice information is transcribed into the test record in real time by means of the voice transcription function of the intelligent writing system in the action process. The key information provided by the target person is retrieved and compared with the relevant department system data platform, and the result is fed back in real time, so that the relevant action person is helped to obtain the relevant information of the case in time, and meanwhile, psychological pressure can be applied to the target person, and the progress is accelerated.

It should be noted that in the embodiments of the present invention, the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, which is only for convenience of describing the embodiments, and do not indicate or imply that the referred device or element must have a specific orientation, be configured and operated in a specific orientation, and thus, should not be construed as limiting the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.

While the present invention has been described in detail with reference to the preferred embodiments, it should be understood that the above description should not be taken as limiting the invention. Various modifications and alterations to this invention will become apparent to those skilled in the art upon reading the foregoing description. Accordingly, the scope of the invention should be determined from the following claims.

Claims

1. A data acquisition and processing system is characterized by comprising a physiological data acquisition unit, an audio and video data acquisition unit and a host, wherein a ballistocardiogram signal BCG of a target person is acquired through the physiological data acquisition unit, a photoplethysmography signal PPG of the target person is acquired through the audio and video data acquisition unit, the host calculates to obtain a physiological signal according to the ballistocardiogram signal BCG or the photoplethysmography signal PPG, a psychological abnormal movement index is calculated according to the physiological signal based on a Support Vector Machine (SVM) machine learning model, and a reliability index is calculated according to the physiological signal based on a nonlinear dynamics characteristic HVG, so that related actors can be assisted to judge the reliability of the target person when answering an inquiry.

2. The data acquisition and processing system as claimed in claim 1, wherein the physiological data acquisition device is internally provided with a ballistocardiogram signal sensor for acquiring ballistocardiogram signals BCG.

3. The data acquisition and processing system as claimed in claim 1, wherein the physiological data acquisition unit has a bluetooth transmission function and transmits the ballistocardiogram signal BCG to the host computer in a bluetooth wireless transmission mode.

4. The data acquisition and processing system of claim 1, wherein the audio/video data acquisition unit comprises a photoplethysmography information acquisition module for acquiring a photoplethysmography signal PPG.

5. The data acquisition and processing system as in claim 1, wherein the physiological signals comprise at least: a respiratory rate value RR, a heart rate value HR, a heart rate variability index and a sample entropy SampEn; the heart rate variability indicators include at least: total standard deviation SDNN, peak-to-peak separation, low frequency power LF, high frequency power HF.