CN114431849A - Aquatic animal heart rate detection method based on video image processing - Google Patents
Aquatic animal heart rate detection method based on video image processing Download PDFInfo
- Publication number
- CN114431849A CN114431849A CN202210023274.7A CN202210023274A CN114431849A CN 114431849 A CN114431849 A CN 114431849A CN 202210023274 A CN202210023274 A CN 202210023274A CN 114431849 A CN114431849 A CN 114431849A
- Authority
- CN
- China
- Prior art keywords
- interest
- region
- heart
- signal sequence
- component
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
- A61B5/024—Detecting, measuring or recording pulse rate or heart rate
- A61B5/0245—Detecting, measuring or recording pulse rate or heart rate by using sensing means generating electric signals, i.e. ECG signals
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0033—Features or image-related aspects of imaging apparatus classified in A61B5/00, e.g. for MRI, optical tomography or impedance tomography apparatus; arrangements of imaging apparatus in a room
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
- A61B5/0077—Devices for viewing the surface of the body, e.g. camera, magnifying lens
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
- A61B5/316—Modalities, i.e. specific diagnostic methods
- A61B5/318—Heart-related electrical modalities, e.g. electrocardiography [ECG]
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B2503/00—Evaluating a particular growth phase or type of persons or animals
- A61B2503/40—Animals
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A40/00—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
- Y02A40/80—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in fisheries management
- Y02A40/81—Aquaculture, e.g. of fish
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Molecular Biology (AREA)
- Animal Behavior & Ethology (AREA)
- Physics & Mathematics (AREA)
- Veterinary Medicine (AREA)
- Biophysics (AREA)
- Pathology (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Public Health (AREA)
- Surgery (AREA)
- General Health & Medical Sciences (AREA)
- Cardiology (AREA)
- Physiology (AREA)
- Signal Processing (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Psychiatry (AREA)
- Image Processing (AREA)
- Ultra Sonic Daignosis Equipment (AREA)
Abstract
The invention discloses a method for detecting the heart rate of aquatic animals based on video image processing, which comprises the steps of obtaining a video image sequence of the ventral surface of an aquatic animal, selecting a heart region of interest and a background region of interest, keeping tracking, extracting a common environmental noise component of the heart region of interest and the background region of interest by adopting a multi-data set joint analysis method, and setting the common environmental noise component to zero; then, according to the bicolor reflection model, further eliminating static components in the bicolor reflection model, and selecting specular reflection components and diffuse reflection components; then, performing independent component analysis on a time signal sequence consisting of the specular reflection component and the diffuse reflection component, and calculating through correlation to obtain an independent component containing most heart rate information; finally, carrying out frequency domain decomposition on the obtained product, screening out the subcomponents with the highest matching degree in the frequency domain according to the specific heart rate range of the aquatic animal, and outputting a cardiac oscillogram or calculating a heart rate value; the scheme provides a robust and accurate method for non-contact detection of the heart rate of the aquatic animals, and has wide application prospects in the aspects of aquaculture monitoring and biological stress resistance evaluation.
Description
Technical Field
The invention relates to the technical field of biological image information, in particular to a method for detecting the heart rate of aquatic animals based on video image processing.
Background
The aquaculture industry is an important component of fishery in China, but under the conditions of climate change and unreasonable culture modes, the problems of disease outbreak, high-temperature death and the like can occur. Breeding of aquatic animals with increased stress resistance or tolerance by selection is one of the most effective measures for dealing with this problem.
The heart rate is an important physiological index and is closely related to the metabolic state of the aquatic animals and the stress on environmental changes. Thus, technicians often use heart rate as an evaluation index to assess the ability of aquatic animals to cope with internal or external environmental changes and to assist in their genetic breeding efforts. The existing heart rate measurement of aquatic animals has certain limitations, mainly shows that the activity of organisms needs to be limited, continuous contact is needed, even surgical operations need to be carried out, and the heart rate of the aquatic animals is measured by adopting an implanted electrode method, a Doppler ultrasonic method and the like. The conventional remote non-contact heart rate measuring method often lacks a high-precision signal denoising method and a signal separation method. Because the physiological structure of aquatic animals is special, and most of the aquatic animals do not have rich capillary vessels distributed under superficial epidermis, the physiological information contained in the video image is very weak, and more accurate and more robust technical means are often needed to extract heart rate information. Therefore, in the existing breeding research, a method for conveniently observing the individual heart rate of the aquatic animal under a non-contact condition is lacked, so that an accurate and robust method for detecting the heart rate of the aquatic animal is imminent.
Disclosure of Invention
In view of the above, the invention aims to provide a method for detecting the heart rate of aquatic animals based on video image processing, which can broaden the stress resistance evaluation index range for genetic breeding researchers, thereby increasing the evaluation dimension in the genetic breeding process.
In order to achieve the technical purpose, the technical scheme adopted by the invention is as follows:
an aquatic animal heart rate detection method based on video image processing, comprising:
1) acquiring t frames of video images of the ventral surface of the aquatic animal, then selecting an interested region according to preset conditions, tracking and maintaining the interested region, and acquiring a time signal sequence of a heart and a background region, wherein the interested region comprises a heart interested region and a background interested region;
2) screening and removing common environmental noise shared by the heart region of interest and the background region of interest according to a multi-dataset joint analysis method;
3) according to a two-color reflection model, regarding the heart region-of-interest time signal sequence subjected to the removal of the common environmental noise in the step 2) as a linear combination of an illumination change component, a specular reflection component and a diffuse reflection component, eliminating a static component of the heart region-of-interest time signal sequence, and projecting the heart region-of-interest time signal sequence to an orthogonal plane to remove the illumination change component;
4) analyzing independent components of the heart region-of-interest time signal sequence which only contains specular reflection components and diffuse reflection components and is obtained through processing in the step 3), and screening out the independent components with the strongest green channel correlation in the original heart region-of-interest signal sequence;
5) decomposing the independent components obtained by the processing in the step 4) in a frequency domain, and screening out the subcomponents with the highest matching degree with the heart rate distribution range specific to the aquatic animal species in the frequency domain;
6) and further outputting a cardiac waveform chart or a heart rate value according to the sub-components obtained by the processing of the step 5).
As a possible implementation manner, further, in step 1), the video image is a digital image describing colors in different color spaces; the aquatic animal is an aquatic animal having a heart organ and including red blood cells in a blood tissue component thereof.
As a possible implementation manner, further, in step 1), a heart region of interest and a background region of interest are selected through a manual selection or target detection algorithm and tracked and maintained through a target detection algorithm; the obtained cardiac and background region time signal sequences are time signal sequences obtained after removing quantization noise by spatial averaging.
As a possible implementation manner, further, in step 2), the multi-dataset joint analysis method is a joint blind source separation method based on multiple datasets.
As a possible implementation manner, further, in step 3), the static component is eliminated by performing time domain normalization processing on the signal.
As a possible implementation manner, further, in step 5), the independent components obtained by the processing in step 4) are decomposed from the frequency domain according to a set empirical mode decomposition method.
As a possible implementation manner, further, in step 6), a heart rate value corresponding to the signal is obtained by calculating through a frequency domain or time domain method according to the sub-components obtained by the processing in step 5), and then the heart rate value is output or converted into a cardiac waveform diagram for output.
Based on the above scheme, the present invention further provides a computer-readable storage medium, wherein at least one instruction, at least one program, a code set, or a set of instructions is stored in the storage medium, and the at least one instruction, at least one program, a code set, or a set of instructions is loaded by a processor and executes the method for detecting a heart rate of an aquatic animal based on video image processing to realize the above method.
By adopting the technical scheme, compared with the prior art, the invention has the beneficial effects that:
1. the scheme of the invention utilizes the background content which is originally redundant information: for the removal of the background noise, a traditional and general processing method is not adopted, but a background region of interest is added, and source signal component vectors shared by the heart region of interest and the background region of interest are extracted through the combined blind source separation and are regarded as the background noise to be removed.
2. The scheme of the invention adopts a bicolor reflection model to separate signals containing heart rate information: according to the two-color reflection model, the signal is regarded as an illumination change component, a specular reflection component and a diffuse reflection component, the illumination change is that the brightness intensity changes along with the distance between the light source, the shooting object and the camera, the specular reflection component is the specular reflection component generated by direct reflection of the skin surface, and the diffuse reflection component is the diffuse reflection component which is re-absorbed by subcutaneous tissues and blood after transmitting the skin surface. The method has the advantages that the forming source of the heart rate information is explained from the physiological and physical level and the combination mode with other components can provide more accurate and robust extraction of the components containing the heart rate information.
3. According to the scheme of the invention, when components containing heart rate information are further screened, the physiological structure of aquatic animals is fully considered, and the green channel signal of the original signal is used as the screening standard. Hemoglobin, which is a constituent of the blood of aquatic animals, is colored and has an absorption spectrum in the visible region. The maximum absorption of hemoglobin in aquatic animals is between 540 and 575 nm for the Oxy (Oxy) type and between 538 and 568 nm for the carbonyl (carbonyl) type, and there is no significant difference compared to mammals. Therefore, the green channel signal is adopted as the standard, so that a more theoretical support and a more accurate screening mode can be realized.
4. The scheme of the invention adopts a set empirical mode decomposition method to further separate the frequency domain of the independent components after the independent components are analyzed and screened. Through ensemble empirical mode decomposition, the selected independent components are divided into intrinsic mode functions on a plurality of frequency bands, and the intrinsic mode functions with the most expected frequencies can be selected for cardiac oscillogram output or heart rate calculation according to the theoretical heart rate distribution range of aquatic animal species. The method has the advantages that the signal-to-noise ratio is further improved from the frequency domain, component interference except the expected frequency is eliminated, and the reliability of the calculation result is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a schematic flow chart of a non-contact aquatic animal heart rate detection method based on video image processing according to the present invention;
FIG. 2 is a schematic diagram of the selection of a heart region of interest and a background region of interest according to an embodiment of the present invention;
FIG. 3 is a signal sequence diagram of a region of interest of the heart involved in an embodiment of the present invention;
FIG. 4 is a background region of interest signal sequence chart involved in an embodiment of the present invention;
FIG. 5 is a vector diagram of the source signal components after joint blind source separation according to embodiments of the present invention;
fig. 6 is a time signal sequence diagram of a cardiac region of interest reconstructed after removing environmental noise according to an embodiment of the present invention;
FIG. 7 is a time signal sequence chart of a region of interest of the heart including only specular and diffuse components after elimination of a time-varying illumination variation component by projection according to an embodiment of the present invention;
FIG. 8 is a diagram of the individual components after analysis of the individual components involved in an embodiment of the present invention;
FIG. 9 is a diagram illustrating eigenmode functions of frequency bands after a set empirical mode decomposition process according to an embodiment of the present invention;
FIG. 10 is a diagram of cardiac waveforms involved in an embodiment of the present invention;
FIG. 11 is a graph of fast Fourier transform spectra involved in an embodiment of the present invention;
FIG. 12 is a diagram of peaks and valleys in a peak detection method according to an embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be noted that the following examples are only illustrative of the present invention, and do not limit the scope of the present invention. Similarly, the following examples are only some but not all examples of the present invention, and all other examples obtained by those skilled in the art without any inventive work are within the scope of the present invention.
In this embodiment, after anesthetizing the large yellow croaker, an implantable electrocardiograph is connected around the pericardium through a surgical operation, and the electrocardiograph acquires data and simultaneously shoots a ventral video of the large yellow croaker through a camera to obtain a video image sequence.
With reference to fig. 1, the present scheme provides a method for detecting a heart rate of an aquatic animal based on video image processing, and the general technical idea is as follows: acquiring t frames of video images of aquatic animals, respectively selecting a heart region of interest and a background region of interest, acquiring a signal sequence of the average pixel intensity of the heart region of interest and the background region of interest along with time change, extracting environmental noise components shared by the heart region of interest and the background region of interest by adopting combined blind source separation, and setting the environmental noise components to zero to obtain a heart region of interest time signal sequence after environmental noise is removed; then, according to a two-color reflection model, a new heart region-of-interest time signal sequence is regarded as a linear combination of an illumination change component, a specular reflection component and a diffuse reflection component, time domain normalization is carried out on the time signal sequence to eliminate static components in the time signal sequence, and then the normalized time signal sequence is projected to a plane orthogonal to the illumination change component to screen out the specular reflection component and the diffuse reflection component; then, carrying out independent component analysis on a time signal sequence consisting of specular reflection components and diffuse reflection components, and calculating the correlation between each independent component and a green channel signal in the time signal sequence of the original heart region of interest to obtain an independent component containing the most diffuse reflection components; and finally, performing ensemble empirical mode decomposition on the independent components containing the most diffuse reflection components, screening out an intrinsic mode function with the highest matching degree in a frequency domain according to the specific heart rate range of the aquatic animal species, and performing sliding average on the intrinsic mode function in a time domain to output the intrinsic mode function as a cardiac oscillogram or further calculate the heart rate value.
Specifically, the scheme comprises the following implementation steps:
1) acquiring t frames of video images of the ventral surface of the large yellow croaker, converting the video images into RGB video images, respectively selecting a heart region of interest and a background region of interest in the video images by adopting a manual selection mode in the 1 st frame of video images as shown in the figure 2, and then completing the tracking and maintaining of the region of interest by adopting a target tracking algorithm in the subsequent t-1 frame of video images so as to complete the work of selecting the heart region of interest and the background region of interest and completing the tracking and maintaining of the region of interest by a target detection algorithm; then, for each region of interest, respectively calculating the average pixel intensity of three color channels of each frame of RGB video image, and obtaining a heart region of interest signal sequence shown in fig. 3 and a background region of interest signal sequence shown in fig. 4, where the formula of the heart region of interest signal sequence is as follows:
Xhr(t)＝[Rhr(t)；Ghr(t)；Bhr(t)]T，
wherein R ishr(t)、Ghr(t)、Bhr(t) average pixel intensities of R, G, B color channels, respectively, of the cardiac region-of-interest video image;
the formula of the background region-of-interest signal sequence is as follows:
Xbg(t)＝[Rbg(t)；Gbg(t)；Bbg(t)]T；
wherein R isbg(t)、Gbg(t)、Bbg(t) average pixel intensities of R, G, B color channels, respectively, of the background region-of-interest video image;
2) according to the combined blind source separation method, the signal sequence X of the heart interesting region is processed by using the mathematical formula SCV (t) ═ W (t)hr(t) and background region of interest signal sequence Xbg(t) performing combined blind source separation processing to obtain a unmixing matrix W, wherein W is the unmixing matrix, and SCV (t) is a source signal component vector;
further calculation can obtain the source signal component vector matrix shown in FIG. 5
SCV(t)＝[SCVhr1；SCVhr2；SCVhr3；SCVbg1；SCVbg2；SCVbg3]T；
The joint blind source separation method is a joint blind source separation frame realized based on Gaussian independent vector analysis;
based on the method, each source signal component vector SCV of the heart region-of-interest signal sequence can be obtained by calculating according to the source signal component vector matrixhr(n)(t) and the respective source signal component vectors SCV of the background region-of-interest signal sequencebg(n)(t); the vector SCV of each source signal component of the cardiac region-of-interest signal sequence can then be obtained from the calculationhr(n)(t) and the respective source signal component vectors SCV of the background region-of-interest signal sequencebg(n)(t) correlation coefficient between the two, the vector SCV of source signal component of the heart region-of-interest signal sequence corresponding to the maximum correlation coefficienthr(m)(t) obtaining SCV after setting to zeronew(t), then according to scv (t), W X (t), a cardiac region-of-interest signal sequence and a background region-of-interest signal sequence X with the common environmental noise removed can be obtainednew(t)＝W\SCVnew(t)；
3) According to the two-color reflection model, the heart region-of-interest time signal sequence X obtained in the step 2) after the environmental noise is removed and shown in FIG. 6new(t)[1:3]X is a linear combination of illumination variation component, specular reflection component and diffuse reflection component, and is normalized in time domainnew(t) projection onto a plane P ═ P orthogonal to the illumination variation component1；p2]TThereby obtaining a heart region-of-interest time signal sequence X only containing specular reflection components and diffuse reflection components as shown in FIG. 7rflct(t)＝P*Xnew(t)；
4) For the heart interested region time signal sequence X only containing the specular reflection component and the diffuse reflection component obtained in the step 3)rflct(t) conducting an independent component analysis process, namely XrflctWhere a is a mixing matrix and s (t) is an independent component of the source signal (as shown in fig. 8), s (t) may be obtained
[s1(t)；s2(t)]TCalculate [ s ]1(t)；s2(t)]TSignal sequence X of interested region of hearthr(t) in Ghr(t) screening out the independent component s (t) with the strongest correlation with the green channel signal of the original signal;
5) processing s (t) obtained in the step 4) by using an ensemble empirical mode decomposition method to obtain s (t) ∑ imf (t) + r (t), as shown in fig. 9, wherein imf (t) is an intrinsic mode function of each frequency band, r (t) is a residual error term, screening imf (t), selecting imf (t) with the highest matching degree with the large yellow croaker heart rate distribution range in a frequency domain, performing sliding average, calculating an average value of consecutive n terms before and after the average value to replace a current value in imf (t), and recording a result as hr (t);
6) outputting directly as a cardiac waveform map according to the HR (t) obtained in step 5); or further, converting the HR (t) into the frequency domain by Fourier transform from the frequency domain to obtain the frequency value f with the maximum amplitude from the frequency spectrummaxThen the heart rate HR is f max60; adopting peak value detection method in time domain to obtain HR (t) with a wave crests and b wave troughs, and determining heart rate
Where t is the video duration in seconds.
As shown in fig. 10, 11 and 12, which are the cardiac waveform diagram, the fast fourier transform spectrum and the peak and trough in the peak detection method, respectively, the heart rate was calculated to be 65BPM by the frequency domain method, the heart rate value was calculated to be 68BPM by the time domain method, and the heart rate of the large yellow croaker was measured to be 67BPM by the electrode method.
The results show that the results generated by the two heart rate calculation methods are consistent with the result obtained by simultaneously adopting the verified method (electrode method), and the effectiveness and the reliability of the method for detecting the heart rate of the large yellow croaker are verified to a certain extent.
In the embodiment, after the target large yellow croaker video is acquired by the low-cost common camera, the heart rate of the large yellow croaker can be acquired by only a digital signal processing means, so that a sensor or an electrode does not need to be placed on the large yellow croaker body, non-contact heart rate measurement can be completed by only using the camera or a camera, and non-contact and non-invasive large yellow croaker heart rate detection is realized.
The scheme of the invention can be widely applied to the genetic breeding work of the large yellow croaker, and has the advantages of low cost, simple and convenient implementation mode and real-time and rapid speed; can effectively observe the tolerance state of the large yellow croaker, and can not cause the stress reaction of the large yellow croaker due to long-term contact measurement.
In the scheme of the embodiment, the original data acquisition is performed based on two interested areas (the heart interested area and the background interested area), and compared with a mode of acquiring the original data only by adopting a single interested area (only acquiring the heart interested area), the method has the advantages that the obtained detection result is more accurate, interference factors in process calculation are less, and the result has higher reference value; in addition, the scheme performs projection decomposition on the signals based on a physical model, namely a two-color reflection model, and the two-color reflection model fully considers the interaction of incident light and the surface of biological skin and can more accurately separate reflected light signals containing heart rate components; according to the scheme, an independent component analysis method is utilized to decompose the signals, and the independent component analysis can further separate sub-components with stronger heart rate components according to the characteristics of reflected light of blood tissues; the scheme also utilizes ensemble empirical mode decomposition to complete frequency domain decomposition, the ensemble empirical mode decomposition is an adaptive method, a cut-off frequency range does not need to be set, traditional frequency filtering needs to set a cut-off frequency in advance, the scheme is a predictive method, and the scheme is applicable to animals living in water, which have heart organs and blood tissue components including red blood cells, and include aquatic animals (such as large yellow croakers) with opaque skin and invisible heartbeat eyes.
The above description is only a part of the embodiments of the present invention, and not intended to limit the scope of the present invention, and all equivalent devices or equivalent processes performed by the present invention through the contents of the specification and the drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. An aquatic animal heart rate detection method based on video image processing is characterized by comprising the following steps:
1) acquiring t frames of video images of the ventral surface of the aquatic animal, then selecting an interested region according to preset conditions, tracking and maintaining the interested region, and acquiring a time signal sequence of a heart and a background region, wherein the interested region comprises a heart interested region and a background interested region;
2) screening and removing common environmental noise shared by the heart region of interest and the background region of interest according to a multi-dataset joint analysis method;
3) according to a two-color reflection model, regarding the heart region-of-interest time signal sequence subjected to the removal of the common environmental noise in the step 2) as a linear combination of an illumination change component, a specular reflection component and a diffuse reflection component, eliminating a static component of the heart region-of-interest time signal sequence, and projecting the heart region-of-interest time signal sequence to an orthogonal plane to remove the illumination change component;
4) analyzing independent components of the heart region-of-interest time signal sequence which only contains specular reflection components and diffuse reflection components and is obtained through processing in the step 3), and screening out the independent components with the strongest green channel correlation in the original heart region-of-interest signal sequence;
5) decomposing the independent components obtained by the processing in the step 4) in a frequency domain, and screening out subcomponents with the highest matching degree with the specific heart rate distribution range of the aquatic animal species in the frequency domain;
6) and further outputting a cardiac waveform chart or a heart rate value according to the sub-components obtained by the processing of the step 5).
2. The method for detecting the heart rate of the aquatic animals based on the video image processing as claimed in claim 1, wherein in the step 1), the video image is a digital image which explains colors in different color spaces; the aquatic animal is an aquatic animal having a heart organ and including red blood cells in a blood tissue component thereof.
3. The method for detecting the heart rate of the aquatic animals based on the video image processing as claimed in claim 1, wherein in the step 1), the heart region of interest and the background region of interest are selected by a manual selection or target detection algorithm and tracked and maintained by the target detection algorithm;
the obtained cardiac and background region time signal sequences are time signal sequences obtained after removing quantization noise by spatial averaging.
4. The method as claimed in claim 1, wherein in step 2), the multi-dataset joint analysis method is a joint blind source separation method based on a plurality of datasets.
5. The method as set forth in claim 1, wherein in step 3), the static component is eliminated by performing time domain normalization processing on the signal.
6. The method as claimed in claim 1, wherein in step 5), the independent components obtained from the processing of step 4) are decomposed from the frequency domain according to a set empirical mode decomposition method.
7. The method as claimed in claim 1, wherein in step 6), the sub-components obtained by the processing in step 5) are used to calculate a heart rate value corresponding to the signal by a frequency domain or time domain method, and then the heart rate value is outputted or converted into a cardiac waveform for outputting.
8. A method as set forth in any one of claims 1 to 7 for detecting heart rate of aquatic animals based on video image processing, comprising:
1) acquiring t frames of video images of the ventral surface of the aquatic animal, converting the video images into RGB video images, selecting a heart region of interest and a background region of interest, and completing the tracking and maintaining of the region of interest by a target detection algorithm; then, for each region of interest, respectively calculating the average pixel intensity of three color channels of each frame of RGB video image, and obtaining a heart region of interest signal sequence and a background region of interest signal sequence, wherein the formula of the heart region of interest signal sequence is as follows:
Xhr(t)＝[Rhr(t)；Ghr(t)；Bhr(t)]T，
wherein R ishr(t)、Ghr(t)、Bhr(t) average pixel intensities of R, G, B color channels, respectively, of the cardiac region-of-interest video image;
the formula of the background region-of-interest signal sequence is as follows:
Xbg(t)＝[Rbg(t)；Gbg(t)；Bbg(t)]T；
wherein R isbg(t)、Gbg(t)、Bbg(t) average pixel intensities of R, G, B color channels, respectively, of the background region-of-interest video image;
2) according to the combined blind source separation method, the signal sequence X of the heart interesting region is processed by using the mathematical formula SCV (t) ═ W (t)hr(t) and background region of interest signal sequence Xbg(t) performing combined blind source separation processing to obtain a unmixing matrix W, wherein W is the unmixing matrix, and SCV (t) is a source signal component vector;
further calculating to obtain a source signal component vector matrix
SCV(t)＝[SCVhr1；SCVhr2；SCVhr3；SCVbg1；SCVbg2；SCVbg3]T；
Calculating and obtaining each source signal component vector SCV of the heart region-of-interest signal sequence according to the source signal component vector matrixhr(n)(t) and the respective source signal component vectors SCV of the background region-of-interest signal sequencebg(n)(t)；
According to each source signal component vector SCV of the heart region-of-interest signal sequence obtained by calculationhr(n)(t) and the respective source signal component vectors SCV of the background region-of-interest signal sequencebg(n)(t) correlation coefficient between the two, the vector SCV of source signal component of the heart region-of-interest signal sequence corresponding to the maximum correlation coefficienthr(m)(t) obtaining SCV after setting to zeronew(t), then according to scv (t), W X (t), a cardiac region-of-interest signal sequence and a background region-of-interest signal sequence X with the common environmental noise removed can be obtainednew(t)＝W\SCVnew(t)；
3) According to a two-color reflection model, the heart interesting region time signal sequence X obtained in the step 2) after the environmental noise is removednew(t)[1∶3]X is a linear combination of illumination variation component, specular reflection component and diffuse reflection component, and is normalized in time domainnew(t) projection onto a plane P ═ P orthogonal to the illumination variation component1；p2]TThereby obtaining a heart interested region time signal sequence X only containing a specular reflection component and a diffuse reflection componentrflct(t)＝P*Xnew(t)；
4) For the heart interested region time signal sequence X only containing the specular reflection component and the diffuse reflection component obtained in the step 3)rflct(t) conducting an independent component analysis process, namely Xrflct(t) ═ a × s (t), where a is the mixing matrix and s (t) is the independent component of the source signal, s (t) ═ s (t) can be obtained1(t)；s2(t)]TCalculate [ s ]1(t)；s2(t)]TSignal sequence X of interested region of hearthr(t) in Ghr(t) screening out the independent component s (t) with the strongest correlation with the green channel signal of the original signal;
5) processing s (t) obtained in the step 4) by adopting an ensemble empirical mode decomposition method to obtain s (t) ∑ imf (t) + r (t), wherein imf (t) is an intrinsic mode function of each frequency band, r (t) is a residual error term, screening imf (t), selecting imf (t) with the highest matching degree with the heart rate distribution range of the aquatic animals on a frequency domain for sliding average, calculating the average value of the continuous n terms before and after the sliding average to replace the current value in imf (t), and recording the result as HR (t);
6) outputting directly as a cardiac waveform map according to the HR (t) obtained in step 5); or further, converting the HR (t) into the frequency domain by Fourier transform from the frequency domain to obtain the frequency value f with the maximum amplitude from the frequency spectrummaxThen the heart rate HR is fmax60, the letter X; adopting peak value detection method in time domain to obtain HR (t) with a wave crests and b wave troughs, and determining heart rate
Where t is the video duration in seconds.
9. The method for detecting the heart rate of the aquatic animal based on the video image processing as claimed in claim 8, wherein in step 1), the heart region of interest and the background region of interest in the video image are respectively selected in a manually selected manner in the 1 st frame of video image, and then the tracking and the keeping of the region of interest are completed in the subsequent t-1 th frame of video image by using the target tracking algorithm;
in the step 2), the joint blind source separation method is a joint blind source separation framework realized based on Gaussian independent vector analysis.
10. A computer-readable storage medium, characterized in that: the storage medium having stored therein at least one instruction, at least one program, code set or set of instructions which is loaded by a processor and which carries out a method of detecting heart rate in an aquatic animal based on video image processing according to any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210023274.7A CN114431849B (en) | 2022-01-10 | 2022-01-10 | Aquatic animal heart rate detection method based on video image processing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210023274.7A CN114431849B (en) | 2022-01-10 | 2022-01-10 | Aquatic animal heart rate detection method based on video image processing |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114431849A true CN114431849A (en) | 2022-05-06 |
| CN114431849B CN114431849B (en) | 2023-08-11 |
Family
ID=81367553
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210023274.7A Active CN114431849B (en) | 2022-01-10 | 2022-01-10 | Aquatic animal heart rate detection method based on video image processing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114431849B (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140206965A1 (en) * | 2011-08-26 | 2014-07-24 | Koninklijke Philips N.V. | Distortion reduced signal detection |
| CN105678780A (en) * | 2016-01-14 | 2016-06-15 | 合肥工业大学智能制造技术研究院 | Video heart rate detection method removing interference of ambient light variation |
| EP3459446A1 (en) * | 2017-09-22 | 2019-03-27 | Leibniz-Institut für Nutztierbiologie | Method for identifying a farm animal having an impairment of regulative capacity in response to metabolic stress |
| CN110269600A (en) * | 2019-08-06 | 2019-09-24 | 合肥工业大学 | Based on polynary empirical mode decomposition and the contactless video heart rate detection method for combining blind source separating |
| CN110276271A (en) * | 2019-05-30 | 2019-09-24 | 福建工程学院 | Merge the non-contact heart rate estimation technique of IPPG and depth information anti-noise jamming |
| CN110384491A (en) * | 2019-08-21 | 2019-10-29 | 河南科技大学 | A kind of heart rate detection method based on common camera |
| CN110680317A (en) * | 2019-10-15 | 2020-01-14 | 中国科学技术大学 | High-density surface electromyographic signal denoising method based on independent vector analysis |
| CN111280098A (en) * | 2020-03-17 | 2020-06-16 | 中国海洋大学 | Rapid determination method for temperature resistance character index ABT of chlamys farreri |
| CN112043257A (en) * | 2020-09-18 | 2020-12-08 | 合肥工业大学 | Non-contact video heart rate detection method for motion robustness |
| CN112043254A (en) * | 2020-08-12 | 2020-12-08 | 厦门大学 | Prawn heart rate detection method and system based on video image |
-
2022
- 2022-01-10 CN CN202210023274.7A patent/CN114431849B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140206965A1 (en) * | 2011-08-26 | 2014-07-24 | Koninklijke Philips N.V. | Distortion reduced signal detection |
| CN105678780A (en) * | 2016-01-14 | 2016-06-15 | 合肥工业大学智能制造技术研究院 | Video heart rate detection method removing interference of ambient light variation |
| EP3459446A1 (en) * | 2017-09-22 | 2019-03-27 | Leibniz-Institut für Nutztierbiologie | Method for identifying a farm animal having an impairment of regulative capacity in response to metabolic stress |
| CN110276271A (en) * | 2019-05-30 | 2019-09-24 | 福建工程学院 | Merge the non-contact heart rate estimation technique of IPPG and depth information anti-noise jamming |
| CN110269600A (en) * | 2019-08-06 | 2019-09-24 | 合肥工业大学 | Based on polynary empirical mode decomposition and the contactless video heart rate detection method for combining blind source separating |
| CN110384491A (en) * | 2019-08-21 | 2019-10-29 | 河南科技大学 | A kind of heart rate detection method based on common camera |
| CN110680317A (en) * | 2019-10-15 | 2020-01-14 | 中国科学技术大学 | High-density surface electromyographic signal denoising method based on independent vector analysis |
| CN111280098A (en) * | 2020-03-17 | 2020-06-16 | 中国海洋大学 | Rapid determination method for temperature resistance character index ABT of chlamys farreri |
| CN112043254A (en) * | 2020-08-12 | 2020-12-08 | 厦门大学 | Prawn heart rate detection method and system based on video image |
| CN112043257A (en) * | 2020-09-18 | 2020-12-08 | 合肥工业大学 | Non-contact video heart rate detection method for motion robustness |
Non-Patent Citations (3)
| Title |
|---|
| JUAN CHENG，等: "Illumination Variation-Resistant Video-Based Heart Rate Measurement Using Joint Blind Source Separation and Ensemble Empirical Mode Decomposition", 《IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS》, vol. 21, no. 5, pages 1423 - 1429 * |
| 刘正成: "基于人脸视频的非接触式心率检测方法研究", 《中国优秀硕士学位论文全文数据库 （医药卫生科技辑）》, pages 7 - 8 * |
| 秦睿星，陈兆学: "人脸运动状态下的非接触式心率稳定测量算法", 《光学技术》, vol. 47, no. 1 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114431849B (en) | 2023-08-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110269600B (en) | Non-contact video heart rate detection method based on multivariate empirical mode decomposition and combined blind source separation | |
| CN110384491A (en) | A kind of heart rate detection method based on common camera | |
| CN112949349B (en) | Method and system for displaying pulse condition waveform in real time based on face video | |
| CN111243739A (en) | Anti-interference physiological parameter telemetering method and system | |
| Paskaranandavadivel et al. | Improved signal processing techniques for the analysis of high resolution serosal slow wave activity in the stomach | |
| WO2006100685A2 (en) | Optical sensor device and image processing unit for measuring chemical concentrations, chemical saturations and biophysical parameters | |
| CN111387959A (en) | Non-contact physiological parameter detection method based on IPPG | |
| Wedekind et al. | Automated identification of cardiac signals after blind source separation for camera-based photoplethysmography | |
| CN112294282A (en) | Self-calibration method of emotion detection device based on RPPG | |
| CN114387479A (en) | Non-contact heart rate measurement method and system based on face video | |
| CN115153473B (en) | Non-contact heart rate detection method based on multivariate singular spectrum analysis | |
| CN114037626A (en) | Blood vessel imaging method, device, equipment and storage medium | |
| CN113591769B (en) | Non-contact heart rate detection method based on photoplethysmography | |
| Boucheham et al. | Piecewise linear correction of ECG baseline wander: a curve simplification approach | |
| CN110584638A (en) | Non-contact heart rate measurement method based on CMOR wavelet | |
| CN112001862B (en) | Non-contact type apparent heart rate detection method for eliminating motion noise of video heart impact signal | |
| KR20100027460A (en) | Apparatus and method for finding characteristic parameters in bio-signal using multiple moving windows | |
| CN110491504B (en) | Method for acquiring medical index data of heart sound signal | |
| Zhong et al. | Maternal ECG removal using short time Fourier transform and convolutional auto-encoder | |
| CN114431849B (en) | Aquatic animal heart rate detection method based on video image processing | |
| Deng et al. | Non-invasive methods for heart rate measurement in fish based on photoplethysmography | |
| CN115381444A (en) | Rapid detection method for blood oxygen saturation | |
| CN114271800A (en) | Non-invasive continuous blood pressure monitoring method in office environment and application | |
| CN114209299B (en) | IPPG technology-based human physiological parameter detection channel selection method | |
| Umadi et al. | Heart Abnormality Detection Technique using PPG Signal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |