CN110826503B - Closed pipeline human body detection method and system based on multi-sensor information fusion - Google Patents
Closed pipeline human body detection method and system based on multi-sensor information fusion Download PDFInfo
- Publication number
- CN110826503B CN110826503B CN201911089339.2A CN201911089339A CN110826503B CN 110826503 B CN110826503 B CN 110826503B CN 201911089339 A CN201911089339 A CN 201911089339A CN 110826503 B CN110826503 B CN 110826503B
- Authority
- CN
- China
- Prior art keywords
- human body
- body detection
- pixel
- infrared
- capacitance
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01V—GEOPHYSICS; GRAVITATIONAL MEASUREMENTS; DETECTING MASSES OR OBJECTS; TAGS
- G01V11/00—Prospecting or detecting by methods combining techniques covered by two or more of main groups G01V1/00 - G01V9/00
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/30—Noise filtering
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Life Sciences & Earth Sciences (AREA)
- Geophysics (AREA)
- Human Computer Interaction (AREA)
- Geophysics And Detection Of Objects (AREA)
- Radiation Pyrometers (AREA)
Abstract
The invention discloses a closed pipeline human body detection method and a system based on multi-sensor information fusion, which belong to the technical field of detection, and the closed pipeline human body detection method based on the multi-sensor information fusion comprises the following steps: s1, acquiring original parameters, specifically: acquiring an infrared image in the closed pipeline; acquiring capacitance data in the closed pipeline; s2, analyzing the original parameters to further prejudge whether people exist in the pipeline, specifically: obtaining a first human body detection result through analysis of the infrared image; obtaining a second human body detection result through analyzing the capacitance data; and S3, adopting data-level and decision-level information fusion, taking the first human body detection result and the second human body detection result as fusion objects, and finally making a judgment result whether the pipeline is occupied or not. The invention compares the basic probability distribution of the two sensors and detects whether the human body detection system is abnormal or not according to the correlation of the basic probability distribution of the two sensors.
Description
Technical Field
The invention relates to the technical field of detection, in particular to a closed pipeline human body detection method and system based on multi-sensor information fusion.
Background
The human body detection means that after an accident or a natural disaster occurs, physiological information of a human body such as heart rate, pulse, blood pressure and respiration or other physical information capable of distinguishing the human body from the background is detected by a certain method by using a certain device, and the information is subjected to signal processing and characteristic extraction to finally generate an electric signal capable of representing the physiological activity of the human body. According to the principle used, the human body detecting devices can be classified into the following categories: radar type human body detector, acoustic human body detector and infrared human body detector.
(1) Radar type human body detection technology
Most of radar-based body detection techniques are based on the doppler effect and on the difference in the absorption characteristics of electromagnetic waves by different dielectric constant media. The vital signs of the human body can be extracted by using the energy attenuation and the phase change of the electromagnetic wave penetrating through the human body. A multi-input multi-output radar human body monitoring technology is provided by Naoki Honma in 2018, and the positioning of a human body can be well completed.
The principle of radar type human body detection technology determines that the method can generate serious electromagnetic wave multipath effect in the environment with limited space, thereby causing the detection reliability to be reduced.
(2) Acoustic human body detection technology
The acoustic human body detection mainly comprises the steps of collecting sound signals such as distress sounds or knocking sounds, shouting sounds and moving sounds emitted by people in the ruins or buried objects by using acoustic sensitive elements, and obtaining characteristic signals of human bodies from surrounding noises through signal processing and characteristic extraction. The acoustic human body detection technology is different from the radar human body detection technology in that the acoustic human body detection technology is passive and has short detection distance. When the probe is applied and rescued, the sound sensitive probe needs to be deeply inserted into the gap of the ruins, the monitoring speed is low, and the signal processing process is complex.
The sound wave vibration human body detector has high requirements on rescue environment, and in a rescue site after a disaster with a complex environment, the sound wave signal of a buried human body is usually submerged by background noise. The problem of extracting weak useful signals from the chaotic signals is to be solved, so that the sound wave human body detection is slow in development.
(3) Infrared human body detection technology
The infrared human body detection adopts an infrared imaging mode. The infrared imaging is to convert the infrared heat radiation energy received by the sensitive element into an electric signal, and the electric signal is amplified, shaped and AD converted into a digital image signal, and the infrared human body detection is to divide the infrared imaging image and extract the human body characteristics.
The infrared detection technology is used for measuring the temperature of a human body, and when the ambient temperature is close to the temperature of the human body, the infrared detection technology cannot judge the existence of the human body.
In summary, the existing human body detection method is difficult to achieve an ideal detection effect in a complex environment due to its own limitations. In a complex environment, noise is generated, illumination brightness is different, electromagnetic interference difference is large, temperature change is large, and the single-type sensor is difficult to achieve universal adaptability of various environments. Based on the method, the method combining capacitance and infrared detection is provided for the human body detection problem at the closed conveying pipeline, the detection accuracy of the system is reduced by using a sensor data fusion algorithm, and the defects of a single sensor human body detection technology can be overcome.
Disclosure of Invention
The invention aims to provide a closed pipeline human body detection method and system based on multi-sensor information fusion, which are used for detecting a human body of a closed pipeline by using a data fusion method of infrared images and capacitance detection, so that the reliability and the accuracy of the human body detection of the closed pipeline are improved.
The invention adopts the following technical scheme:
the invention aims to provide a closed pipeline human body detection method based on multi-sensor information fusion, which comprises the following steps of:
s1, obtaining original parameters, specifically:
acquiring an infrared image in the closed pipeline;
acquiring capacitance data in the closed pipeline;
s2, analyzing the original parameters to further prejudge whether people exist in the pipeline, specifically comprising the following steps:
obtaining a first human body detection result through analysis of the infrared image;
obtaining a second human body detection result through analyzing the capacitance data;
and S3, adopting data-level and decision-level information fusion, taking the first human body detection result and the second human body detection result as fusion objects, and finally making a judgment result whether the pipeline is occupied or not.
Further, the analyzing the infrared image specifically includes:
firstly, signal amplification and calibration are carried out on infrared images to generate preliminary infrared thermal imaging images, and blind pixel compensation is carried out on a plurality of continuous infrared image detection positions at random;
then carrying out image enhancement and denoising treatment;
and finally, segmenting the human body contour in the infrared image by using an image segmentation method for subsequent judgment, and judging whether a person falls into the conveying pipeline according to the size of the segmented target.
Further, before blind pixel compensation, blind pixel detection is firstly carried out, and the specific detection process is as follows:
A. under the background of uniform temperature, respectively solving a fitted curved surface for 100 infrared images, subtracting the fitted curved surface from the original infrared image, and judging as a blind pixel if the absolute value is greater than 2;
B. counting the distribution of the 100 blind pixels, and when the frequency of the pixels corresponding to one pixel element is judged to be more than 30 percent, determining that the pixel element is a blind pixel or a dead pixel, and performing neighborhood compensation on the pixel element in the subsequent infrared image;
C. the detection of the random blind pixels needs a large number of infrared images, 30 infrared images are continuously shot for a static background target to form a group, the variation coefficient of each group of infrared images is respectively calculated, when the variation coefficient of one pixel is larger than 0.1, the pixel is regarded as the random blind pixel, and neighborhood blind pixel compensation is carried out on the random blind pixel in the subsequent infrared images.
Furthermore, a variation coefficient is introduced on the basis of surface fitting to serve as a discrimination basis for discriminating whether a certain pixel is a blind pixel, and a calculation formula of the variation coefficient is as follows:
wherein, σ is the standard deviation of the N pictures at different pixel points, and μ is the average value of the N pictures at different pixel points; when the coefficient of variation of a certain pixel is greater than 0.1, the pixel value is unreliable, and blind pixel compensation is required; for a plurality of consecutive infrared images, formula (2) holds:
in the formula, G (x, y) represents a pixel value at coordinates (x, y) in one original infrared image, and N represents the number of pictures.
Furthermore, the denoising processing adopts a neighborhood smooth filtering method, and the image segmentation method is an OTSU segmentation method; and in the OTSU segmentation process, adding prior experience meeting the detection environment, forcibly setting the segmentation threshold as the ambient temperature when the threshold obtained by the OTSU segmentation algorithm is less than the ambient temperature, and performing image segmentation by using the threshold determined by the OTSU algorithm when the threshold is higher than the ambient temperature.
Further, after image segmentation processing, a binary image with infrared human body characteristics is obtained, the sum of pixel points in two areas with the largest bright spots in the binary image is taken as infrared image information of a human body, and the bright spot characteristics are calculated by using a region growing algorithm, wherein the specific calculation steps are as follows:
A. extracting a point with a pixel value of 1 in the binary image into a set F: p (x, y) =1;
B. selecting the point P with the minimum subscript value in the set i (x min ,y min ) To P i Growing in the neighborhood direction from left to top on the set F, and comparing the growth with P i Put the connected points with 1 into the set f i In, then set f i The represented area is a bright spot, and the statistical setf i The number of medium elements is recorded as the area S of the bright spot i Recording the area value S i The set F is then updated:
C. repeating step B until the set F is empty;
D. will obtain S i The largest two are added up as the infrared image information of the human body.
Further, the specific process of obtaining the second human body detection result through the analysis of the capacitance data is as follows:
firstly, carrying out temperature compensation on the dielectric constant and the capacitance threshold of a measured object according to the temperature;
then, calculating the capacitance variable according to the dielectric constant of the measured object after temperature compensation;
and finally, comparing the variable of the capacitor with a capacitance threshold value, if the variable of the capacitor is within the capacitance threshold value range, determining that no person exists, otherwise, determining that a person exists.
Further, the S3 specifically is:
it is assumed that the setting of the space,
and classifying the temperature, the infrared image and the capacitance, converting the classified temperature, the infrared image and the capacitance into an evidence space expression, and obtaining probability distribution of various results according to experimental data. For the detection system of the present invention, the assumed space of DS evidence theory is Θ = [ safety, hazard ].
The combination BPA is calculated and,
for the
Two mass functions m on Θ 1 ,m 2 The Dempster synthesis rule is as follows:
wherein: a and b are arbitrary elements in the assumed space, K is a normalization constant, and the calculation formula is as follows:
trust function
The trust function value for the Θ element a in the hypothetical space is:
m (B) is a combination BPA of the elements B in the assumed space, and the belief function value of the elements B can be obtained by the same method:
Bel(B)＝m(B)＝0.61 (11)
likelihood function
The likelihood function values for element a over the space Θ are assumed to be:
the likelihood function values for element B over the space Θ are assumed to be:
the second purpose of the invention is to provide a closed pipeline human body detection system based on multi-sensor information fusion, which at least comprises:
the original parameter obtaining module specifically includes:
the infrared image sensor is used for acquiring an infrared image in the closed pipeline;
the capacitance sensor is used for acquiring capacitance data in the closed pipeline;
a temperature sensor for acquiring temperature data in the pipeline;
the preliminary prejudgment module is used for prejudging whether people exist in the pipeline or not by analyzing the original parameters, and specifically comprises the following steps:
obtaining a first human body detection result through analysis of the infrared image;
obtaining a second human body detection result through analyzing the capacitance data;
and the information fusion module adopts data-level and decision-level information fusion, takes the first human body detection result and the second human body detection result as fusion objects, and finally makes a judgment result whether a person exists in the pipeline.
The invention has the beneficial effects that:
the invention provides a human body detection method based on the combination of infrared thermal imaging and capacitance detection, data detected by two sensors are fused at a data level and a decision level, and fused output of two human body detection results is finally completed through a D-S evidence theory. The adaptability of the detection system to the environment is increased, and the accuracy of the detection result is improved. The detection related algorithm of the overheating elements and the dead elements is mature, and the algorithm is simple and can meet the real-time requirement of a detection system; for the random blind pixels, a detection method with small algorithm time complexity and high detection precision does not exist, the invention improves the blind pixel detection algorithm of surface fitting, and introduces a variation coefficient to detect the random blind pixels. By processing a plurality of continuous infrared images, the accuracy of the original blind pixel detection algorithm based on surface fitting is improved, and meanwhile, the algorithm has the capability of detecting random blind pixels. The invention adopts an alternating current method to measure capacitance, obtains the relation between the relative dielectric constant of the medium and the temperature by microscopic analysis of the dielectric constant of the medium, and carries out temperature compensation on the measured value of the capacitance by utilizing the temperature information measured by the temperature sensor PT 1000. The method comprises the steps of converting spot information and capacitance of infrared thermal imaging into initial probability of detection reliability through setting of a threshold value and combining a curve fitting algorithm, and performing decision-level fusion on output results of the heterogeneous multi-sensor system by using a Dempster synthesis rule-based D-S evidence theory. The detection results of the two heterogeneous sensors are complemented, and the probability of the detection result error is reduced through a data fusion algorithm of the multiple sensors
Drawings
Fig. 1 is a block diagram of a multi-sensor human body detection system.
Figure 2 incorporates the results of the a priori empirical OTSU threshold segmentation.
Fig. 3 is a structural view of a human body detection capacitive sensor.
FIG. 4 illustrates a dielectric distribution and equivalent capacitance model for capacitive body sensing.
Fig. 5 is a block diagram of the whole system hardware structure.
FIG. 6 shows the infrared image and the blind pixel monitoring result when the background temperature is not uniform.
FIG. 7 decision-level data fusion of infrared and capacitive detection results.
Detailed Description
The invention is described in detail below with reference to the accompanying drawings:
with reference to fig. 1 to 7, a human body detection method based on the combination of infrared thermal imaging and capacitance detection includes the following steps:
step 1: and (5) establishing a human body detection model. The invention is divided into four parts: infrared vision detection, a capacitance sensor, an ambient temperature sensor and multi-sensor information fusion. The optimal application environments of different types of sensors are different, the sensitivity to different types of interference in the environment is also different, when the humidity is higher, the sensitivity of the capacitive sensor is obviously reduced due to the relative dielectric constant of water being 80, but the detection result of the capacitive sensor is still reliable after temperature compensation correction within a certain temperature range; the infrared sensor is not sensitive to humidity, and the temperature is a sensitive variable of the infrared sensor. The structure of the multi-sensor human body detection system is shown in fig. 1.
Step 2: an infrared thermal imaging human body detection model. After the infrared sensitive device receives the infrared radiation, signal amplification and calibration are carried out (dead elements and overheating elements are removed); generating a preliminary infrared thermal imaging image, and performing blind pixel compensation on a plurality of continuous infrared image detection positions to obtain an infrared image which accurately reflects the temperature distribution of a target; and carrying out image enhancement and denoising treatment on the infrared image obtained in the last step, and further segmenting the human body contour in the infrared image by using an image segmentation method for subsequent judgment. And judging whether a person falls into the conveying pipeline according to the size of the divided target.
And step 3: and (3) an improved infrared blind pixel detection algorithm. The dead elements and the overheated elements are static for detection because the response of the dead elements and the overheated elements to different temperatures is basically unchanged, and the dead elements and the overheated elements are detected by applying an averaging method. The invention introduces the coefficient of variation on the basis of surface fitting as a basis for distinguishing whether a certain pixel is a blind pixel or not. The coefficient of variation is calculated as:
in the formula (1), σ is a standard deviation of the N pictures at different pixel points, and μ is a mean value of the N pictures at different pixel points. When the coefficient of variation of a certain pixel is greater than 0.1, the pixel value is not reliable, and blind pixel compensation is required. For a plurality of consecutive infrared images, formula (2) holds:
in the formula, G (x, y) represents a pixel value at a coordinate (x, y) in an original infrared image, and N represents the number of pictures.
The overall process of the blind pixel detection algorithm established by the invention is as follows:
(1) Under the background of uniform temperature, respectively solving a fitted curved surface for 100 infrared images, subtracting the fitted curved surface from the original infrared image, and judging as a blind pixel if the absolute value is greater than 2.
(2) Counting the 100 blind pixel distributions obtained in the step (1), and judging the pixel corresponding to one pixel as the frequency of the blind pixel
If the rate is more than 30%, the pixel can be determined to be a blind pixel or a dead pixel, and neighborhood compensation is required in the subsequent infrared image.
(3) The detection of the random blind pixels needs a large number of infrared images, 30 infrared images are continuously shot for a static background target to form a group, the variation coefficient of each group of infrared images is respectively calculated, when the variation coefficient of one pixel is more than 0.1, the pixel can be regarded as the random blind pixel, and neighborhood blind pixel compensation is carried out on the random blind pixel in the subsequent infrared images.
And 4, step 4: when people exist, the gray level of the background image is changed greatly, the noise is serious, and the method adopts neighborhood smoothing filtering to perform noise reduction processing on the image. The pixel distribution of the infrared image background after smooth filtering becomes more uniform, and the noise of the original image is removed. The method comprises the steps of performing threshold segmentation on a denoised infrared image to extract a human body contour in the image, performing threshold segmentation on the image through an OTSU algorithm, wherein the human body contour is completely segmented, when no person exists or a human body enters a detection area to be small, the OTSU segmentation can generate serious misjudgment, prior experience meeting a detection environment needs to be added, when a threshold obtained by the OTSU algorithm is smaller than an environment temperature, the segmentation threshold is forcibly set to the environment temperature, and when the threshold is higher than the environment temperature, the image is segmented by using the threshold determined by the OTSU algorithm. The threshold segmentation result added with the prior experience of the environmental temperature can obtain a more ideal result when no one is available. The results of OTSU thresholding with a priori experience are shown in figure 2.
And 5: and obtaining a binary image with infrared human body characteristics after image enhancement and segmentation processing. And when the sum of pixel points in the two areas with the largest bright spots in the binary image is taken as the infrared image information of the human body, higher human body detection precision can be ensured. And calculating the bright spot characteristics by using a region growing algorithm, wherein the specific calculation steps are as follows:
(1) Extracting a point with a pixel value of 1 in the binary image into a set F: p (x, y) =1;
(2) Selecting the point P with the minimum subscript value in the set i (x min ,y min ) To P i Growing in the neighborhood direction from left to top on the set F, and comparing the growth with P i Put the connected points with 1 into the set f i In (1), then set f i The represented area is a bright spot, and the statistical set f i The number of medium elements is recorded as the area S of the bright spot i Recording the area value S i The set F is then updated:
(3) Repeating the step 2 until the set F is empty;
(4) Will obtain S i The largest two are added up as the infrared image information of the human body.
The FOV of the infrared sensitive array element selected by the invention is 110 degrees x75 degrees, the infrared element is arranged at the position 35cm above the reversed loader and can cover the range of 100 x 54cm2, the number of infrared pixel points corresponding to the head (according to 20 x 20cm 2) of a person in the area is about 60, and an alarm threshold value of 50 bright spots is selected for improving the detection precision.
And 6: the capacitance sensor used in the invention applies the principle of variable dielectric constant, capacitance plates with certain sizes are arranged on the upper side and the lower side of the conveying pipeline, the change of the medium in the conveying pipeline can reflect the change of capacitance, and the relative dielectric constant of the medium between the plates can be obviously changed suddenly when a person mistakenly enters the conveying pipeline. The change condition of the medium between the two plates of the capacitor can be measured by detecting the charge quantity of the plates. The structure of the human body detection capacitive sensor is shown in fig. 3. And eliminating the edge effect of the capacitor by adopting a grounding ring technology. Namely, a circle of metal plate is added around the positive plate of the detection capacitor and is connected with the negative electrode of the detection capacitor, so that the scattered electric field at the edge of the capacitor is eliminated. In the human body detection model of the conveying pipeline, three media, namely human body, air and coal, are arranged between capacitor plates, and the distribution of the three media accords with a series-parallel mixed medium model. The medium distribution and equivalent capacitance model of the capacitance human body detection are shown in fig. 4. The left diagram in fig. 4 is the equivalent diagram of the medium distribution of capacitance human body detection, and the human body is assumed to be distributed at the edge of the closed conveying pipeline for simplifying the model. The right graph in the figure is series-parallel connection of equivalent capacitors with medium distribution, c 1 、c 2 Representing the capacitance formed by the air in the closed conveying pipe as a medium, c 3 Representing the capacitance formed by the body falling into the conveying pipe as a capacitive medium, c 4 Is the capacitance formed by the coal medium. From the series-parallel theorem of capacitance, we can get:
c′＝c 2 +c 3 (4)
in the formula (3), c represents the total capacitance of the detection capacitor. c. C 1 The capacitance value of the model with the uppermost air as the medium, c 2 A capacitance value of the medium air in the middle layer, c 3 Capacitance value when human body is used as medium, c 4 The capacitance value of coal as a medium. The specific calculation for each capacitance is:
l = L in formula (5) 1 +L 2 +L 3 Is the length of the electrode, W is the width of the electrode, ε 0 ＝8.85*10 -12 F/m ε 1 Is the relative dielectric constant of air, epsilon 2 Is the relative dielectric constant, epsilon, of the human body 3 Is the relative dielectric constant of coal.
The equivalent relative permittivity of the mixed medium is:
the total length of the detection polar plate is set as the length L =1000mm of the conveying pipeline, and the width of the detection capacitor polar plate is set as W, then the detection capacitance value can be obtained by the calculation formula of the capacitor:
assuming that the equivalent dielectric constant of the mixed medium between the capacitor plates changes by Δ ∈, the capacitance becomes:
the sensitivity of a variable dielectric type capacitor is defined as: this variation in capacitance value due to the change in unit dielectric constant. The sensitivity D of the capacitor is detected c Comprises the following steps:
the sensitivity of the detection capacitor is increased along with the increase of the width of the detection polar plate, the narrow polar plate can cause the edge of a signal to be too steep, and the metal capacitor polar plate with the polar plate width of 10cm is finally adopted in the invention.
The environmental temperature range of the multi-sensor fused human body detection application designed by the invention is generally as follows: and fitting the corresponding relation between the temperature in the temperature range and the relative dielectric constants of the human body and the coal body between 20 ℃ and 50 ℃:
ε R ＝-0.312t+86.216 (10)
ε m ＝-0.0475t+5.2 (11)
in the formula of R Is a relative dielectric constant of human,. Epsilon m Is the relative dielectric constant of the coal, and t is the ambient temperature in degrees Celsius.
In combination with the above analysis, the dimensions of the detection electrode are L =1000mm, w =100mm, L2=300mm, L1= l3=350mm, d1=100mm, d2=300mm, and d3=400mm. The relative dielectric constant of air is epsilon 1 And =1. The measured capacitance value must be compensated by temperature to reflect the change of dielectric constant of the measured object more objectively, and the setting of the capacitance threshold value phi also needs to be set according to the temperature.
And 7: the invention adopts data-level and decision-level information fusion on data fusion, and takes the detection results of the two methods as fusion objects. The capacitance measurement is temperature dependent and the capacitance sensor is compensated by measuring ambient temperature information through the PT 1000. The invention relates to human body detection based on heterogeneous multi-source sensors, which adopts the comprehensive detection result output of an infrared and capacitance sensor based on the Dempster synthesis rule D-S evidence theory at a decision level through the research on a decision-level multi-information fusion algorithm.
(1) Setting of hypothetical spaces
And classifying the detection results of different sensors, converting the detection results into evidence space expressions, and obtaining probability distribution of various results according to experimental data. For the detection system of the present invention, the assumed space of DS evidence theory is Θ = [ safety, hazard ].
(2) Calculating the combination BPA
For the
Two mass functions m on Θ 1 ,m 2 The Dempster synthesis rule is:
in the formula (3.14), a and b are arbitrary elements in the assumed space. K is a normalization constant, and the calculation formula is as follows (3.15):
(3) Trust function
The trust function value for the Θ element a in the hypothetical space is:
m (B) is the combination BPA of the elements B in the hypothetical space. The same reasoning can be used to obtain the trust function value of element B as:
Bel(B)＝m(B)＝0.61 (15)
(4) Likelihood function
The likelihood function values for element a over the space Θ are assumed to be:
the likelihood function values for element B over space Θ are assumed to be:
and 8: and designing an information acquisition circuit. The infrared thermal imaging-based human body detection and the capacitance-based human body detection are arranged as independent modules. The whole is divided into 4 modules: the device comprises a power supply module, an infrared imaging module, a capacitance detection module and an information fusion and output module. The overall structure block diagram of the system hardware is shown in fig. 5.
The AD-DC module reduces AC127V alternating current to DC12V direct current, and the DC12V direct current is sent to each subsystem through the independent DC-DC isolation module after overvoltage and overcurrent protection. Each subsystem further stabilizes the DC12V to DC5V or DC3.3V according to the requirement; the thermal infrared imaging acquisition part is completed by an MLX90640 infrared focal plane array sensor, a main control chip STM32F401 completes the related operation and processing of the infrared image, and the processed decision result is transmitted to an information fusion and output module through a UART; the capacitance detection module adopts an integrated capacitance measurement chip PCAP02AE to complete capacitance collection and environment temperature collection, transmits the capacitance collection and the environment temperature collection to an STM32F030F4 through an SPI bus to complete capacitance temperature compensation and digital filtering processing, and transmits compensated capacitance information to an information fusion and output module through a UART; and the information fusion and output module receives the judgment result of the thermal infrared imaging, completes a data fusion algorithm based on a D-S evidence theory, displays the final decision information and the equipment running state through an LCD, and sends different action instructions to the acousto-optic alarm module and the relay according to the judgment result.
And step 9: and (3) performing blind pixel detection of a surface fitting method on 100 infrared pictures, representing the times of judging the pixels as blind pixels by different colors, and selecting the pixels of which the times of judging the pixels as the blind pixels are more than 30 to obtain the blind pixel positions of dead pixels or overheated pixels needing blind pixel compensation. During the process that a trolley of the conveying pipeline passes through a detection area, infrared images are collected and divided into images of people carried by the trolley and images of coal carried by the trolley. The method comprises the steps of conducting self-adaptive threshold segmentation, finding the largest bright spots in images through region growing, collecting a large number of infrared images when coal is carried by a small vehicle at different environmental temperatures by using the bright spots as human body distinguishing information according to an infrared human body detection model, and displaying results, wherein the probability that the area of the bright spots of the infrared images is larger than 50 when no person exists is increased along with the increase of the environmental temperature, and the probability that the area of a light spot is larger than or equal to 50 when the environmental temperature is 26 ℃. The misjudgment rate of the infrared image on human body detection increases with the rise of the environmental temperature. The infrared image and the blind pixel monitoring result when the background temperature is not uniform are shown in fig. 6. The situation of false alarm and false alarm of dangerous cases can occur when a single sensor deals with the adverse effect of the environment. And performing decision-level fusion on the acquired infrared images and capacitance data through a D-S evidence theory, so as to improve the accuracy of the detection result. D-S evidence theory fusion is carried out on the infrared method danger occurrence judgment result and the capacitance detector danger occurrence probability judgment result data, and the fused danger occurrence probability result is obtained and shown in figure 7. When a person exists, the result of experimental fusion is well consistent with the infrared and capacitance independent detection results; in the infrared detection method, 3 points are misjudged, in the capacitance detection method, 12 points are misjudged, and only 1 point is misjudged after the data fusion algorithm, so that the misjudgment rate is obviously reduced. The infrared and capacitance detection result decision-level data fusion is shown in fig. 7.
The human body detection method comprises hardware structures of infrared image acquisition, capacitance measurement and the like of human body detection and data fusion human body detection to form a data fusion human body detection system based on the infrared image and the capacitance detection; establishing a human body detection model based on infrared thermal imaging, improving an infrared sensor array blind pixel detection algorithm based on a variation coefficient aiming at the problem that an infrared imaging array has random blind pixels, and compensating blind pixel pixels by using a neighborhood mean value method; and processing the infrared image through image enhancement and improved OTSU threshold segmentation to obtain human body thermal infrared image characteristics. And establishing a variable dielectric constant type capacitance human body detection model, and analyzing the relation between the environment temperature and the dielectric constant from a microscopic angle. The change characteristic of the capacitance when the human body falls into the closed pipeline is obtained. Temperature information is extracted through the temperature sensor to carry out temperature compensation on the capacitor, and fusion of multi-source heterogeneous sensor information is completed on a data level. And (4) performing decision-level data fusion of heterogeneous multi-sensor information by applying a D-S evidence theory to complete information complementation of the infrared sensor and the capacitive sensor. A set of closed conveying pipeline human body detection data acquisition system based on infrared imaging and capacitance detection is built, and the design of modules such as an intrinsic safety power supply, infrared thermal image acquisition, capacitance detection, information fusion and output is completed. When a single sensor detects a human body, the interference caused by other factors such as environment on the result can seriously influence the accuracy of human body detection; in the human body detection system with heterogeneous sensors, detection results of the two heterogeneous sensors are complemented, and the probability of detection result error is reduced through a data fusion algorithm of the multiple sensors; and comparing the basic probability distributions of the two sensors, and detecting whether the human body detection system is abnormal or not according to the correlation of the basic probability distributions of the two sensors.
It is to be understood that the above description is not intended to limit the present invention, and the present invention is not limited to the above examples, and those skilled in the art may make modifications, alterations, additions or substitutions within the spirit and scope of the present invention.
Claims (6)
1. A closed pipeline human body detection method based on multi-sensor information fusion is characterized by comprising the following steps:
s1, obtaining original parameters, specifically:
acquiring an infrared image in the closed pipeline;
acquiring capacitance data in the closed pipeline;
acquiring temperature data in a pipeline;
s2, analyzing the original parameters to further prejudge whether people exist in the pipeline, specifically:
obtaining a first human body detection result through analysis of the infrared image; the analysis of the infrared image specifically comprises: firstly, signal amplification and calibration are carried out on infrared images to generate preliminary infrared thermal imaging images, and blind pixel compensation is carried out on a plurality of continuous infrared image detection positions at random; then carrying out image enhancement and denoising treatment; finally, by using an image segmentation method, segmenting the human body outline in the infrared image for subsequent judgment, and judging whether a person falls into the conveying pipeline according to the size of the segmented target; wherein: before blind pixel compensation, blind pixel detection is firstly carried out, and the specific detection process is as follows: A. under the background of uniform temperature, respectively solving a fitted curved surface for 100 infrared images, subtracting the fitted curved surface from the original infrared image, and judging as a blind pixel if the absolute value is greater than 2; B. counting the distribution of the 100 blind pixels, and when the frequency of the pixels corresponding to one pixel element is judged to be more than 30 percent, determining that the pixel element is a blind pixel or a dead pixel, and performing neighborhood compensation on the pixel element in the subsequent infrared image; C. the detection of the random blind pixels needs a large number of infrared images, 30 infrared images are continuously shot for a static background target to form a group, the variation coefficient of each group of infrared images is respectively calculated, when the variation coefficient of one pixel is more than 0.1, the pixel is regarded as the random blind pixel, and neighborhood blind pixel compensation is carried out on the random blind pixel in the subsequent infrared images;
obtaining a second human body detection result through analyzing the capacitance data;
s3, adopting data-level and decision-level information fusion, taking the first human body detection result and the second human body detection result as fusion objects, and finally making a judgment result whether a person exists in the pipeline; the method comprises the following specific steps:
assuming the setting of the space, it is possible to,
classifying the temperature, the infrared image and the capacitance, converting the classified temperature, the infrared image and the capacitance into evidence space expressions, obtaining probability distribution of various results according to experimental data, and ensuring that the assumed space of DS evidence theory is theta = [ safety, danger ];
the combination BPA is calculated and,
for the
Two mass functions m on Θ 1 ,m 2 The Dempster synthesis rule is:
wherein: a and b are arbitrary elements in the assumed space, K is a normalization constant, and the calculation formula is as follows:
the trust function value at the hypothetical space Θ element a is:
m (B) is a combination BPA of the elements B in the assumed space, and the trust function value of the element B obtained by the same method is as follows:
Bel(B)＝m(B)＝0.61(11)
the likelihood function values for element a over the space Θ are assumed to be:
the likelihood function values for element B over the space Θ are assumed to be:
2. the human body detection method of the closed pipeline based on the multi-sensor information fusion as recited in claim 1,
on the basis of surface fitting, a variation coefficient is introduced as a discrimination basis for discriminating whether a certain pixel is a blind pixel, and a calculation formula of the variation coefficient is as follows:
wherein, σ is the standard deviation of the N pictures at different pixel points, and μ is the average value of the N pictures at different pixel points; when the coefficient of variation of a certain pixel is greater than 0.1, the pixel value is unreliable and blind pixel compensation is needed; for a plurality of consecutive infrared images, formula (2) holds:
in the formula, G (x, y) represents a pixel value at coordinates (x, y) in one original infrared image, and N represents the number of pictures.
3. The closed pipeline human body detection method based on multi-sensor information fusion as claimed in claim 2, wherein the denoising process adopts a neighborhood smoothing filter method, and the image segmentation method is an OTSU segmentation method; and in the OTSU segmentation process, adding prior experience meeting the detection environment, forcibly setting the segmentation threshold as the ambient temperature when the threshold obtained by the OTSU segmentation algorithm is less than the ambient temperature, and performing image segmentation by using the threshold determined by the OTSU algorithm when the threshold is higher than the ambient temperature.
4. The closed pipeline human body detection method based on multi-sensor information fusion as claimed in claim 3,
after image segmentation processing, obtaining a binary image with infrared human body characteristics, taking the sum of pixel points in two regions with the largest bright spots in the binary image as the infrared image information of a human body, and calculating the bright spot characteristics by using a region growing algorithm, wherein the specific calculation steps are as follows:
A. extracting a point with a pixel value of 1 in the binary image into a set F: p (x, y) =1;
B. selecting the point P with the minimum subscript value in the set i (x min ,y min ) To P is to P i Growing in the neighborhood direction from left to top on the set F, and comparing the growth with P i Put the connected points with 1 into the set f i In, then set f i The represented area is a bright spot, and the statistical set f i The number of the medium elements is recorded as the area S of the bright spot i Record the area value S i The set F is then updated:
C. repeating the step B until the set F is empty;
D. will obtain S i The largest two are added up as the infrared image information of the human body.
5. The closed pipeline human body detection method based on multi-sensor information fusion of claim 1,
the specific process of obtaining the second human body detection result through analyzing the capacitance data is as follows:
firstly, temperature compensation is carried out on the dielectric constant and the capacitance threshold of a measured object according to the temperature;
then calculating the capacitance change amount according to the dielectric constant of the measured object after temperature compensation;
and finally comparing the capacitance change quantity with a capacitance threshold value, if the capacitance change quantity is within the capacitance threshold value range, determining that no person exists, otherwise, determining that a person exists.
6. A closed pipeline human body detection system based on multi-sensor information fusion is characterized by being used for executing the closed pipeline human body detection method based on multi-sensor information fusion of any one of claims 1-5, and at least comprising the following steps:
the original parameter acquisition module specifically comprises:
the infrared image sensor is used for acquiring an infrared image in the closed pipeline;
the capacitance sensor is used for acquiring capacitance data in the closed pipeline;
the temperature sensor is used for acquiring temperature data in the pipeline;
the preliminary prejudgment module is used for prejudging whether people exist in the pipeline or not by analyzing the original parameters, and specifically comprises the following steps:
obtaining a first human body detection result through analysis of the infrared image;
obtaining a second human body detection result through analyzing the capacitance data;
and the information fusion module adopts data-level and decision-level information fusion, takes the first human body detection result and the second human body detection result as fusion objects, and finally makes a judgment result whether people exist in the pipeline or not.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911089339.2A CN110826503B (en) | 2019-11-08 | 2019-11-08 | Closed pipeline human body detection method and system based on multi-sensor information fusion |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911089339.2A CN110826503B (en) | 2019-11-08 | 2019-11-08 | Closed pipeline human body detection method and system based on multi-sensor information fusion |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110826503A CN110826503A (en) | 2020-02-21 |
| CN110826503B true CN110826503B (en) | 2023-04-18 |
Family
ID=69553734
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911089339.2A Active CN110826503B (en) | 2019-11-08 | 2019-11-08 | Closed pipeline human body detection method and system based on multi-sensor information fusion |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110826503B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112388635B (en) * | 2020-10-30 | 2022-03-25 | 中国科学院自动化研究所 | Method, system and device for fusing sensing and space positioning of multiple sensors of robot |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101666682A (en) * | 2009-08-06 | 2010-03-10 | 重庆邮电大学 | Neural network nonuniformity correction method based on scene statistics |
| CN101908209A (en) * | 2010-07-29 | 2010-12-08 | 中山大学 | Cubic spline-based infrared thermal image blind pixel compensation algorithm |
| CN104330164A (en) * | 2014-08-05 | 2015-02-04 | 凯迈（洛阳）测控有限公司 | Infrared focal plane array blind pixel detection method and device |
| CN105628337A (en) * | 2016-03-18 | 2016-06-01 | 烟台艾睿光电科技有限公司 | Infrared detector blind pixel detection system and method |
| CN108919360A (en) * | 2018-07-12 | 2018-11-30 | 山东科技大学 | A kind of conveyance conduit apparatus for detecting human body based on capacitance detecting |
| CN109558848A (en) * | 2018-11-30 | 2019-04-02 | 湖南华诺星空电子技术有限公司 | A kind of unmanned plane life detection method based on Multi-source Information Fusion |
-
2019
- 2019-11-08 CN CN201911089339.2A patent/CN110826503B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101666682A (en) * | 2009-08-06 | 2010-03-10 | 重庆邮电大学 | Neural network nonuniformity correction method based on scene statistics |
| CN101908209A (en) * | 2010-07-29 | 2010-12-08 | 中山大学 | Cubic spline-based infrared thermal image blind pixel compensation algorithm |
| CN104330164A (en) * | 2014-08-05 | 2015-02-04 | 凯迈（洛阳）测控有限公司 | Infrared focal plane array blind pixel detection method and device |
| CN105628337A (en) * | 2016-03-18 | 2016-06-01 | 烟台艾睿光电科技有限公司 | Infrared detector blind pixel detection system and method |
| CN108919360A (en) * | 2018-07-12 | 2018-11-30 | 山东科技大学 | A kind of conveyance conduit apparatus for detecting human body based on capacitance detecting |
| CN109558848A (en) * | 2018-11-30 | 2019-04-02 | 湖南华诺星空电子技术有限公司 | A kind of unmanned plane life detection method based on Multi-source Information Fusion |
Non-Patent Citations (3)
| Title |
|---|
| 杨敬松 等."基于DSmT的多传感器协同生命探测技术研究".《国际地震动态》.2015,第187页. * |
| 王书伟 等."人体生命信息探测系统中的传感器技术".《红外》.2008,第29卷(第29期),第24-26页. * |
| 雷龙飞 等."一种基于电容检测的密闭管道人体掉入探测方法".《科学技术与工程》.2019,第19卷(第19期),第126-130页. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110826503A (en) | 2020-02-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6667596B2 (en) | Object detection system, autonomous vehicle using the same, and object detection method thereof | |
| CN112401856B (en) | Nursing home monitoring method and system based on millimeter wave radar | |
| CN109409368A (en) | Mine leather belt is vertical to tear detection device and detection method | |
| CN102928435A (en) | Aircraft skin damage identification method and device based on image and ultrasound information fusion | |
| CN117274722B (en) | Intelligent detection method for distribution box based on infrared image | |
| CN112782664B (en) | Toilet falling detection method based on millimeter wave radar | |
| CN112525201B (en) | Underwater target tracking method based on electromagnetic field characteristic multi-information fusion | |
| CN114942434A (en) | Fall attitude identification method and system based on millimeter wave radar point cloud | |
| CN114814832A (en) | Millimeter wave radar-based real-time monitoring system and method for human body falling behavior | |
| CN110826503B (en) | Closed pipeline human body detection method and system based on multi-sensor information fusion | |
| CN103745216A (en) | Radar image clutter suppression method based on airspace feature | |
| CN110473168A (en) | A kind of Lung neoplasm automatic checkout system squeezing-motivate residual error network based on 3D | |
| WO2023284764A1 (en) | Method and apparatus for detecting living body in vehicle by means of radar, and terminal device | |
| CN117309159A (en) | Train cabinet flame early warning method based on infrared thermal imaging | |
| CN110398735B (en) | Multi-radar-based perception data processing method and system | |
| CN113900101A (en) | Obstacle detection method and device and electronic equipment | |
| CN116250832A (en) | Fall detection method based on target following millimeter wave radar | |
| JP7081720B2 (en) | Foreign object detector, foreign object detection method, and program | |
| CN111414967A (en) | Method for improving robustness of temperature measurement system and monitoring system | |
| CN116862829A (en) | Overhead line sag monitoring method and device in ice-covered environment | |
| CN116520315A (en) | Target recognition system, target recognition method and target recognition device | |
| CN109800686A (en) | A kind of driver's smoking detection method based on active infrared image | |
| Raeis et al. | InARMS: Individual activity recognition of multiple subjects with FMCW radar | |
| CN112001896B (en) | Thyroid gland border irregularity detection device | |
| CN112258458A (en) | Detection method and system for super high-rise concrete pump pipe based on thermal imaging identification |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |