WO2022104817A1

WO2022104817A1 - Non-contact body temperature measurement method and system

Info

Publication number: WO2022104817A1
Application number: PCT/CN2020/130987
Authority: WO
Inventors: 王云龙; 邹捷
Original assignee: 江苏镭博智能科技有限公司
Priority date: 2020-11-23
Filing date: 2020-11-24
Publication date: 2022-05-27
Also published as: CN112434598A

Abstract

A non-contact body temperature measurement method and system, relating to the technical field of temperature measurement. Provided are a non-contact body temperature measurement method and system targeting the problems in the prior art of the relatively large limitations on measurement position and low measurement accuracy in infrared temperature measurement; a visible light camera and a heat-sensitive camera are simultaneously used for data acquisition, image data collected by the cameras being detected by means of a face detection algorithm, face matching being performed on the face data collected by the visible light camera and the face data detected by the heat-sensitive camera, and a forehead temperature set being read by combining the forehead range; the forehead temperature data is processed to obtain a final measurement result, and a human-machine interaction device may display or raise an alarm for the measurement result; the present non-contact body temperature measurement method and system can implement long-distance non-contact body temperature measurement, have high measurement accuracy and high reliability, and are particularly suitable for use in security checks in public places such as stations and airports.

Description

A non-contact body temperature measurement method and system

technical field

The present invention relates to the technical field of temperature measurement, and more particularly, to a non-contact body temperature measurement method and system.

Background technique

The coronavirus was discovered in 1956, but people's current understanding of the coronavirus is rather limited. It is currently known that after the human body is infected with the coronavirus, there will be respiratory symptoms such as fever. Some coronaviruses are highly contagious and have the characteristics of human-to-human transmission. In just a few months, they will spread wildly, causing a large number of people to be infected. There is currently no effective treatment for many coronaviruses, and no relevant vaccines have been developed. Once a large-scale infection occurs, a large number of people will die, and it will also cause immeasurable losses to the economy.

Fever is one of the main symptoms of coronavirus. Since most viruses are very contagious, measuring body temperature can effectively screen out potentially infected people. However, contact measurement needs to be measured by the measuring personnel one by one, and the measurement efficiency is low, and close contact with asymptomatic infected persons will increase the risk of infection. Therefore, the contact temperature measurement method is not suitable for crowded public places such as airports, stations, and shopping malls. Suitable for.

In the prior art, in a crowded public place, an infrared camera is mainly used for temperature measurement. In the process of temperature measurement, the person to be tested needs to stand at the designated position. The distance between the person to be tested and the infrared camera should generally not exceed 1 meter. The temperature measurement efficiency is relatively low, and it is easy to gather people during centralized measurement, which increases infection. risk. At the same time, due to the influence of factors such as the environment, the infrared measurement of the prior art has low measurement accuracy, and is not reliable enough and not very practical when applied to the investigation of coronavirus-infected personnel.

SUMMARY OF THE INVENTION

1. Technical problems to be solved

Aiming at the problems existing in the prior art that infrared temperature measurement has a large limitation on the measurement position and low measurement accuracy, the present invention provides a non-contact body temperature measurement method and system, which can realize long-distance non-contact body temperature measurement, and The measurement accuracy is high, especially suitable for the security check in public places such as stations and airports.

2. Technical solutions

The object of the present invention is achieved through the following technical solutions.

A non-contact body temperature measurement method, which uses a visible light camera to collect a visible light image, uses a thermal camera to collect a thermal image and temperature, performs face detection on the collected visible light image and thermal image respectively, and detects the visible light obtained after the face detection. Face pairing is performed between the face image and the thermal face image, and the paired face image is matched with the temperature collected by the thermal camera, and the forehead temperature of the face image is obtained and processed to obtain body temperature data. The invention matches the image collected by the visible light camera with the image collected by the thermal camera, reads the temperature collected by the thermal camera based on the image collected by the visible light camera, and realizes non-contact measurement. The present invention has no requirements on measurement environment or measurement distance, can realize mobile non-contact measurement, and has high measurement accuracy and high reliability.

Further, the face detection model used in face detection is constructed by using a deep convolutional neural network detection algorithm.

Further, input the collected visible light face image to the visible light face detection model to detect the position of the face in the visible light image and determine whether the face wears a mask; input the collected thermal face image to the thermal face detection model. The face detection model is used to detect the location of faces in thermal images. The face detection model of the present invention includes a visible light face detection model and a heat-sensitive face detection model. During face detection, the face detection model will be retrained according to the collected data, and more accurate face detection results have been obtained.

Furthermore, the epipolar geometry algorithm is used in face pairing, and the thermal face image is matched with the visible light face image according to the face center of the visible face image and the epipolar line corresponding to the thermal face image. When faces are paired, the epipolar line in the thermal camera is calculated from the center of the face detected in the visible light image. If there is also a detected face near the epipolar line in the thermal image, then the face closest to the epipolar line is Corresponding face, so as to complete the matching of the detected face in the visible light image and the thermal image. Generally speaking, in an image with a resolution of 320*240, the vicinity of the epipolar line refers to a distance of 0-40 pixels from the epipolar line.

Furthermore, when the faces are paired, the spatial correspondence between the visible light camera and the thermal camera reconstructed by the stereo vision algorithm is used to calculate the epipolar line, and the correspondence is represented by the basic matrix. When the grid is calibrated, the thermal camera uses sunlight to illuminate the checkerboard to form a temperature difference. Based on the characteristics of high absorption temperature of black and high reflection temperature of white, when the checkerboard is illuminated by sunlight for a long time, there will be obvious temperature difference. , to calculate the fundamental matrix.

Further, according to the ratio of the upper and lower boundary lines of the forehead in the visible light face image to the rectangular frame of the face, the temperature in the corresponding proportional area of the thermal face image is read, that is, the forehead temperature of the face image. First, a large amount of data is collected by the visible light camera, and the upper and lower boundary lines of the face forehead are counted in the detected face rectangle frame ratio. The face image is detected in combination with the thermal image, the forehead position is estimated according to the ratio of the upper and lower boundaries of the forehead in the rectangular frame, and the temperature of the forehead position is read.

Further, perform data processing on the temperature of the forehead position of the face image, sort the obtained temperature of all pixel points of the forehead position in descending order, select the X1 temperature values before the sequence for calibration, and then sort the temperature after calibration in descending order, select The average of X2 temperature values before the sequence is taken, and the average is the body temperature value. Both X1 and X2 are positive integers, and X1≥X2.

Further, the temperature measurement results are displayed or alarmed through the human-computer interaction device. The temperature of the tested person, whether they wear a mask, and other information obtained based on the data collected by the visible light camera combined with the algorithm are displayed through the human-computer interaction device. For public places, there is no required mask or abnormal body temperature, or the history of travel and residence is expressed according to big data. Real-time alarm for abnormal personnel. The invention uses the visible light camera and the thermal camera to collect data at the same time, performs face detection and matching on the data collected by the visible light camera and the thermal camera, and obtains the temperature of the subject's forehead as the measurement temperature, and the measurement method is simple and does not require close contact. Using the measuring method of the present invention to measure within the range of 0.5 meters to 5 meters, the absolute value of the difference between the detected temperature and the marked temperature is 0.208, and the proportion of the absolute value of the difference between the detected temperature and the marked temperature is less than 0.3 is 73.5% , the proportion of the absolute value of the difference between the detected temperature and the marked temperature is less than 0.5 is 89%, the average value of the difference between the detected temperature and the marked temperature is -0.0054, and the standard deviation is 0.258.

A non-contact body temperature measurement system, using the non-contact body temperature measurement method, the measurement system includes a visible light camera, a thermal camera, an embedded processor and a human-computer interaction device, and the visible light camera and the thermal camera are connected by connecting The device is connected to the embedded processor, and both the visible light camera and the thermal camera send the collected information to the embedded processor, and the embedded processor sends the processed data to the human-computer interaction device.

Furthermore, the relative positions of the visible light camera and the thermal camera are fixed. The positions of the visible light camera and the thermal camera are relatively fixed. The measuring system of the present invention does not need to consider the distance between cameras during the face matching calculation, has fast calculation speed, high matching efficiency and stronger system reliability.

The measurement system of the invention has a simple structure, uses a high-performance embedded processor for data processing, has fast measurement speed, high measurement accuracy, and simple maintenance in the later period. High measurement accuracy, suitable for applications in public places such as stations, airports, schools, etc.

3. Beneficial effects

Compared with the prior art, the advantages of the present invention are:

The body temperature measurement system of the present invention realizes non-contact measurement, which is different from the traditional infrared camera, and does not require a person to stand in a fixed position, and the temperature can be measured as long as a human face appears in the imaging of the camera. The present invention can measure multiple faces at the same time. In theory, all faces within the visual field of the thermal image can measure body temperature. The invention has a fast measurement speed, is especially suitable for body temperature measurement in public places, and can greatly improve the efficiency of body temperature measurement. The invention does not have high requirements on the distance to be measured, and can be ready to measure between 0.5 meters and 5 meters, effectively widening the distance between people during measurement, realizing mobile body temperature measurement, and avoiding short distances during temperature measurement. contact to avoid the spread of the virus.

The measurement method of the present invention has very strong robustness. For a common infrared thermometer, if the hair covers the forehead, the hair needs to be removed by hand, otherwise the measurement result will be affected. The body temperature measurement system of the present invention uses heat-sensitive temperature measurement, and as long as the forehead is not completely blocked, the gap can be accurately found and the body temperature can be measured. In most cases, the measurement can be achieved simply by exposing the subject in front of the camera of the measurement system, which is very convenient.

The present invention adopts the method of deep learning, and whether it is a visible light image or a thermal image, the position of the human face can be accurately found in various complicated situations. Different from the prior art which requires close-range temperature measurement in the range of 0.5m to 1m, the present invention can realize temperature measurement from 0.5m to 5m. The absolute value is 0.208 degrees Celsius, the proportion of the absolute value of the difference between the detected temperature and the marked temperature is less than 0.3 degrees Celsius is 73.5%, and the proportion of the absolute value of the difference between the detected temperature and the marked temperature is less than 0.5 degrees Celsius is 89%, The average value of the difference between the detected temperature and the marked temperature is -0.0054, the standard deviation is 0.258, and the measurement accuracy is high and the reliability is strong.

Description of drawings

1 is a block diagram of a body temperature measurement system of the present invention;

FIG. 2 is a schematic diagram of an epipolar geometry algorithm when faces are paired according to the present invention.

Detailed ways

The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

Example 1

A non-contact body temperature measurement system is shown in Figure 1. The measurement system includes a visible light camera, a thermal camera, an embedded processor and human-computer interaction equipment, as well as other auxiliary equipment; Both the camera and the human-computer interaction device are connected. The visible light camera and the thermal camera are connected to the embedded processor through the FAKRA connector; the visible light camera is used to collect visible light image information, and the thermal camera is used to collect thermal image information and temperature information. The processor uses the face detection algorithm, the face pairing algorithm and the forehead temperature measurement algorithm to process the data collected by the camera, and the human-computer interaction device displays and alarms the processed data.

The embedded processor uses a high-performance deep convolutional neural network algorithm to detect the face information collected by the visible light camera and the thermal camera, and detects whether the face in the face data collected by the visible light camera is wearing a mask. The corresponding face in the image captured by the visible light camera and the image captured by the thermal camera is paired by a stereo vision algorithm and an epipolar geometry algorithm. In the image collected by the thermal camera, the position of the forehead is estimated according to the proportion of the face and the temperature is collected, and the image collected by the visible light camera, whether a mask is worn, the temperature of the forehead and other relevant information obtained by the big data algorithm based on the face information are output to the human-computer interaction. On the device, realize information display or alarm.

Example 2

This embodiment is based on the non-contact body temperature measurement system described in Embodiment 1, and specifically describes the measurement method of the system.

When measuring body temperature, the system first collects data through a visible light camera and a thermal camera, respectively, and performs face detection on the collected data. The visible light camera collects the RGB image, that is, the visible light image, and the thermal camera collects the thermal image. Faces are detected in visible light images and thermal images through deep face detection algorithms. The system uses NVIDIA Jetson-TX2 embedded processor in face detection, and it takes about 75 milliseconds to detect a face in a visible light image with a resolution of 480x640, and about 75 milliseconds for a thermal image with a resolution of 320x240. 45ms, compared with the prior art face detection which takes about 100ms, the detection time of this measurement system is shorter and the efficiency is higher.

Specifically, this embodiment adopts the object detection algorithm of deep convolutional neural network. The specific method is to transmit the picture to the network, first extract the general features of the picture through the backbone network framework (mobilenet_v2), and then extract the general features through the feature pyramid. C3, C4, C5layer, C3, C4, C5layer pass the upper layer information to the bottom from top to bottom through feature fusion, and finally connect the head layer to output the classification of objects and the positioning information of objects. When the face detection model is trained, it adopts multi-task loss, which mainly includes sample classification loss and sample positioning loss. The specific formula is: L=a1*Lcls+a2*Lbox, where L represents the sum of losses, and a1 and a2 represent samples respectively. The classification loss coefficient and the sample localization loss coefficient, Lcls is the sample classification loss, and Lbox is the sample localization loss. The face detection algorithm used in this embodiment can quickly and reliably detect faces in the images collected by the visible light camera and the image collected by the thermal camera.

According to the face data collected by the thermal camera, mark the position of the face in the data collected by the thermal camera, and train the thermal face detection model based on the deep convolutional neural network from the marked thermal camera collection data set, and continuously optimize the thermal face detection model. A sensitive face detection model, which is used to detect the location of faces in thermal images.

According to the face data collected by the visible light camera, mark the position of the face collected by the visible light camera and whether to wear a mask. The marked data set collected by the light camera can be used to train a visible light face detection model based on a deep convolutional neural network to optimize visible light. face detection model. The visible light face detection model is used to detect the position of the face in the visible light image and determine whether the face is wearing a mask. The face information detected by the visible light face detection model is sent to the human-computer interaction device for display. For those who do not wear masks as required in public places and have abnormal body temperature, the human-computer interaction device will issue an alarm.

Match the face image collected by the visible light camera and the face image collected by the thermal camera after face detection, and match the temperature data collected by the thermal camera with the face image data to obtain the temperature corresponding to each face. data.

When face pairing, first find the face center XL of the visible light face image, as shown in Figure 2, the schematic diagram of the epipolar geometry, the face center XL must correspond to an epipolar line on the thermal face image, and the epipolar line is Fig. 2 The straight line formed by X _R and e _R on the thermal image. If there is a detected face near the epipolar line in the thermal image, then the face closest to the epipolar line is the thermal face image corresponding to the visible light face image, thus completing the detection in the visible light image and the thermal image. face matching. Generally speaking, in an image with a resolution of 320*240, the vicinity of the epipolar line refers to a distance of 0-40 pixels from the epipolar line.

The corresponding relationship between the visible light image and the thermal image can be represented by the fundamental matrix in stereo vision, which describes the internal parameters such as focal length in the two cameras and the spatial correspondence between the two cameras. Since the positions of the visible light camera and the thermal camera are relatively fixed, the fundamental matrix can be calculated in advance by the method of stereo camera calibration.

This embodiment adopts the traditional checkerboard method to perform stereo camera calibration (Stereo Camera Calibration) on the visible light camera and the thermal camera. Stereo camera calibration is to calculate the fundamental matrix according to the correspondence between the midpoints of the checkerboard captured by the two cameras. That is, within the field of view of the two cameras, shoot a large number of different poses, checkerboards of different distances, take multiple photos, and then calculate the internal and external parameters of the camera through the camera calibration algorithm, and then according to the camera's internal parameters and external parameters Parameter combination fundamental matrix.

Since the thermal camera image is based on temperature, it is necessary to heat the checkerboard when calibrating the stereo camera. Because the thermal camera can display a clear picture only when the temperature difference is relatively large, special heating treatment is performed on the checkerboard, which makes the white grid and the black grid produce a clear temperature difference. At this time, it is based on the characteristics of high absorption temperature of black and high reflection temperature of white. When sunlight is used to illuminate the checkerboard for a long time, there will be a significant temperature difference. algorithm to calculate the fundamental matrix.

The specific calculation formula of the stereo vision algorithm is Z=P*X, where Z represents the coordinates of the object in the image coordinate system, X represents the coordinates of the object in the world coordinate system, and P represents the projection matrix. When the coordinates of the object in the world coordinate system are known When X and the coordinate Z of the object image in the image coordinate system, the projection matrix P can be obtained. The projection matrix P includes the rotation matrix R, the parallel matrix T and the camera parameters K. E represents the essential matrix, F represents the fundamental matrix, and the expression formula of the essential matrix E is:

E=T^R or

According to the rotation matrix R and the parallel matrix T in the projection matrix P, the essential matrix E can be obtained, and then the fundamental matrix F can be obtained. For a pair of points with the same name in a stereo pair, their homogenized image coordinates are P and P ₁ , respectively, and the formula is expressed as

Therefore, when the fundamental matrix F and one of the homogenized coordinates are known, a straight line passing through the point in another image coordinate system can be obtained, which is the epipolar line in the epipolar geometry algorithm.

The embedded processor calculates the corresponding epipolar line of the center point of the face detected by the visible light camera in the thermal camera according to the fundamental matrix, and searches for the face image detected by the nearest thermal camera near the epipolar line. Realize face matching. The relative positions of the visible light camera and the thermal camera are required to be fixed when the face is paired, that is to say, after the two cameras are calibrated to calculate the basic matrix, the relative position cannot change greatly. , the two cameras in the measuring system have been fixed in position by the housing, so in principle no positional movement will occur during use.

After the visible light image and the thermal image are paired with the face, the forehead temperature data in the face image is obtained by using the forehead temperature measurement algorithm and recorded as the body temperature data. In the forehead temperature measurement algorithm, a large amount of data is first collected by a visible light camera, and the upper and lower boundary lines of the face forehead are counted in the detected face rectangle frame ratio. Detect the face image in combination with the thermal image, estimate the forehead position of the thermal face image according to the ratio of the upper and lower boundaries of the forehead in the rectangular frame calculated from the visible light face image, and read the forehead position of the thermal face image. temperature. Due to the large number of pixels collected from the forehead position, and each person's hairstyle is different, the coverage of the hair will also offset the temperature information collected by the thermal camera, resulting in a larger temperature range collected.

To process images with a large range of forehead temperature data, first read the temperature of a single pixel according to the coordinates of the image collected by the thermal camera, then sort all pixel temperatures in descending order, and select the temperature value in the first 1%-10% range , and then compensate and calibrate the read temperature; then select the first 1%-50% of the data by descending order, and take the average value of the selected data as the final temperature. According to a limited number of experiments, the absolute value of the difference between the temperature detected by the non-contact measurement method in this embodiment and the marked temperature is 0.208 degrees Celsius, and the proportion of the absolute value of the difference between the detected temperature and the marked temperature is less than 0.3 degrees Celsius is 73.5 %, the absolute value of the difference between the detected temperature and the marked temperature is less than 0.5 degrees Celsius, the proportion is 89%, the average value of the difference between the detected temperature and the marked temperature is -0.0054, and the standard deviation is 0.258.

This embodiment can be used for non-contact mobile temperature measurement. The person under test can measure the body temperature without standing in a fixed position. The applicable distance for the measurement is 0.5 meters to 5 meters. Due to the different distances between the face and the camera, the radiation of Different attenuation, as well as factors such as ambient temperature and humidity, lead to large fluctuations in the original temperature obtained from thermal images.

Compensate the measured temperature of the thermal camera in real time, connect the front of the thermal camera to the temperature sensor, and calibrate the offset of the thermal camera according to the temperature measured by the thermal camera and the actual temperature of the temperature sensor; by setting the front of the thermal camera The temperature compensation function is constructed by the black body to calibrate the measurement error caused by the inhomogeneity of the thermal camera, the refractive index of the temperature sensor, the ambient temperature and the measurement distance, which greatly reduces the error of the non-contact measurement and increases the accuracy.

Temperature compensation calibration for thermal cameras consists of the following steps.

Step 1. Calibrate the drift of the thermal camera itself:

The thermal camera is connected with the temperature sensor, the temperature sensor is arranged in front of the thermal camera, the temperature sensor is coated with black body paint, and the thermal camera uses the temperature sensor to calibrate the drift of the camera itself.

The drift of the thermal camera itself is between ±2 degrees Celsius to ±5 degrees Celsius. Since the drift of the camera itself is the drift of the entire image, there will be no error in the measurement of a certain part or point in the image, but the measurement in other places is normal. Therefore, , if the exact temperature of a part of the image is known, the temperature of the entire screen can be compensated.

A thermal camera is used to measure a certain part or a point of the temperature sensor for multiple times at different times and temperatures to obtain i thermal cameras to measure the temperature Tx _1i , where i is a natural number greater than 1. Obtain the actual temperature Ty _1i of the corresponding temperature sensor when the temperature is measured by the thermal camera. For discrete data Tx _1i and Ty _1i , use linear fitting to obtain the function f1 between the temperature measured by the thermal camera and the actual temperature of the temperature sensor. The expression of f1 is Ty _1i =n ₁ *Tx _1i , where n ₁ is the thermal The camera drift compensation coefficient, using the function f1 to calibrate the camera itself drifts to cause errors.

In the prior art, most of the thermal cameras are calibrated by setting an additional black body, but the black body is expensive and requires high ambient temperature. In this embodiment, the thermal camera is connected with a high-precision temperature sensor in consideration of economical issues, and the temperature sensor is coated with high-reflectivity black body paint. Through various calibration algorithm experiments, the temperature sensor coated with black body paint is equivalent to a black body. It can achieve the effect similar to the black body, and greatly reduce the cost while ensuring the accuracy.

The high reflectivity black body paint uses oily 95 black body paint with a reflectivity of 0.95±0.02. The thermal camera reads the temperature of the high-precision temperature sensor through the I2C bus, and the thermal camera also measures the temperature of the temperature sensor. According to The actual temperature of the temperature sensor and the temperature measured by the thermal camera update the parameter n ₁ of the function f1 after linear fitting in real time. The measured temperature of the thermal camera is calibrated in real time through this parameter n _1. The high-precision temperature sensor is located at the upper left of the thermal camera. , about 20cm away from the thermal camera, the temperature sensor does not need to maintain a fixed position with the thermal camera during use, even if the temperature sensor position shifts during use, as long as the reference area of the temperature sensor for the thermal camera is corrected, it will not be Affects temperature measurement and data acquisition.

Step 2. Calibrate the thermal camera inhomogeneity:

Since the thermal camera is composed of thermal imaging lattices, the values obtained by objects of the same temperature in different screen positions will be inconsistent. The non-uniformity of thermal cameras refers to the inconsistency of the values obtained by objects of the same temperature at different screen positions. In step 2, a thermally uniform black body is set at a fixed position in front of the thermal camera, and the black body covers the entire image of the thermal camera, that is, the entire image captured by the thermal camera is the reflective surface of the black body. Set different black body temperatures, save the data image corresponding to the thermal camera, and obtain the function f2 between the temperature measured by the thermal camera and the black body temperature for the coordinates (x, y) of the position of each pixel in the data image, and calibrate by the function f2 Camera inhomogeneity causes errors.

During calibration, the distance between the black body and the thermal camera is about 10cm, and the temperature fluctuation of the entire black body image is within 0.3°C. When the black body temperature is set to T1, a data image Img T1 of the thermal camera is saved; then set the black body temperature to T2 to save A thermal camera data image Img T2.

First set the temperature T1 in the low temperature interval, the temperature range of the low temperature interval is 26 to 30 °C, wait for the temperature to stabilize, that is, when the temperature fluctuation of the reflective surface of the black body is less than 0.3 °C, collect several pictures, and take the temperature average for all pixels in the image. Value obtained data image Img T1. Then set the black body temperature to the high temperature interval temperature T2, the high temperature interval temperature range is 40 to 42 ℃, wait for the temperature to stabilize, that is, when the temperature fluctuation of the black body's reflective surface is less than 0.3 ℃, collect several pictures, for all pixels in the image The temperature average was obtained for the data image Img T2. By measuring the average value of the image data twice, the slope gain and offset offset of the temperature non-uniformity compensation function of the thermal camera are calculated to obtain the compensated temperature. The calculation formulas of gain and offset are as follows:

offset[x,y]＝T ₁ -gain[x,y]*ImgT ₁ [x,y] (2)

In the above formula, Img T ₁ [x, y] represents the temperature of the data image Img T1 at the (x, y) coordinate point, and Img T ₂ [x, y] represents the data image Img T2 at the (x, y) coordinate point The temperature of the pixel, gain[x, y] represents the slope gain of the (x, y) coordinate point pixel, and offset[x, y] represents the offset offset of the (x, y) coordinate point pixel. The function f2 is constructed by gain and offset, that is, T[x,y]=gain[x,y]*Img[x,y]+offset[x,y], where Img[x,y] is measured by the thermal camera (x,y) The temperature of the pixel, T[x,y] is the temperature after non-uniformity calibration. Use the function f2 to linearly correct the original measured temperature, so as to obtain the temperature after calibration of each coordinate point, that is, to obtain the measured value of the thermal camera after the inhomogeneity calibration.

Step 3. Calibration for the reflectivity of the temperature sensor:

Steps 1 and 2 calibrate the temperature of the thermal camera itself. Step 1 calibrates by reading the temperature at the position of the high-precision temperature sensor in the image and calculating the difference with the actual temperature of the high-precision temperature sensor. In step 2, the non-uniformity calibration has not taken into account the different reflectivity of the temperature sensor and the reflectivity of the black body, and the temperature sensor is very critical in the thermal imaging system of the thermal camera. Since the high-precision temperature sensor still has certain errors, after the steps After the camera drift calibration in step 1 and the non-uniformity calibration in step 2, the blackbody temperature sweep calibration is used in step 3 to further calibrate the reflectivity of the temperature sensor.

The black body temperature scanning calibration process is simply to fit the relationship between the temperature of a certain point read by the thermal camera after the first two steps of calibration and the actual temperature of this point, and the actual temperature of this point is obtained by reading the black body temperature. . Since the temperature fluctuates and is an interval, the relationship between the temperature of a certain point read by the thermal camera after calibration and the actual temperature of the black body is calculated by dynamically setting the temperature of the black body.

When calibrating, place the black body in front of the thermal camera, and connect the black body to the serial port of the TX2 box. The TX2 box is an embedded small computer of NVIDIA, and its serial port is an ordinary USB interface. After the two are connected, the TX2 box can read the temperature of the black body. . Set the temperature of the black body reflective surface, which is the plane of the temperature read by the thermal camera. After connecting directly to the TX2 box, click the center of the black body with the mouse, and select the black body temperature scanning range. The scanning range starts from the temperature in the low temperature range to the temperature in the high temperature range. The interval temperature is generally 40 to 42°C.

After each temperature measurement point is stable, the thermal camera reads the temperature of the selected area, selects the temperature read by multiple pictures, and takes the average temperature, which is the final reading of the thermal camera. At this time, the independent variable of the function f3 is the reading of the thermal camera after the uniformity calibration in step 2 and the high-precision sensor calibration in step 1; the strain variable of the function f3 is the temperature of the black body, which is our target temperature. The function f3 is obtained by linearly fitting the temperature measured by the independent variable thermal camera and the actual temperature of the strain variable black body.

Specifically, set a black body in front of the thermal camera, set different temperatures Ty _3i on the black body reflective surface, read the measured temperature Tx _3i of the thermal camera after calibration in steps 1 and 2, and obtain the measured temperature of the thermal camera and the black body reflective surface. The function f3 between temperatures, the formula of the function f3 is Ty _3i =n ₃ *Tx _3i , where n ₃ is the temperature sensor reflectance compensation coefficient; the relationship between Tx _3i and the thermal camera measurement value Tx _3i0 is Tx _3i =(gain ×Tx _3i0 +offset)*n ₁ , Tx _3i [x, y] represents the measured temperature of the (x, y) coordinate point pixel after calibration in step 1 and step 2, and the error caused by the reflectivity of the temperature sensor is caused by the function f3 calibration .

The thermal camera is not connected to the black body in actual use, and the black body is used here only to calculate the fitting function f3. The user reads the temperature after calibrating the drift and inhomogeneity of the camera itself, and then further corrects it through the linear transformation function f3 calibrated by the temperature sensor, so that the temperature read by the thermal camera is more accurate.

Step 4. Calibration for ambient temperature:

Different ambient temperatures will affect the measurement of thermal cameras. Since thermal radiation will reach the lens to heat the lens, and the camera itself will also heat up, it is easy to cause the lens to radiate heat to the thermal imaging chip, affecting the temperature in degrees. Although the thermal camera itself also has an ambient temperature calibration, after actual measurement, the ambient temperature calibration standard of the thermal camera itself is lower. Moreover, the thermal imaging system in this embodiment needs to read a high-precision temperature sensor to calibrate the camera drift, so that the system needs to perform more accurate ambient temperature calibration.

When calibrating the ambient temperature, first set different ambient temperatures, such as setting the ambient temperature to T3 and T4, read the camera readings V1 and V2 after calibration in step 3, and draw the correlation curve between the readings V1 and V2 and the ambient temperatures T3 and T4, Calculate the linear ambient temperature calibration coefficient n ₄ , n ₄ =(V2-V1)/(T3-T4).

The ambient temperature calibration process requires a temperature-controlled, or slowly rising or falling temperature test environment. When calibrating, place the black body in front of the thermal camera and set the temperature of the black body. Then run the software, set the relevant parameters, click the center of the black body with the mouse and collect data. At this time, the independent variable in the function f4 is the ambient temperature, and the dependent variable is the temperature of the black body read. Observe the influence of the ambient temperature on the final reading, fit and calculate the functional relationship between the two, and obtain the function f4 between the temperature measured by the thermal camera and the ambient temperature, that is, Ty _4i =Tx _4i *n ₄ , Tx _4i is the current different environment In the temperature, the temperature is measured by the thermal camera after calibration in step 3. Ty _4i represents the temperature compensation value after calibration of the ambient temperature, and the ambient temperature of the thermal camera is compensated by the function f4.

Step 5. Calibration for test distance:

Thermal cameras measure temperature by reading energy in far-infrared wavelengths. In the process of transmission, the energy of infrared light will be absorbed by the air and attenuated, so the farther the distance, the lower the measured temperature. Step ₅ : Measure the temperature of the same black body at different distances, calculate the distance coefficient, and calibrate the test distance of the thermal camera through the distance coefficient n5.

The calibration process for the test distance is as follows, first place the black body in front of the thermal camera, and set the temperature of the black body. Then by moving the distance of the black body, read the temperature of the black body at the corresponding distance. At this time, the independent variable in the function f5 is the distance between the black body and the camera, and the dependent variable is the temperature of the black body read by the camera. By fitting the data, the relationship between the independent variable and the dependent variable, that is, the function f5, is obtained. The function f5 is a polynomial formula, and the specific calculation formula is Ty _5i =n ₅₁ *Tx _5i ^m-1 +n ₅₂ *Tx _5i ^m-2 +...+n _5(m-1) *Tx _5i ¹ +n _5m , the distance coefficient n ₅ =[n ₅₁ ,n ₅₂ ,...,n _5(m-1) ,n _5m ], where m is an integer greater than 1, Tx _5i is the measured temperature calibrated by the thermal camera in step 4, and Ty _5i represents Temperature compensation value after distance calibration. When the distance between the object and the camera is known, the temperature after calibration can be obtained through the distance compensation coefficient n ₅ and the polynomial order m. In this embodiment, the polynomial order m=7.

The temperature fluctuations read by thermal cameras in the prior art are about 2-5 degrees Celsius. According to a limited number of experimental data, the absolute value of the difference between the temperature of a single point read and the actual temperature of the black body is 0.133 degrees Celsius on the data set collected by the calibrated thermal camera of the present invention from 0.5 meters to 5 meters. The average value of the error is 0.0026, the standard deviation (std) of the error is 0.1718, the proportion of pixels with an absolute value of error less than 0.3°C is about 92%, and the proportion of pixels with an absolute value of error less than 0.5°C can reach 99.44%. The accuracy and precision were significantly increased.

The human-computer interaction device of the present invention uploads the face data collected by the visible light camera, body temperature, whether to wear a mask and other related information to the cloud and to the display and alarm device to realize non-contact long-distance human body temperature measurement. When collecting human body temperature, the present invention can realize non-contact body temperature measurement in the area of 0.5 meters to 5 meters, and the application range is very wide. During measurement, the thermal camera also calibrates the reading data through a temperature compensation method to obtain higher accuracy and precision. The measuring system of the present invention is applied in the security inspection with a large flow of people in public places, and can accurately measure the body temperature, and timely alarm prompts for those who do not wear masks or have abnormal body temperature, with high accuracy and high reliability, and has good practicability.

The invention and its embodiments have been described above schematically, and the description is not restrictive. The invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. What is shown in the accompanying drawings is only one of the embodiments of the present invention, and the actual structure is not limited thereto, and any reference signs in the claims shall not limit the related claims. Therefore, if those of ordinary skill in the art are inspired by it, and without departing from the purpose of the present invention, any structure and embodiment similar to this technical solution are designed without creativity, which shall belong to the protection scope of this patent. Furthermore, the word "comprising" does not exclude other elements or steps, and the word "a" preceding an element does not exclude the inclusion of "a plurality" of that element. Several elements recited in a product claim can also be implemented by one element by means of software or hardware. The terms first, second, etc. are used to denote names and do not denote any particular order.

Claims

A non-contact body temperature measurement method, characterized in that a visible light camera is used to collect a visible light image, a thermal camera is used to collect a thermal image and a temperature, face detection is performed on the collected visible light image and thermal image respectively, and the face detection The obtained visible light face image and the thermal face image are then paired with the face, and the paired face image is matched with the temperature collected by the thermal camera, and the forehead temperature of the face image is obtained and processed to obtain body temperature data.
A non-contact body temperature measurement method according to claim 1, wherein the face detection model used in face detection is constructed by using a deep convolutional neural network detection algorithm.
A non-contact body temperature measurement method according to claim 2, wherein the collected visible light face image is input to the visible light face detection model for detecting the position of the face in the visible light image, and judging the face Whether to wear a mask; input the collected thermal face image to the thermal face detection model to detect the position of the face in the thermal image.
A kind of non-contact body temperature measurement method according to claim 1, it is characterized in that, using the epipolar geometry algorithm when face pairing, according to the face center of the visible light face image and the epipolar line corresponding to the thermal face image, Match the thermal face image with the visible light face image.
A non-contact body temperature measurement method according to claim 4, wherein the epipolar line is calculated using the spatial correspondence between the visible light camera and the thermal camera reconstructed by the stereo vision algorithm when the faces are paired, and the correspondence is calculated by The basic matrix indicates that the basic matrix is calculated by using the checkerboard calibration method, and the temperature difference is formed by using sunlight to illuminate the checkerboard for the thermal camera during checkerboard calibration.
A non-contact body temperature measurement method according to claim 1, wherein the corresponding ratio of the thermal face image is read according to the ratio of the upper boundary line and the lower boundary line of the forehead in the visible light face image to the rectangular frame of the face. The temperature in the area, that is, the temperature of the forehead of the face image.
A non-contact body temperature measurement method according to claim 6, wherein data processing is performed on the temperature of the forehead position of the face image, the obtained temperature of all pixel points of the forehead position are sorted in descending order, and X1 before the sequence is selected. The temperature values are calibrated, and then the temperature after calibration is sorted in descending order, and the average of the X2 temperature values before the selection sequence is taken. The average is the body temperature value. X1 and X2 are positive integers, and X1≥X2
The non-contact body temperature measurement method according to claim 1, wherein the temperature measurement result is displayed or alarmed by a human-computer interaction device.
A non-contact body temperature measurement system, characterized in that, using a non-contact body temperature measurement method according to any one of claims 1-8, the measurement system includes a visible light camera, a thermal camera, an embedded processor and Human-computer interaction equipment, visible light camera and thermal camera are connected to the embedded processor through the connector, both the visible light camera and the thermal camera send the collected information to the embedded processor, and the embedded processor sends the processed data to the human-machine interactive device.
The non-contact body temperature measurement system according to claim 9, wherein the relative positions of the visible light camera and the thermal camera are fixed.