TWI730795B - Multi-target human body temperature tracking method and system - Google Patents

Abstract

The invention discloses a multi-target human body temperature tracking method and system, which includes a thermal imager, an information processing unit and a warning device. The thermal imager is set in the area with the moving line, and can perform continuous image capture of people walking on the moving line to form a thermal image of the human body. The information processing unit is used to analyze and process the color temperature distribution of the current thermal image of the human body to determine whether the body temperature of a person exceeds the preset body temperature value. When the judgment result is yes, the person in the thermal image of the human body is set as the tracking target, and The tracking target is marked; the next frame of human thermal image is analyzed and processed for the color temperature distribution to track and calculate the tracking target's movement trajectory information, and determine whether the movement trajectory information moves in the direction of the moving line, and the judgment result is yes , The warning signal is output. The warning device is used to output the warning signal as warning information, so that multiple specific objects can be locked to track the movement of multiple specific body temperatures, thereby reducing the occurrence of missing tracking objects due to too many people being tracked.

Description

Multi-target human body temperature tracking method and system

The present invention relates to a multi-target human body temperature tracking method and system, in particular to a human body temperature tracking technology that can lock multiple specific objects to track multiple body temperature trends.

Press, the Wuhan new coronavirus (new coronavirus for short) was first discovered in Wuhan, China in December 2019. Since the outbreak of the epidemic, up to now, there have been more than millions of confirmed cases in many countries, and more than hundreds of thousands of deaths have been caused, and the mortality rate is as high as 3-10%. Based on the serious threat of the aforementioned viruses Therefore, in areas with frequent and dense movement of people, for example, school entrances, company entrances, airport lobby entrances, hospital lobby entrances, office building entrances, department store entrances and other regional places have mostly set up temperature screening stations to detect Entering the body temperature of a person, once a person with a hyperthermia is found, they will be prohibited from entering these areas, so as to achieve the purpose of fever and epidemic screening.

According to what we know, the best body temperature screening system at this stage is a thermal imaging camera. However, the current human group temperature images captured by the thermal imaging camera are still based on manual judgments. When the monitoring crowd moves, there are too many targets. Misjudgment and misplacement will occur, and people with abnormal body temperature will have the opportunity to cross the border. In other words, this manual inspection method lacks a consistent standardized procedure, and the test results obtained may vary from person to person, so it cannot be predicted in advance; or it can only be blocked randomly, and different inspectors will block the wrong object due to misjudgment. If different results are detected, which leads to disputes between the two parties, both parties will suffer. Not only that, but relying solely on the inspectors’ Regardless of the differences caused by individual subjective identification, the direct use of purely manual methods will also cause problems of low efficiency and high labor costs, and the judgment of the inspectors may also be affected as their mental power decreases. This further affects the accuracy of the body temperature test results, and causes inconvenience and troubles in the body temperature screening.

In order to improve the above-mentioned deficiencies, the relevant technical field has a patent as shown in the Republic of China Invention No. I626926 "System and Method for Detecting Individuals with Abnormal Body Temperature"; this patent can be based on the time when the target thermal image is generated and the target thermal image. For the coordinates of each grid image, the image analysis module finds the grid images corresponding to the actual image, and restores the grid images into three-dimensional images, so that the feature recognition module can target the facial features Or the skull feature is used to identify the three-dimensional image; however, the patent does not have a multi-person movement tracking function setting, so that it is impossible to lock multiple specific objects to perform multiple specific body temperature movement tracking, so it is more likely to be caused by too many people being tracked. The resulting loss of the tracked object.

In view of this, the conventional thermal imager technology and the above-mentioned patent do not have an automatic tracking technology, patent or paper that can lock multiple specific objects at the same time to perform multiple specific body temperature movements, and is based on the urgency of related industries. Under the demand, the inventors of the present invention have finally developed a set of the present invention which is different from the above-mentioned conventional technology through continuous research and development efforts.

The main purpose of the present invention is to provide a multi-target human body temperature tracking method and system, which can lock multiple specific objects to perform tracking of multiple specific body temperatures, in addition to reducing the loss of tracking objects caused by too many people being tracked In addition, the image recognition procedure of the appearance contour can be omitted, and the center of gravity position information of the tracking target can be directly obtained, so as to achieve the purpose of rapid recognition and tracking. The technical means used to achieve the main purpose of the invention include a thermal imager, an information processing unit and a warning device. The thermal imager is set in the area with moving lines, which can Continuous image capture is performed for people walking on moving lines, and the image is taken as a thermal image of the human body. The information processing unit is used to analyze and process the color temperature distribution of the current thermal image of the human body to determine whether the body temperature of a person exceeds the preset body temperature value. When the judgment result is yes, the person in the thermal image of the human body is set as the tracking target, and The tracking target is marked; the next frame of human thermal image is analyzed and processed for the color temperature distribution to track and calculate the tracking target's movement trajectory information, and determine whether the movement trajectory information moves in the direction of the moving line, and the judgment result is yes , The warning signal is output. The warning device is used to output the warning signal as warning information.

10: Thermal imager

20: Information Processing Unit

21: Image recognition module

210: Block parameter database

211: Personal parameter database

22: Display

30: Warning device

40: The first image capture device

41: Second image capture device

50: unmanned aircraft

51: The third image capture device

52: Satellite positioning module

53: Flight Control Module

54: power output unit

A: area

L1: Moving line

m1: movement trajectory

m2: moving direction

ob, ob1, ob2, ob3: track the target

Figure 1 is a schematic diagram of the specific implementation of the present invention.

Fig. 2 is a schematic diagram of the frame selection and tracking of the thermal image of the human body according to the present invention.

Figure 3 is a schematic diagram of image tracking implementation of the image positioning method of the present invention.

FIG. 4 is a schematic diagram of the result of the movement track and the movement direction displayed by the image positioning method of the present invention.

Figure 5 is a functional block diagram of the basic implementation architecture of the present invention.

Fig. 6 is a functional block diagram of the specific implementation architecture of the present invention.

In order for your reviewer to further understand the overall technical features of the present invention and the technical means to achieve the purpose of the invention, a detailed description is given with specific examples and accompanying drawings:

Please refer to FIGS. 1 to 6 in order to achieve the main purpose of the present invention. The embodiment includes a thermal imager 10, an information processing unit 20, and a warning device 30 and other technical features. The thermal imager 10 is set in an area A with a moving line L1, and can perform continuous image capture of a person walking on the moving line L1 to form a plurality of frames of human thermal images. The information processing unit 20 is used to analyze and process the color temperature distribution of the currently captured thermal image of the human body to determine whether the body temperature of at least one person walking on the moving line L1 exceeds the preset body temperature value. When the determination result is yes, the human body heat At least one person in the image is set as the tracking target ob, and the tracking target ob is marked; then the color temperature distribution of the next thermal image of the human body is analyzed and processed to track and calculate the movement trajectory information of the tracking target ob, and determine the movement Whether the trajectory information moves in the same direction as the moving line L1, and the judgment result is yes, a warning signal is output. The warning device 30 is used to output the warning signal as warning information representing the tracking and attention tracking target ob.

As mentioned above, the mark can be a combination of the frame of the tracking target ob and the temperature mark. The color of the frame matches the color of the highest color temperature of the tracking target ob. The frame shown in Figure 2 is A black rectangular frame, where the first tracking target ob1 is 38 degrees, the second tracking target ob2 is 37.8 degrees, and the third tracking target ob1 is 38.5 degrees. The several circular dashed frames and dots shown in Figure 3 are the center of gravity of the thermal image of the human body in the previous frame (the position with the highest color temperature can be defined, such as the head as the center of gravity); several circular solid lines and dots Is the center of gravity of the human body thermal image currently input. From Fig. 4, the movement trajectory m1 and the movement direction m2 of each tracking target ob1, ob2, ob3 are obtained.

In the first multi-target tracking application embodiment of the present invention, the information processing unit 20 uses the Mahalanobis distance detection method and the characteristic cosine distance detection method to track and calculate at least one tracking target ob, and the Mahalanobis distance detection method uses To confirm and obtain the movement track information of at least one tracking object ob, the characteristic cosine distance detection method is used to confirm and obtain the appearance identity information of at least one tracking object ob in the movement.

Specifically, the present invention proposes a multi-target tracking detection method that is performed when the temperature of the group of people captured by the thermal imager 10 exceeds a set temperature, which mainly uses the motion index, Mahalanobis distance (Mahalanobis distance) to calculate and predict Kalman The distance between the state and the newly arrived measurement value. (Kalman filter can predict the position coordinates and speed of the tracking target ob through the observation sequence of the tracking target ob's position), the first level of correlation uses the cosine distance as the cost function, but sets two threshold constraints of the Mahalanobis distance and the cosine distance; The second level of association attempts to associate unconfirmed and unmatched trajectories, as shown in the following formula:

d ⁽¹⁾ ( i,j )=( d _j -y _i ) ^T S _i ^-1 ( d _j -y _i ) (1)

Among them, (yi, Si) represents the projection of the i-th trajectory distribution into the measurement space, and dj represents the j-th detection bounding box. The Mahalanobis distance considers the uncertainty of the state estimation by measuring the standard deviation of the distance between the detection and the average trajectory position. The present invention also uses appearance indicators to prevent a large number of unmatched Mahalanobis distances when the uncertainty of the object movement is high (the uncertainty of the Kalman filter prediction increases, the correlation between the states becomes smaller, and the covariance matrix The inverse matrix becomes larger, which makes the motion index invalid. Therefore, importing the appearance index, as the name suggests, is to compare the similarity of the image between the frame predicted by the Kalman filter and the frame obtained by the target detection of the next frame, and the image The similarity is to embed the image into a 128d space through CNN and limit the length to 1, and then calculate the cosine distance, as shown in the following formula:

Finally, the two metrics are weighted by a weight, as shown in the following formula:

c _i,j =λ d ⁽¹⁾ ( i,j )+(1-λ) d ⁽²⁾ ( i,j ) (3)

The two detection methods of Mahalanobis distance and cosine distance have their own advantages. They are matched with each other to adjust the weight of the combination on tasks in different environments. Finally, after the union matching, the unsigned tracking and unconfirmed targets are separated and calculated The updated time is 1 and the tracking target is negotiated using the Union Intersection (IoU) method. Use faster-RCNN (or other detection algorithms) to detect the target in the first frame --> Pass the Kalman filter to predict the trajectory state of the next frame (u, v, r, h) --> Use faster- RCNN detects the target in the second frame-->matches the detected target in the second frame with the predicted trajectory state. For example, if the two IOUs are close to 1, it means that the target in the previous frame corresponds to this frame successfully.

Please refer to Figures 3, 4 and 6, which is a second embodiment of the multi-target tracking application of the present invention. The information processing unit 20 includes an image recognition module 21. The group 21 uses an image positioning method to track and calculate the movement trajectory information of the tracking target ob. The image recognition module 21 creates a block parameter database 210. The block parameter database 210 is set with a plurality of frames arranged in a grid. Each block is set with at least one standard parameter data. The image recognition module 21 inputs the current thermal image of the human body and performs image recognition processing after analyzing and processing the color temperature distribution to calculate the center of gravity position of the tracking target ob , And then substitute the center of gravity position into the block parameter database 210 to interpret the coordinate parameter data of the block where the center of gravity position is located to generate the corresponding coordinate position information; when the image recognition module 21 inputs the next frame of human body thermal image , The steps of the image positioning method are repeated to obtain the coordinate position information of the tracking target ob of the thermal image of the human body in the next frame. When the thermal image of the human body input by the image recognition module 21 reaches the preset number of frames, the image positioning method is repeated The step of obtaining the coordinate position information of at least one tracking target ob of the last thermal image of the human body, and the movement trajectory information and movement of the tracking target ob can be obtained by the coordinate position information of each tracking target ob of consecutive frames Speed information and moving direction information.

Please refer to Figures 1, 3, 4 and 6, which is the third embodiment of the multi-target tracking application of the present invention. This embodiment further includes a first image capture set in the area A close to the thermal imager 10. The capturing device 40 can perform continuous image capturing of a person walking on the moving line L1, and use the continuous imaging as a plurality of frames of the first human body image roughly in line with the field of view of the thermal image of the human body. The information processing unit 20 includes an image recognition module 21. The image recognition module 21 uses an image positioning method to track and calculate the movement trajectory information of the tracking target ob. The image recognition module 21 creates a block parameter database 210. The block parameter database 210 is set with a plurality of blocks arranged in a grid, and each block is set with at least one standard parameter data. The image recognition module 21 inputs the current thermal image of the human body and the color temperature distribution After analysis and processing, image recognition is performed to calculate the center of gravity position of the tracking target ob position, and then the center of gravity position is substituted into the block parameter database 210 to interpret the coordinate parameter data of the block where the center of gravity position is located to generate the corresponding coordinates Position information; the image recognition module 21 inputs the current first human image and then processes the image as a foreground image, and substitutes the foreground image into the block parameter database 210, and then based on the obtained coordinate position information of the tracking target ob in Select the first tracking image corresponding to the tracking target ob from the foreground image frame; when the image recognition module 21 inputs the next thermal image of the human body, it determines whether a new tracking target ob is generated. If the determination result is no, the image The identification module 21 inputs the first tracking image of the next frame, and substitutes the first tracking image of the next frame into the block parameter database 210 to obtain the real-time coordinate position information of the first tracking image, and then repeat the above Step to obtain the coordinate position information of the first tracking image of the last frame, and obtain the movement trajectory information and the movement speed information of the at least one tracking target ob from the coordinate position information of the first tracking images of different frames And moving direction information.

Continuing from the above-mentioned embodiment, as shown in FIGS. 1 and 6, this embodiment further includes at least one second image capturing device 41, which is arranged at a position of the area A relative to the first image capturing device 40. Continuous image capture is performed for the person walking on the moving line L1, and the continuous imaging is used as a plurality of frames of second human body images. The image recognition module 21 respectively performs image processing on the plurality of frames of second human body images as second tracking images, and then Integrate the first tracking image and the second tracking image through a Bayesian network to form a target detection probability distribution, which is used to detect a new tracking target ob and provide a continuous first tracking image and a second tracking image Corresponding correlation characteristics in the time domain.

Specifically, the target detection probability distribution is the probability G1(X) that makes the new tracking target ob easy to be detected, and the probability G2 that provides the corresponding correlation characteristics in the time domain between the continuous first tracking image and the second tracking image. The sum of (X), where X is the position in area A.

Please refer to FIG. 6, which is a fourth multi-target tracking application embodiment of the present invention. This embodiment further includes at least one unmanned aircraft 50. The unmanned aircraft 50 is configured to include a third image capturing device 51 and a A satellite positioning module 52 for generating position signals, a power output unit 54 for controlling the flight state of the unmanned aircraft, and a flight control module 53 for controlling the operation of the power output unit 54. The image recognition module 21 establishes a personal parameter database 211 for storing plural personal data. The flight control module 53 is connected to the information processing unit 20 via a mobile communication system 60. When a warning signal is generated, the information processing unit 20 transmits the flight command corresponding to the movement trajectory information to the flight via the mobile communication system 60. The control module 53 is used to The remote-controlled unmanned aircraft 50 tracks and tracks the target ob. When the unmanned aircraft 50 arrives near the tracking target ob, the third image capturing device 51 is used to continuously capture images of the tracking target ob to form multiple frames of monitoring images, and multiple frames of monitoring images and location signals It is sequentially transmitted to the information processing unit 20 through the mobile communication system 60, and then the image recognition module 21 performs image recognition processing, and substitutes the characteristic data of the image recognition result into the personal parameter database 211 to search for matching personal data. When the search result is yes, the personal data, feature data, location signal and the corresponding time axis will be recorded in one of the personal file databases of the personal parameter database 211; when the search result is no, the personal data will be added as pending File, and then record the personal data, feature data, location signal and corresponding time axis to be filed in the second personal parameter database 211. Wherein, the information processing unit 20 is electrically connected to a display screen 22. When the information processing unit 20 receives a plurality of frames of monitoring images, it displays a plurality of frames of monitoring images continuously played on the display screen 22 to control the drone 50 to track at least 1. The basis for tracking the target ob.

As mentioned above, the characteristic data may be at least one of the age, gender, or clothing of the tracking target. When the personal data of the tracking target ob is added as a file to be created, the image recognition module 21 will identify the age, At least one characteristic data of gender and clothing (such as wearing a hat; or clothing color) is output as a corresponding warning signal, and the corresponding warning information is played by the warning device 30 in a voice mode. In addition, the warning device 30 can use The audio broadcast informs the tracking target ob (that is, the subject) to a predetermined point for further temperature screening.

Through the description of the above specific embodiments, the present invention is indeed a multi-target tracking detection method for the population temperature captured by the thermal imager exceeds the set temperature. It is suitable for multiple specific targets in the crowd and is used in the preparation step. At the time, confirm that the thermal imager lens has been heated and set the external temperature compensation; when tracking the measurement step, predefine a detection line through the measurement line, and use the algorithm to estimate the image returned by the thermal imager Analyze its color temperature distribution and perform temperature labeling, and then select multiple specific objects through the lens of other tracking image capture devices to perform multiple specific target tracking to prevent too many people from losing tracked objects.

The above are only feasible embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent implementation of other changes based on the content, characteristics and spirit of the following claims shall be It is included in the scope of the patent of the present invention. The structural features of the invention specifically defined in the claim are not found in similar articles, and are practical and progressive. They have already met the requirements of a patent for invention. The application is filed in accordance with the law. I would like to request that the Bureau of Junction approve the patent in accordance with the law to protect this The legitimate rights and interests of the applicant.

10: Thermal imager

40: The first image capture device

41: Second image capture device

A: area

L1: Moving line

Claims (8)

A multi-target human body temperature tracking system, which includes: a thermal imager, which is set in an area with a moving line, and can perform continuous image capture of a person walking on the moving line, and the continuous imaging is a plurality of frames of human body heat Image; an information processing unit for analyzing the color temperature distribution of the currently captured thermal image of the human body to determine whether the body temperature of at least one person walking on the moving line exceeds the preset body temperature value, when the result of the determination is If yes, set the at least one person in the thermal image of the human body as the at least one tracking target, and mark the at least one tracking target; then perform the color temperature distribution analysis processing on the thermal image of the human body in the next frame to track and calculate Output the movement trajectory information of the at least one tracking target, and determine whether the movement trajectory information moves in a direction with the moving line, and if the judgment result is yes, output a warning signal; and a warning device for the warning signal The output is warning information representing tracking and paying attention to the tracking target; among them, it further includes a first image capturing device and at least one second image capturing device located near the thermal imager in the area, which can respond to the moving line The walking person performs continuous image capture to form a plurality of frames of the first human body image roughly in line with the field of view of the thermal image of the human body; the information processing unit includes an image recognition module, and the image recognition module is An image positioning method is used to track and calculate the movement trajectory information of the at least one tracking target. The image recognition module establishes a block parameter database. The block parameter database is set with a plurality of blocks arranged in a frame. Each of the blocks is set with at least one piece of standard parameter data, the image recognition module inputs the current thermal image of the human body and performs image recognition processing after the analysis and processing of the color temperature distribution to calculate the center of gravity of the at least one tracking target position Position, and then substitute the at least one center of gravity position into the block parameter database to interpret the coordinate parameter data of the block where the at least one center of gravity position is located to generate corresponding coordinate position information; the image recognition model After inputting the current first human body image, image processing is performed as a foreground image, and the foreground image is substituted into the block parameter database, and then based on the obtained coordinate position information of the at least one tracking target before The scene image frame selects the first tracking image corresponding to the at least one tracking target; the second image capturing device is set at the position of the area relative to the first image capturing device, and can be performed on the person walking on the moving line Continuous image capture uses continuous imaging as a plurality of frames of second human body images. The image recognition module separately performs image processing on the plurality of frames of second body images into second tracking images, and then passes through a Bayesian network (Bayesian network). ) Integrating the first tracking image and the second tracking image to form a target detection probability distribution for detecting a new tracking target and providing a continuous time domain between the first tracking image and the second tracking image The corresponding correlation characteristics of the target; the target detection probability distribution is the probability G1(X) that makes the new tracking target easy to be detected and provides continuous correspondence between the first tracking image and the second tracking image in the time domain The sum of the probability of characteristics G2(X), where X is the position in the area. The multi-target human body temperature tracking system according to claim 1, wherein the mark is selected from one of a frame used to frame the tracking target and a temperature mark of the tracking target, and the color of the frame is the same as that of the tracking The color of the target's highest color temperature matches. The multi-target human body temperature tracking system according to claim 1, wherein the information processing unit uses a Mahalanobis distance detection method and a characteristic cosine distance detection method to track and calculate the at least one tracking target, and the Mahalanobis distance detection method uses To confirm and obtain the movement trajectory information of the at least one tracking target, the characteristic cosine distance detection method is used to confirm and obtain the appearance identity information of the at least one tracking target in movement. The multi-target human body temperature tracking system according to claim 1, wherein the information processing unit includes an image recognition module, and the image recognition module uses an image positioning method to track and calculate the movement trajectory of the at least one tracking target Message, the image recognition module creates a block parameter database, the block parameter database is set with a plurality of blocks arranged in a frame, and each block is set with at least one standard parameter data, the image recognition module The group inputs the current thermal image of the human body and performs image recognition processing after the analysis and processing of the color temperature distribution to calculate the center of gravity position of the at least one tracking target, and then substitute the at least one center of gravity position into the block parameter database To Interpret the coordinate parameter data of the block where the at least one center of gravity is located to generate corresponding coordinate position information; when the image recognition module inputs the next frame of the thermal image of the human body, the steps of the image positioning method are repeated To obtain the coordinate position information of the at least one tracking target of the thermal image of the human body in the next frame, and repeat the steps of the image positioning method when the thermal image of the human body input by the image recognition module reaches the preset number of frames The coordinate position information of the at least one tracking target of the last frame of the thermal image of the human body is obtained, and the movement of the at least one tracking target can be obtained by the coordinate position information of each tracking target of different frames Trajectory information, moving speed information and moving direction information. For example, in the multi-target human body temperature tracking system described in claim 1, when the image recognition module inputs the next frame of the human body thermal image, it determines whether there is a new tracking target, and when the determination result is no, the image recognition module Group input the next frame of the first tracking image, and substitute the next frame of the first tracking image into the block parameter database to obtain the real-time coordinate position information of the first tracking image, and then repeat the above Step to obtain the coordinate position information of the first tracking image in the last frame, and obtain the movement trajectory of the at least one tracking target by using the coordinate position information of each of the first tracking images of different frames Information, moving speed information, and moving direction information. The multi-target human body temperature tracking system according to claim 4, which further includes at least one unmanned aircraft, and the unmanned aircraft configuration includes a third image capturing device, a satellite positioning module for generating position signals, and The power output unit for controlling the flight status of the unmanned aircraft and a flight control module for controlling the operation of the power output unit; the image recognition module establishes a personal parameter database for storing plural personal data; the flight control The module is connected to the information processing unit signal by a mobile communication system. When the warning signal is generated, the information processing unit transmits the flight command corresponding to the movement track information to the flight control module through the mobile communication system , Used to remotely control the unmanned aircraft to track the at least one tracking target, when the unmanned aircraft When the machine reaches the vicinity of the at least one tracking target, the third image capture device is used to continuously capture the at least one tracking target and image it as a plurality of frames of monitoring images, and the plurality of frames of monitoring images and the location The position signal is sequentially transmitted to the information processing unit through the wireless communication system, and then the image recognition module performs image recognition processing, and substitutes the characteristic data of the image recognition result into the personal parameter database to search for matching The personal data, when the search result is yes, the personal data, the characteristic data, the location signal and the corresponding time axis will be recorded in one of the personal file databases of the personal parameter database; when the search result is no, the The personal data is added as a file to be created, and the personal data to be filed, the characteristic data, the location signal and the corresponding time axis are recorded in the second personal parameter database. The multi-target human body temperature tracking system according to claim 6, wherein the information processing unit is electrically connected to a display screen, and when the information processing unit receives the plurality of frames of monitoring images, it displays continuously played The multiple frames of monitoring images are used as a basis for controlling the unmanned aircraft to track the at least one tracking target. The multi-target human body temperature tracking system according to claim 6, wherein the characteristic data is selected from at least one of the age, gender, and clothing of the at least one tracking target, and when the personal data of the at least one tracking target is added When the file is to be created, the image recognition module outputs at least one of the characteristic data of age, gender, and clothing as the corresponding warning signal, and the warning device broadcasts the corresponding warning information by voice.

Images