CN111914635B - Human body temperature measurement method, device, system and electronic equipment - Google Patents

Human body temperature measurement method, device, system and electronic equipment Download PDF

Info

Publication number
CN111914635B
CN111914635B CN202010583724.9A CN202010583724A CN111914635B CN 111914635 B CN111914635 B CN 111914635B CN 202010583724 A CN202010583724 A CN 202010583724A CN 111914635 B CN111914635 B CN 111914635B
Authority
CN
China
Prior art keywords
infrared
visible light
frame
video frame
visible
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010583724.9A
Other languages
Chinese (zh)
Other versions
CN111914635A (en
Inventor
刘伟舟
胡晨
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Megvii Technology Co Ltd
Original Assignee
Beijing Megvii Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Megvii Technology Co Ltd filed Critical Beijing Megvii Technology Co Ltd
Priority to CN202010583724.9A priority Critical patent/CN111914635B/en
Publication of CN111914635A publication Critical patent/CN111914635A/en
Application granted granted Critical
Publication of CN111914635B publication Critical patent/CN111914635B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/49—Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J5/00—Radiation pyrometry, e.g. infrared or optical thermometry
    • G01J5/0022—Radiation pyrometry, e.g. infrared or optical thermometry for sensing the radiation of moving bodies
    • G01J5/0025—Living bodies
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J5/00—Radiation pyrometry, e.g. infrared or optical thermometry
    • G01J2005/0077—Imaging

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Computing Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Radiation Pyrometers (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a human body temperature measurement method, a device, a system and electronic equipment. Wherein the method comprises the following steps: obtaining a visible light video frame sequence and an infrared video frame sequence corresponding to the same view field; performing object detection on the visible light video frame sequence to obtain a visible light video frame marked with a visible object detection frame, and performing object detection and tracking on the infrared video frame sequence to obtain an infrared video frame marked with an infrared object detection frame and infrared track information of the same object; and selecting a high-quality detection frame corresponding to the first object based on the infrared track information of the first object, and searching out a target object detection frame matched with the high-quality detection frame from the visible object detection frames marked in the visible light video frame to determine the temperature of the first object. The invention carries out human body temperature measurement based on the visible light video frame sequence and the infrared video frame sequence, ensures the accuracy of temperature measurement, does not need staff to approach a measured object to carry out temperature measurement, and reduces the labor cost to a certain extent.

Description

Human body temperature measurement method, device, system and electronic equipment
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to a method, an apparatus, a system, and an electronic device for measuring temperature of a human body.
Background
The remote human body temperature measurement generally adopts an infrared temperature measurement technology, and a specific area in a human body image is acquired based on an infrared camera, and the maximum temperature in the specific area is taken as the human body temperature.
In order to improve the accuracy of remote temperature measurement, the related technology generally adopts a technology combining visible light and infrared temperature measurement, in the temperature measurement mode, a target area is determined on a visible image through an object detection technology, then an infrared area corresponding to the target area is found in an infrared image, and the temperature of the infrared area is used as a temperature measurement result. Because the infrared image and the visible light image are difficult to be precisely aligned in the airspace, the infrared region and the target region have deviation, and therefore the temperature measurement accuracy is poor.
Disclosure of Invention
Accordingly, the present invention is directed to a method, apparatus, system and electronic device for measuring temperature of a human body, so as to alleviate the above technical problems.
In a first aspect, an embodiment of the present invention provides a method for measuring temperature of a human body, where the method is applied to an electronic device, and includes: obtaining a visible light video frame sequence and an infrared video frame sequence corresponding to the same view field; the visible light video frame sequence and the infrared video frame sequence are aligned in a time domain and a space domain; performing object detection on the visible light video frame sequence to obtain a visible light video frame marked with a visible object detection frame; wherein the object comprises a human face and/or a human body; performing object detection and tracking on the infrared video frame sequence to obtain an infrared video frame marked with an infrared object detection frame and infrared track information of the same object; the infrared track information is an infrared video frame sub-sequence containing the same detection frame identification; taking the object corresponding to each piece of infrared track information as a first object respectively, and executing the following operations for each first object: selecting a high-quality detection frame corresponding to the first object based on each infrared object detection frame corresponding to the infrared track information of the first object; searching a target object detection frame matched with the high-quality detection frame from visible object detection frames marked in the visible light video frame; the temperature of the first object is determined based on the target object detection frame and the quality detection frame.
With reference to the first aspect, an embodiment of the present invention provides a first possible implementation manner of the first aspect, where the step of acquiring a visible light video frame sequence and an infrared video frame sequence corresponding to the same field of view includes: the video stream is acquired from the same view field through a visible light camera and an infrared camera, so that a visible light original video stream and an infrared original video stream are obtained; and performing time domain and space domain alignment on the visible light original video stream and the infrared original video stream to obtain a visible light video frame sequence and an infrared video frame sequence.
With reference to the first possible implementation manner of the first aspect, the embodiment of the present invention provides a second possible implementation manner of the first aspect, where the step of performing temporal and spatial alignment on the visible original video stream and the infrared original video frame sequence to obtain the visible video frame sequence and the infrared video frame sequence includes: determining a time domain alignment mode based on the frame rates of the visible light camera and the infrared camera; determining an airspace alignment mode based on space coordinates corresponding to positions of the visible light camera and the infrared camera; and performing time domain and space domain alignment on the visible light original video stream and the infrared original video stream according to the determined time domain alignment mode and space domain alignment mode to obtain a visible light video frame sequence and an infrared video frame sequence.
With reference to the first aspect, an embodiment of the present invention provides a third possible implementation manner of the first aspect, where the step of performing object detection on the sequence of visible light video frames to obtain a visible light video frame marked with a visible object detection frame includes: performing object detection on the visible light video frame sequence through a pre-trained first detection model to obtain a visible light video frame marked with a visible object detection frame; the method for detecting and tracking the infrared video frame sequence to obtain the infrared video frame marked with the infrared object detection frame and the infrared track information of the same object comprises the following steps: performing object detection on the infrared video frame sequence through a pre-trained second detection model to obtain an infrared video frame marked with an infrared object detection frame; and tracking each infrared video frame to obtain infrared track information of the same object.
With reference to the first aspect, an embodiment of the present invention provides a fourth possible implementation manner of the first aspect, where the step of selecting, based on each infrared object detection frame corresponding to the infrared track information of the first object, a high-quality detection frame corresponding to the first object includes: and evaluating each infrared object detection frame corresponding to the infrared track information of the first object through a pre-trained infrared detection frame quality evaluation model, and determining a high-quality detection frame corresponding to the first object based on the evaluation score.
With reference to the fourth possible implementation manner of the first aspect, an embodiment of the present invention provides a fifth possible implementation manner of the first aspect, wherein the step of determining, based on the evaluation score, a quality detection frame corresponding to the first object includes: taking the infrared object detection frame with the highest evaluation score as a high-quality detection frame corresponding to the first object; or taking the infrared object detection frame with the evaluation score larger than the preset quality threshold as the high-quality detection frame corresponding to the first object.
With reference to the first aspect, an embodiment of the present invention provides a sixth possible implementation manner of the first aspect, where the step of searching for a target object detection frame matching with the quality detection frame from the visible object detection frames marked in the visible video frame includes: determining a first frame identification of an infrared video frame where a high-quality detection frame is located; searching a target visible light video frame with a frame identifier being a first frame identifier from the visible light video frames; calculating IOU values of the high-quality detection frames and the detection frames of all visible objects in the target visible light video frame; and taking the visible object detection frame corresponding to the maximum IOU value as a target object detection frame.
With reference to the first aspect, an embodiment of the present invention provides a seventh possible implementation manner of the first aspect, where the method further includes: tracking each visible light video frame to obtain visible light track information of the same object; the visible light track information is a visible light video frame sub-sequence containing the same detection frame mark; the step of searching for a target object detection frame matched with the high-quality detection frame from the marked visible object detection frames in the visible light video frame comprises the following steps: searching target visible light track information matched with the infrared track information of the first object from the visible light track information of each same object; and selecting a target object detection frame from the visible object detection frames corresponding to the target visible light track information.
With reference to the seventh possible implementation manner of the first aspect, the embodiment of the present invention provides an eighth possible implementation manner of the first aspect, wherein the step of searching for target visible light track information matched with infrared track information of the first object from visible light track information of each same object includes: scoring the matching degree of a visible object detection frame corresponding to the visible light track information of each same object and an infrared object detection frame corresponding to the infrared track information of the first object; and taking the visible light track information with the highest matching degree scoring value as target visible light track information matched with the infrared track information of the first object.
With reference to the seventh possible implementation manner of the first aspect, an embodiment of the present invention provides a ninth possible implementation manner of the first aspect, where the method further includes:
determining a spatial correction relationship of the visible video frame sequence and the infrared video frame sequence based on the visible light track information and the infrared track information of the first object;
the step of selecting a target object detection frame from the visible object detection frames corresponding to the target visible light track information comprises the following steps: correcting each visible object detection frame corresponding to the target visible light track information according to the space correction relation to obtain a visible object correction detection frame group; calculating IOU values of the high-quality detection frames and the visible object correction detection frames in the visible object correction detection frame group; and taking the visible object correction detection frame corresponding to the maximum IOU value as a target object detection frame.
With reference to the first aspect, an embodiment of the present invention provides a tenth possible implementation manner of the first aspect, wherein the step of determining the temperature of the first object based on the target object detection frame and the quality detection frame includes: performing key point detection on the target object detection frame to obtain visible light coordinate information containing a local object region; the local object area is a forehead area or a wrist area; and determining a sub-detection area in the high-quality detection frame based on the visible light coordinate information, and determining the temperature corresponding to the sub-detection area as the temperature of the first object.
With reference to the first aspect, an embodiment of the present invention provides an eleventh possible implementation manner of the first aspect, where the method further includes: based on the characteristic information of the target object detection frame, searching the identity information of the first object in a pre-stored portrait database; and determining the corresponding relation between the retrieved identity information and the temperature of the first object.
In a second aspect, an embodiment of the present invention further provides a human body temperature measurement device, where the device is applied to an electronic apparatus, and includes: the acquisition module is used for acquiring a visible light video frame sequence and an infrared video frame sequence corresponding to the same view field; the visible light video frame sequence and the infrared video frame sequence are aligned in a time domain and a space domain; the detection module is used for carrying out object detection on the visible light video frame sequence to obtain a visible light video frame marked with a visible object detection frame; wherein the object comprises a human face and/or a human body; the detection tracking module is used for detecting and tracking the infrared video frame sequence to obtain an infrared video frame marked with an infrared object detection frame and infrared track information of the same object; the infrared track information is an infrared video frame sub-sequence containing the same detection frame identification; the execution module is used for taking the object corresponding to each piece of infrared track information as a first object respectively, and executing the following operations for each first object: the selection module is used for selecting a high-quality detection frame corresponding to the first object based on each infrared object detection frame corresponding to the infrared track information of the first object; the searching module is used for searching a target object detection frame matched with the high-quality detection frame from the visible object detection frames marked in the visible light video frame; and the first determining module is used for determining the temperature of the first object based on the target object detection frame and the high-quality detection frame.
In a third aspect, the embodiment of the invention further provides a human body temperature measurement system, which comprises a server and a camera set, wherein the camera set comprises a visible light camera and an infrared camera; the camera group is used for respectively acquiring a visible light video sequence and an infrared video sequence corresponding to the same field of view through a visible light camera and an infrared camera; the server comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the computer program to realize the steps of the human body temperature measuring method.
In a fourth aspect, an embodiment of the present invention further provides an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, where the steps of the above-mentioned human body temperature measurement method are implemented when the processor executes the computer program.
In a fifth aspect, an embodiment of the present invention further provides a computer readable storage medium, where a computer program is stored on the computer readable storage medium, where the computer program performs the steps of the above-mentioned method for measuring a temperature of a human body when the computer program is run by a processing device.
The embodiment of the invention has the following beneficial effects:
according to the human body temperature measurement method, the human body temperature measurement device and the electronic equipment, the visible light video frame sequence and the infrared video frame sequence corresponding to the same view field are obtained; performing object detection on the visible light video frame sequence to obtain a visible light video frame marked with a visible object detection frame, and performing object detection and tracking on the infrared video frame sequence to obtain an infrared video frame marked with an infrared object detection frame and infrared track information of the same object; and selecting a high-quality detection frame corresponding to the first object based on the infrared track information of the first object, and searching a target object detection frame matched with the high-quality detection frame from the visible object detection frames marked in the visible light video frame, so as to further determine the temperature of the first object. According to the temperature measurement mode, the visible light video frame and the infrared video frame are combined and applied, so that the same target object in the same area can be tracked, and the temperature of the same object is determined based on track information obtained by tracking and a high-quality detection frame of the same object in the infrared video frame and a target object detection frame in the visible light video frame, so that the accuracy of temperature measurement is effectively improved.
Additional features and advantages of the disclosure will be set forth in the description which follows, or in part will be obvious from the description, or may be learned by practice of the techniques of the disclosure.
The foregoing objects, features and advantages of the disclosure will be more readily apparent from the following detailed description of the preferred embodiments taken in conjunction with the accompanying drawings.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are needed in the description of the embodiments or the prior art will be briefly described, and it is obvious that the drawings in the description below are some embodiments of the present invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic structural diagram of an electronic device according to an embodiment of the present invention;
FIG. 2 is a flow chart of a method for measuring temperature of a human body according to an embodiment of the present invention;
FIG. 3 is a flowchart of another method for measuring temperature of a human body according to an embodiment of the present invention;
FIG. 4 is a flowchart of another method for measuring temperature of a human body according to an embodiment of the present invention;
FIG. 5 is a flowchart of another method for measuring temperature of a human body according to an embodiment of the present invention;
FIG. 6 is a flowchart of a spatial correction method according to an embodiment of the present invention;
FIG. 7 is a flowchart of another method for measuring temperature of a human body according to an embodiment of the present invention;
FIG. 8 is a flowchart of another method for measuring temperature of a human body according to an embodiment of the present invention;
FIG. 9 is a schematic diagram of a human body temperature measurement device according to an embodiment of the present invention;
FIG. 10 is a schematic diagram of another embodiment of a device for measuring temperature of a human body;
FIG. 11 is a schematic diagram of a human body temperature measurement system according to an embodiment of the present invention;
fig. 12 is a schematic diagram of a detection frame based on spatial coordinate alignment according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The method of using an infrared camera to measure temperature is often used at entrances of communities, subways, markets, etc. to measure the body temperature of a person entering a target area for reference by a worker. In order to effectively improve the temperature measurement efficiency and accuracy, the embodiment of the invention provides a human body temperature measurement method, device, system and electronic equipment, the technology can be applied to personnel temperature measurement in public places with dense personnel, and also can be applied to personnel temperature measurement in places with relatively sparse personnel such as communities, by combining a visible light video frame and an infrared video frame, not only can a plurality of target objects be accurately measured, but also the temperature measurement efficiency is improved to a certain extent without the need of a worker to be close to the measured target temperature measurement, and the temperature measurement method, the device and the system are described below through the embodiment.
As shown in fig. 1, an electronic device 100 includes one or more processors 102, one or more memories 104, an input device 106, an output device 108, and one or more image capture devices 110, which are interconnected by a bus system 112 and/or other forms of connection mechanisms (not shown). It should be noted that the components and structures of the electronic device 100 shown in fig. 1 are exemplary only and not limiting, as electronic devices may have other components and structures as desired.
The processor 102 may be a server, a smart terminal, or a device that includes a Central Processing Unit (CPU) or other form of processing unit having data processing and/or instruction execution capabilities, may process data from other components in the electronic device 100, and may control other components in the electronic device 100 to perform body temperature measurement functions.
Memory 104 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. Volatile memory can include, for example, random Access Memory (RAM) and/or cache memory (cache) and the like. The non-volatile memory may include, for example, read Only Memory (ROM), hard disk, flash memory, and the like. One or more computer program instructions may be stored on a computer readable storage medium and executed by the processor 102 to perform functions in the embodiments of the present invention (implemented by a processing device) below and/or other desired functions. Various applications and various data, such as visible and infrared video sequences, as well as various data used and/or generated by the applications, etc., may also be stored in the computer readable storage medium.
The input device 106 may be a device used by a user to input instructions and may include one or more of a keyboard, mouse, microphone, touch screen, and the like.
The output device 108 may output various information (e.g., images or sounds) to the outside (e.g., a user), and may include one or more of a display, a speaker, and the like.
The image capture device 110 may capture a visible light video sequence and an infrared video sequence and store the captured video sequences in the memory 104 for use by other components.
Illustratively, the devices in the electronic apparatus for implementing the human body temperature measurement method and apparatus according to the embodiment of the present invention may be integrally disposed, or may be separately disposed, such as integrally disposing the processor 102, the memory 104, the input device 106 and the output device 108, and disposing the image capturing device 110 at a designated position where the video frame may be captured. When the devices in the above electronic apparatus are integrally provided, the electronic apparatus may be implemented as an intelligent terminal such as a camera, a smart phone, a tablet computer, a vehicle-mounted terminal, or the like.
The embodiment provides a human body temperature measurement method, which is applied to the electronic equipment, and is shown in a flow chart of the human body temperature measurement method in fig. 2, and the method specifically comprises the following steps:
Step S202, a visible light video frame sequence and an infrared video frame sequence corresponding to the same field of view are obtained; the visible light video frame sequence and the infrared video frame sequence are aligned in a time domain and a space domain;
the field of view represents the maximum range that can be observed by the camera, usually expressed in terms of angle, the larger the field of view, the larger the range of observation. The same view field can be understood as that the view field angles of the acquisition devices corresponding to the visible light video frame sequence and the infrared video frame sequence are the same, and the video frames are the video frames of the same region.
In order to achieve a better temperature measurement effect, in this embodiment, the visible light video frame sequence and the infrared video frame sequence are video frame sequences after performing time domain and space domain alignment processing in advance; the number of the video frames contained in the time domain aligned visible light video frame sequence and the infrared video frame sequence is the same, namely the visible light video frames and the infrared video frames in the visible light video frame sequence and the infrared video frame sequence are in one-to-one correspondence, and the visible light video frames and the infrared video frames which are mutually corresponding have the same frame identification. The sequence of visible light video frames and the sequence of infrared video frames after spatial domain alignment can be understood that the visible light video frames and the infrared video frames can be unified into the same coordinate system through coordinate transformation, and then the coordinates of the position in one frame are obtained from the appointed position in the other frame.
Step S204, performing object detection on the visible light video frame sequence to obtain a visible light video frame marked with a visible object detection frame; wherein the object comprises a human face and/or a human body and the like;
the object may be a human face object or a human body object, or other parts of a human body, and for convenience of explanation, the human face object is taken as an example for explanation; when the face object is detected on the visible light video frame sequence, the face detection frame of each face object can be marked on the visible light video frame comprising the face object, for example, the visible light video frames of the face detection frame comprising the face object 1 in the visible light video frame sequence have 5 frames, and the frame identifiers of the corresponding visible light video frames are 1, 2, 3, 4 and 5; the visible light video frames of the face detection frame containing the face object 2 have 4 frames in total, and the frame identifiers of the visible light video frames respectively corresponding to the visible light video frames are 3, 4, 5 and 6. The frame identification may be used to identify the video frame in the form of numbers or other forms, and is not limited herein.
Step S206, performing object detection and tracking on the infrared video frame sequence to obtain an infrared video frame marked with an infrared object detection frame and infrared track information of the same object; the infrared track information is an infrared video frame sub-sequence containing the same detection frame identification;
The infrared track information not only comprises an infrared video frame sub-sequence of the same object, but also comprises a frame identifier of a video frame where the infrared object detection frame is located and coordinate information of the infrared object detection frame; the coordinate information may include a relative position to origin coordinates set in advance in the infrared video frame.
Similarly, when the face object detection and tracking are performed on the infrared video frame sequence, the face detection frame of each face object can be marked on the infrared video frame comprising the face object, and the infrared track information corresponding to each face object can be obtained. It will be appreciated that infrared object detection frames in different video frames that are identified as corresponding to the same face object by tracking the face object have the same detection frame identification.
The infrared track information can be represented by frame identification of an infrared video frame and coordinate information of an infrared object detection frame. For example, in the above-mentioned infrared video frame sequence, the total number of the infrared video frames of the face detection frame containing the face object 1 is 5, and the frame identifiers of the respectively corresponding infrared video frames are 1, 2, 3, 4, 5, in order to accurately find the coordinate information of each infrared object detection frame, in this embodiment, the coordinate information of the upper left corner position point and the lower right corner position point of the object detection frame can be selected; specifically, the coordinate information of the infrared object detection frame of the face object 1 in these 5 frames is [ a1 (x) a1 ,y a1 ),b1(x b1 ,y b1 )]，[a2(x a2 ,y a2 ),b2(x b2 ,y b2 )]，[a3(x a3 ,y a3 ),b3(x b3 ,y b3 )]，[a4(x a4 ,y a4 ),b4(x b4 ,y b4 )]，[a5(x a5 ,y a5 ),b5(x b5 ,y b5 )](a 1-a5 represent the upper left corner position point of the infrared object detection frame, b1-b5 represent the lower right corner position point, x a1- x a5 An abscissa, y, representing a point of the upper left corner of each infrared object detection frame a1- y a5 Representing the ordinate, x, of the point at the upper left corner of each infrared object detection frame b1- x b5 An abscissa, y, representing a point at the lower right corner of each infrared object detection frame b1- y b5 Representing the ordinate of the lower right corner position point of each infrared object detection frame), the infrared track information of the face object 1 can be represented as an infrared video frame sub-sequence formed by frames 1, 2, 3, 4 and 5 or an infrared video frame sub-sequence formed by the matting of the face object 1 at the positions of the infrared object detection frames in the frames 1, 2, 3, 4 and 5; the infrared video frames of the face detection frame containing the face object 2 have 4 frames, the frame identifiers of the infrared video frames corresponding to the frames are 3, 4, 5 and 6, and the coordinate information of the infrared object detection frame of the face object 2 in the 4 frames is [ a6 (x a6 ,y a6 )，b6(x b6 ,y b6 )]，[a7(x a7 ,y a7 )，b7(x b7 ,y b7 )]，[a8(x a8 ,y a8 )，b8(x b8 ,y b8 )]，[a9(x a9 ,y a9 )，b9(x b9 ,y b9 )]The infrared track information of the face object 2 may be represented as an infrared video frame sub-sequence composed of the frame 3, the frame 4, the frame 5 and the frame 6 or an infrared video frame sub-sequence composed of the frame 3, the frame 4, the frame 5 and the frame 6 of the face object 2.
Step S208, taking the object corresponding to each piece of infrared track information as a first object, and executing the following operations in step S210 to step S214 for each first object;
step S210, selecting a high-quality detection frame corresponding to the first object based on each infrared object detection frame corresponding to the infrared track information of the first object;
in general, due to shooting environment or artificial reasons, the obtained infrared track information of the first object includes blocked or unclear infrared object detection frames, in order to accurately measure the temperature of the first object, a high-quality detection frame corresponding to the first object may be selected from the infrared video frame sub-sequence, where the high-quality detection frame is a clear infrared object detection frame of the first object without blocking, and the number of selection of the high-quality detection frames may be set according to actual needs, which is not limited herein.
Step S212, searching a target object detection frame matched with the high-quality detection frame from visible object detection frames marked in the visible light video frame;
from the visible object detection frames marked in the visible light video frames, searching a target object detection frame matched with the high-quality detection frame based on the frame identification of the video frame where the high-quality detection frame is positioned and the coordinate information of the high-quality detection frame; first, a visible light video frame with the same frame identification as the video frame where the high-quality detection frame is located is found in a visible light video frame sequence, and then, based on the coordinate information of the high-quality detection frame in the infrared video frame, a visible object detection frame matched with the high-quality detection frame in the marked visible object detection frame in the visible light video frame is taken as a target object detection frame.
Step S214, determining the temperature of the first object based on the target object detection frame and the quality detection frame.
In one example, the quality detection frame is an image in an infrared video frame, so that temperature information can be determined, and the temperature corresponding to the first object can be obtained. Based on the method, the identity of the target object detection frame can be identified to obtain the identity information of the first object, and then the temperature corresponding to the identity information of the first object can be obtained.
According to the human body temperature measurement method provided by the embodiment, the visible light video frame sequence and the infrared video frame sequence corresponding to the same field of view are obtained; performing object detection on the visible light video frame sequence to obtain a visible light video frame marked with a visible object detection frame, and performing object detection and tracking on the infrared video frame sequence to obtain an infrared video frame marked with an infrared object detection frame and infrared track information of the same object; and selecting a high-quality detection frame corresponding to the first object based on the infrared track information of the first object, and searching a target object detection frame matched with the high-quality detection frame from the visible object detection frames marked in the visible light video frame, so as to further determine the temperature of the first object. According to the temperature measurement mode, the visible light video frame and the infrared video frame are combined and applied, so that the same target object in the same area can be tracked, and the temperature of the same object is determined based on track information obtained by tracking and a high-quality detection frame of the same object in the infrared video frame and a target object detection frame in the visible light video frame, so that the accuracy of temperature measurement is effectively improved.
The embodiment of the invention also provides another human body temperature measurement method, which is realized on the basis of the embodiment; the present embodiment focuses on a specific implementation manner of acquiring a visible light video frame sequence and an infrared video frame sequence corresponding to the same field of view. As shown in fig. 3, the method for measuring the temperature of the human body in the embodiment includes the following steps:
step S302, video streams are acquired from the same view field through a visible light camera and an infrared camera, and a visible light original video stream and an infrared original video stream are obtained;
the method comprises the steps that a visible light camera and an infrared camera are regarded as a group of cameras, are placed in parallel side by side or are vertically placed at the same position side by side, and collect the same field of view; if the area range is larger, different camera groups can be arranged at different positions, and the video sequences collected by each camera group are the visible light original video stream and the infrared original video stream of the same small area.
Step S304, performing time domain and space domain alignment on the visible light original video stream and the infrared original video stream to obtain a visible light video frame sequence and an infrared video frame sequence;
taking a visible light original video stream and an infrared original video stream obtained by a camera group as an example to carry out time domain and space domain alignment explanation; specifically, the step of performing time domain and space domain alignment on the visible light original video stream and the infrared original video stream to obtain a visible light video frame sequence and an infrared video frame sequence may be performed by steps A1-A3:
A1, determining a time domain alignment mode based on frame rates of a visible light camera and an infrared camera;
in the actual shooting process, since the frame rate of the video of the infrared camera is generally lower than that of the video of the visible light camera, the number of video frames contained in the infrared original video stream is less than that of the video frames contained in the visible light original video stream, in order to ensure that the number of video frames contained in the two original video streams is the same, in this embodiment, the video frames collected by the infrared camera may be duplicated to achieve the same number as that of the video frames collected by the visible light camera, for example, a duplicated video frame is inserted at a position of every second frame or every third frame in the infrared original video stream, and the duplicated video frame may be a video of a previous frame or a video of a next frame at a duplicated insertion position.
Besides the above manner of determining the time domain alignment of the two original video sequences by using the frame rate, the time domain alignment may be performed based on the acquisition time, for example, the visible light original video stream is filtered based on the acquisition time of the infrared original video stream, and only the visible light video stream with consistent acquisition time is reserved to achieve the time domain alignment of the two original video streams.
A2, determining an airspace alignment mode based on space coordinates corresponding to positions of the visible light camera and the infrared camera;
converting the position information of the visible light camera and the infrared camera into geographic position information under a geographic coordinate system, obtaining a spatial coordinate conversion relation between the two pieces of geographic position information through the geographic position information corresponding to the visible light camera and the infrared camera, and realizing the airspace alignment based on the spatial coordinate conversion relation; the visible light video frame sequence and the infrared video frame sequence after the airspace alignment can determine the coordinates of the appointed position in the infrared video frame through the coordinates of the appointed position in the visible light video frame.
And step A3, performing time domain and space domain alignment on the visible light original video stream and the infrared original video stream according to the determined time domain alignment mode and space domain alignment mode to obtain a visible light video frame sequence and an infrared video frame sequence.
Step S306, performing object detection on the visible light video frame sequence to obtain a visible light video frame marked with a visible object detection frame; wherein the object comprises a human face or a human body;
specifically, object detection is performed on the visible light video frame sequence through a pre-trained first detection model, and a visible light video frame marked with a visible object detection frame is obtained.
The first detection model may be a visible light face detection model obtained by training a face training sample with a label on a deep neural network such as SSD (Single Shot MultiBox Detector) algorithm, retinaNet algorithm or fasterRCNN algorithm, and the face objects contained in each visible light video frame can be accurately detected by using the trained visible light face detection model, and the face objects can be marked in the visible light video frame in a box form so as to be convenient for a user to identify the face objects.
Step S308, performing object detection and tracking on the infrared video frame sequence to obtain an infrared video frame marked with an infrared object detection frame and infrared track information of the same object; the infrared track information is an infrared video frame sub-sequence containing the same detection frame identification;
the implementation of the above step S305 may be performed by steps B1 to B2:
step B1, performing object detection on the infrared video frame sequence through a pre-trained second detection model to obtain an infrared video frame marked with an infrared object detection frame;
the second detection model can use the same structure as the visible light face detection model, the infrared face detection model is obtained by training a face temperature map with a label on the basis of the parameters of the visible light detection model, the face object contained in each infrared video frame can be accurately detected by using the trained infrared face detection model, and the face object can be marked out in the infrared video frame in a square frame mode, so that a user can conveniently identify the face object.
And step B2, tracking each infrared video frame to obtain infrared track information of the same object.
In this embodiment, IOU (Intersection over Union, cross-correlation) matching tracking algorithm may be used to track each infrared video frame to obtain infrared track information of the same object; the IOU matching tracking algorithm is a standard for measuring the accuracy of detecting corresponding objects in a specific data set, the standard is used for measuring the correlation degree between reality and prediction, and the higher the correlation degree is, the higher the standard value is; in the target tracking of the present embodiment, for simplicity of implementation, the above standard value may be implemented by a pre-trained infrared face detection model, through which the overlapping rate between the detection frames, that is, the ratio of their intersection to union, may be calculated in the infrared video frame containing the face object; if the calculated overlapping rate between the detection frames in the two adjacent infrared video frames (for example, the video frames with the frame identification of 5 and the frame identification of 6) is higher than a preset overlapping rate threshold value or is the maximum overlapping rate, determining that the face objects corresponding to the two detection frames are the same target object; the frame identification of the infrared video frame where the infrared object detection frame of the same face target is located and the coordinate information of the infrared object detection frame in the infrared video frame sequence can be obtained through the IOU matching tracking algorithm, and the infrared detection frame of the same face target is endowed with the same detection frame identification.
In addition to tracking the face object by using the IOU matching tracking algorithm, the linear correlation degree of the infrared object detection frames in the two infrared video frames can be calculated by using the mahalanobis distance or the covariance distance based on the coordinate information of the infrared object detection frames to track the face object, and the higher the linear correlation degree is, the same face object is described to acquire the infrared track information of each object.
Step S310, taking the object corresponding to each piece of infrared track information as a first object, and executing the following operations in step S312 to step S316 for each first object;
step S312, selecting a high-quality detection frame corresponding to the first object based on each infrared object detection frame corresponding to the infrared track information of the first object;
step S314, searching a target object detection frame matched with the high-quality detection frame from visible object detection frames marked in the visible light video frame;
step S316, determining the temperature of the first object based on the target object detection frame and the quality detection frame.
The embodiment of the invention provides the human body temperature measurement method, which can be used for carrying out time domain and space domain alignment on a visible light camera and a visible light original video stream acquired by an infrared camera based on frame rate and space coordinates to obtain a visible light video frame sequence and an infrared video frame sequence, wherein the visible light video frame marked with a visible object detection frame of a first object is obtained according to a first detection model, the infrared video frame marked with an infrared object detection frame of the first object is obtained according to a second detection model, each infrared video frame is tracked to obtain infrared track information of the first object, and a high-quality detection frame and a target object detection frame corresponding to the first object are obtained based on the visible light video frame, the infrared video frame and the infrared track information of the first object, and then the temperature of the first object is determined based on the target object detection frame and the high-quality detection frame. According to the method, on one hand, the time domain and the space domain of the visible light original video stream and the infrared original video stream can be aligned, on the other hand, the temperature of the first object can be measured based on the aligned visible light video frame sequence and infrared video frame sequence, and by aligning the time domain and the space domain of the visible light original video frame sequence and the infrared original video frame sequence, the high-quality detection frame and the target object detection frame can be accurately obtained, and the accuracy of temperature measurement is effectively improved.
The embodiment of the invention also provides another human body temperature measurement method, which is realized on the basis of the embodiment; the present embodiment focuses on selecting a high-quality detection frame corresponding to a first object, and searching for a target object detection frame matching the high-quality detection frame. As shown in fig. 4, the method for measuring the temperature of the human body in the embodiment includes the following steps:
step S402, a visible light video frame sequence and an infrared video frame sequence corresponding to the same field of view are obtained; the visible light video frame sequence and the infrared video frame sequence are aligned in a time domain and a space domain;
step S404, performing object detection on the visible light video frame sequence to obtain a visible light video frame marked with a visible object detection frame; wherein the object comprises a human face and/or a human body;
step S406, performing object detection and tracking on the infrared video frame sequence to obtain an infrared video frame marked with an infrared object detection frame and infrared track information of the same object; the infrared track information is an infrared video frame sub-sequence containing the same detection frame identification;
step S408, taking the object corresponding to each piece of infrared track information as a first object, and executing the following operations in step S410 to step S420 for each first object;
Step S410, through a pre-trained infrared detection frame quality evaluation model, evaluating each infrared object detection frame corresponding to the infrared track information of the first object, and determining a high-quality detection frame corresponding to the first object based on an evaluation score;
the infrared detection frame quality evaluation model is obtained by training a neural network model by a sample set which is manually marked on the quality of an infrared face in advance, and gives higher evaluation score to the infrared detection frame quality evaluation model under the conditions that the face object corresponding to the infrared object detection frame in an infrared video frame is proper in size, small in shielding, small in posture change and reasonable in temperature distribution of the infrared object detection frame, and otherwise, gives low evaluation score; when the infrared object detection frame corresponding to the first object in the infrared video frame sequence is evaluated through the infrared detection frame quality evaluation model, the infrared object detection frame with the highest evaluation score can be used as a high-quality detection frame corresponding to the first object; or taking the infrared object detection frame with the evaluation score larger than the preset quality threshold as the high-quality detection frame corresponding to the first object, specifically, setting according to actual needs, and not limiting.
Step S412, determining a first frame identification of an infrared video frame where the high-quality detection frame is located;
for convenience of explanation, in this embodiment, a quality detection frame is taken as an example, after the quality detection frame is obtained based on the evaluation score of the quality evaluation model of the infrared detection frame, a first frame identifier of an infrared video frame where the quality detection frame is located is determined, where the first frame identifier is a frame identifier of the infrared video frame in an infrared video frame sequence, for example, the quality detection frame of the object 1 appears in the infrared video frame with a frame identifier of 3.
Step S414, searching a target visible light video frame with a frame identification being a first frame identification from the visible light video frames;
continuing with the previous example, if the quality detection frame of object 1 appears in the infrared video frame with frame number 3, then the visible video frame with frame number 3 is determined as the target visible video frame in the above visible video frame sequence.
Step S416, calculating IOU values of the high-quality detection frames and each visible object detection frame in the target visible light video frame;
in order to determine that the first object corresponding to the high-quality detection frame is the same object in the target visible light video frame, in this embodiment, the high-quality detection frame may be matched with each visible object detection frame in the target visible light video frame by using an IOU matching tracking algorithm to determine the same object.
For example, the target visible light video frame includes a visible object detection frame 1, a visible object detection frame 2 and a visible object detection frame 3, when the detection frames are matched, the IOU values of the high-quality detection frame and the 3 visible object detection frames are calculated respectively by utilizing the algorithm characteristic of the IOU matching tracking algorithm, wherein the IOU values of the high-quality detection frame and the visible object detection frame 1 are 0.4, the IOU values of the high-quality detection frame and the visible object detection frame 2 are 0.6, and the IOU values of the high-quality detection frame and the visible object detection frame 3 are 0.9.
Step S418, taking the visible object detection frame corresponding to the maximum IOU value as a target object detection frame;
in the previous example, the IOU values of the high-quality detection frame and the visible object detection frame 3 are highest, and therefore, it can be determined that the visible object detection frame 3 is the target object detection frame corresponding to the high-quality detection frame.
Step S420, determining the temperature of the first object based on the target object detection frame and the quality detection frame.
According to the human body temperature measurement method provided by the embodiment of the invention, the high-quality detection frame corresponding to the first object can be accurately obtained by using the infrared detection frame quality evaluation model, the target object detection frame corresponding to the high-quality detection frame can be accurately obtained by using the first frame identification of the high-quality detection frame and the IOU matching tracking algorithm, the temperature measurement of the same object is realized based on the high-quality detection frame and the target object detection frame, and the accuracy of object temperature measurement is improved.
The embodiment of the invention also provides another human body temperature measurement method, which is realized on the basis of the embodiment; this embodiment focuses on a specific embodiment of finding a target object detection frame that matches a good quality detection frame. As shown in fig. 5, the method for measuring the temperature of the human body in the embodiment includes the following steps:
step S502, a visible light video frame sequence and an infrared video frame sequence corresponding to the same field of view are obtained; the visible light video frame sequence and the infrared video frame sequence are aligned in a time domain and a space domain;
step S504, the visible light video frame sequence carries out object detection to obtain a visible light video frame marked with a visible object detection frame; wherein the object comprises a human face and/or a human body;
step S506, each visible light video frame is tracked to obtain visible light track information of the same object; the visible light track information is a visible light video frame sub-sequence containing the same detection frame mark;
likewise, the frame identification of the visible light video frame where the detection frame of the same object is located and the coordinate information of the detection frame of the visible light object can be determined in the visible light video frame sequence by utilizing an IOU matching tracking algorithm; the process is the same as the process of performing IOU tracking on each infrared video frame based on the IOU matching tracking algorithm to obtain the infrared track information of the same object, and details are not repeated here.
Step S508, performing object detection and tracking on the infrared video frame sequence to obtain an infrared video frame marked with an infrared object detection frame and infrared track information of the same object; the infrared track information is an infrared video frame sub-sequence containing the same detection frame identification;
step S510, taking the object corresponding to each piece of infrared track information as a first object, and executing the following operations in step S512 to step S518 for each first object;
step S512, selecting a high-quality detection frame corresponding to the first object based on each infrared object detection frame corresponding to the infrared track information of the first object;
step S514, searching target visible light track information matched with the infrared track information of the first object from the visible light track information of each same object;
the implementation process of step S514 may be performed through steps C1-C2:
step C1, matching degree scoring is carried out on a visible object detection frame corresponding to visible light track information of each same object and an infrared object detection frame corresponding to infrared track information of a first object;
similarly, matching degree scoring of track information can be performed by using an IOU matching tracking algorithm, so that the IOU values of the infrared object detection frames corresponding to the infrared track information of the first object and the visible object detection frames corresponding to the visible track information of each same object are obtained, and in order to improve the matching rate, the matching degree scoring process of the infrared object detection frames corresponding to the infrared track information of the first object and the visible object detection frames in 3 or 4 video frames can be randomly selected from the visible video frame subsequences of each same object, and the matching degree scoring process is the same as the process of calculating the IOU values of the high-quality detection frames and the visible object detection frames in the target visible video frames, so that details are not repeated here.
And C2, taking the visible light track information with the highest matching degree scoring value as target visible light track information matched with the infrared track information of the first object.
Likewise, the visible light track information with the highest IOU value can be used as the target visible light track information matched with the infrared track information of the first object.
Therefore, whether the two tracks are matched or not is judged through the multi-frame images in the tracks, and the accuracy is higher.
The implementation process of step S514 may be performed by step C3:
and C3, searching a visible light video frame with the same frame identification as the infrared video frame where the high-quality detection frame corresponding to the first object is located, searching a visible object detection frame matched with the high-quality detection frame corresponding to the first object in the visible light video frame (for example, taking the visible object detection frame with the high-quality detection frame IOU larger than a threshold value or the maximum IOU as a matched visible object detection frame), and taking the visible light track information corresponding to the visible object detection frame as target visible light track information.
Step S516, selecting a target object detection frame from the visible object detection frames corresponding to the target visible light track information;
step S518, determining the temperature of the first object based on the target object detection frame and the quality detection frame.
According to the human body temperature measurement method provided by the embodiment of the invention, after the visible light track information of each object is obtained, the target visible light track information matched with the infrared track information of the first object can be determined based on the matching degree scoring mode, further, the target object detection frame corresponding to the high-quality detection frame can be obtained according to the target visible light track information, the temperature measurement of the same object is realized based on the high-quality detection frame and the target object detection frame, and the accuracy of object temperature measurement is ensured.
Based on the human body temperature measurement method provided by the above embodiment, the present embodiment provides a spatial correction method for performing spatial domain correction on a visible light video frame sequence and an infrared video frame sequence. A flow chart of a spatial correction method as shown in fig. 6, the method comprising the steps of:
step S602, a visible light video frame sequence and an infrared video frame sequence corresponding to the same field of view are obtained; the visible light video frame sequence and the infrared video frame sequence are aligned in a time domain and a space domain;
in the embodiment, the time domain and the space domain of the visible light original video stream and the infrared original video stream of the visible light camera and the infrared camera are aligned to obtain a visible light video frame sequence and an infrared video frame sequence. The spatial alignment is an alignment operation based on a spatial variation relationship determined by the placement positions of the visible light camera and the infrared camera, and the accuracy may not be very high, so that the spatial alignment accuracy of the visible light camera and the infrared camera is improved through the following spatial correction.
Step S604, performing object detection on the visible light video frame sequence to obtain a visible light video frame marked with a visible object detection frame; wherein the object comprises a human face and/or a human body;
step S606, each visible light video frame is tracked to obtain visible light track information of the same object; the visible light track information is a visible light video frame sub-sequence containing the same detection frame mark;
step S608, performing object detection and tracking on the infrared video frame sequence to obtain an infrared video frame marked with an infrared object detection frame and infrared track information of the same object;
step S610, for infrared estimation information of a first object in infrared track information of each same object, searching target visible track information matched with the infrared track information of the first object from visible track information of each same object to obtain visible track information of the first object;
step S612, determining a spatial correction relationship of the visible light video frame sequence and the infrared video frame sequence based on the visible light track information and the infrared track information of the first object;
the first object may be an object corresponding to any one of the tracks, and is not limited herein.
In the embodiment, a preset feature extraction and matching algorithm can be applied to extract feature points, such as corner points, of a first object in the same time domain in a visible light video frame and an infrared video frame respectively, so as to obtain coordinate values of the feature points in the two video frames, and match and correspond the extracted feature points; performing self-adaptive alignment on a visible light video frame sequence and an infrared video frame sequence in a space domain, further determining a projection transformation matrix between a visible light camera and the infrared camera, wherein the projection transformation matrix is a spatial correction relation of the two video frame sequences, and through the projection transformation matrix, coordinate conversion between the visible light video frame and the infrared video frame can be realized, and also can be understood that according to the projection transformation matrix, the coordinate of a designated position in the visible light video frame can be mapped into the infrared video frame, and the difference between the mapped coordinate and the coordinate of the position corresponding to the designated position in the infrared video frame is minimized; or mapping the coordinates of the appointed position in the infrared video frame to the visible light video frame, and minimizing the difference between the mapped coordinates and the coordinates of the position corresponding to the appointed position in the visible light video frame, thereby realizing accurate alignment of the airspace.
For convenience of explanation, for example, the visible light coordinate system is a, the infrared coordinate system is B, the transformation relationship between the two coordinate systems obtained by the spatial coordinates corresponding to the positions of the visible light camera and the infrared camera is a 1 x m1=b, and M1 is the airspace change matrix corresponding to the visible light camera and the infrared camera. Referring to a schematic diagram based on a spatial coordinate alignment detection frame shown in fig. 12, for more visual representation, a first object is taken as an example for explanation, and in fig. 12, an infrared object detection frame and a visible object detection frame corresponding to the first object respectively are both placed in the same coordinate system for explanation. In fig. 12, a large rectangular frame on the left side represents a 1 st video frame of the first object, in this embodiment, the first object is represented in the 1 st infrared video frame and the 1 st visible light video frame in the first video frame of fig. 12 by corresponding detection frames, the white frame represents the infrared object detection frame, and the diagonal frame represents the visible object detection frame. The large rectangular frame on the right side represents the 2 nd video frame of the first object, and the same is true of the 1 st video frame, the white frame in the 2 nd video frame is the infrared object detection frame of the first object in the 2 nd infrared video frame, and the oblique line frame in the 2 nd video frame is the visible object detection frame of the first object in the 2 nd visible light video frame. As can be seen from fig. 12, after the spatial alignment operation corresponding to the spatial variation matrix, the infrared object detection frame and the visible object detection frame are not completely overlapped, in order to eliminate the alignment deviation existing between the two types of detection frames, in this embodiment, the spatial correction relationship M2 may be determined by using the position relationship between the visible light track information of the first object and the visible light detection frame and the infrared detection frame that are matched with each other in the infrared track information, the transformation relationship between two coordinate systems based on the spatial correction relationship is a×m1×m2=b, and after the alignment is corrected and aligned corresponding to M2, the visible object detection frame and the corresponding infrared object detection frame may be overlapped or substantially overlapped. It will be appreciated that in one example, M2 may be determined based on the positions of the respective matched visible object detection frames and infrared object detection frames in the visible light trace information of the first object and the infrared trace information of the first object, e.g., solving for M2 to minimize the sum of the positional deviations of the respective matched visible object detection frames and infrared object detection frames.
Step S614, correcting the visible light track information and/or the infrared track information of other objects except the first object according to the spatial correction relation to obtain corrected visible light track information and/or corrected infrared track information; or correcting the visible object detection frames and/or the infrared object detection frames of other objects except the first object according to the spatial correction relation to obtain a visible object correction detection frame group and/or an infrared object correction detection frame group.
The correction of the visible object detection frame will be described as an example. The method comprises the steps of inquiring coordinate information of an infrared object detection frame corresponding to each visible object detection frame, correcting the coordinate information of each visible object detection frame based on the coordinate information of the infrared object detection frame and the spatial correction relation to obtain a visible object correction detection frame group, wherein the visible object correction detection frame group comprises a plurality of corrected visible object detection frames, and each corrected visible object detection frame can obtain an infrared object detection frame contained in a corresponding video frame through the spatial correction relation.
The corrected track information or the corrected detection frame group obtained through correction may be used to determine a matching relationship between the infrared track information and the visible light track information, or determine a matching relationship between the infrared detection frame and the visible light detection frame, for example, for the temperature measurement of a second object (i.e., an object after the first object), the IOU value of each of the high-quality detection frame and the visible object correction detection frame in the visible object correction detection frame group corresponding to the second object may be calculated; taking the visible object correction detection frame corresponding to the maximum IOU value as a target object detection frame matched with the second object; and taking the visible object correction detection frame corresponding to the calculated maximum IOU value as a target object detection frame corresponding to the high-quality detection frame so as to realize temperature measurement of the second object.
The calculation process is the same as the process of calculating the IOU value of each of the high-quality detection frame and the target visible-light video frame in the above embodiment, and the IOU value calculation can be implemented by using the IOU matching tracking algorithm, so that a detailed description is omitted here.
In the next temperature measurement process, based on the visible light track information and the infrared track information of the same object acquired by the two cameras and the spatial correction relation, the detection frames matched with each other can be acquired more accurately, and accurate temperature measurement is completed.
The embodiment provides another human body temperature measurement method, which is realized on the basis of the embodiment; the present embodiment focuses on a specific embodiment of determining the temperature of the first object based on the target object detection frame and the quality detection frame. As shown in fig. 7, the method for measuring the temperature of the human body in the embodiment includes the following steps:
step S702, a visible light video frame sequence and an infrared video frame sequence corresponding to the same field of view are obtained; the visible light video frame sequence and the infrared video frame sequence are aligned in a time domain and a space domain;
step S704, performing object detection on the visible light video frame sequence to obtain a visible light video frame marked with a visible object detection frame; wherein the object comprises a human face and/or a human body;
Step S706, performing object detection and tracking on the infrared video frame sequence to obtain an infrared video frame marked with an infrared object detection frame and infrared track information of the same object; the infrared track information is an infrared video frame sub-sequence containing the same detection frame identification;
step S708, taking the object corresponding to each piece of infrared track information as a first object, and executing the following operations in step S710 to step S714 for each first object;
step S710, selecting a high-quality detection frame corresponding to the first object based on each infrared object detection frame corresponding to the infrared track information of the first object;
step S712, searching a target object detection frame matched with the high-quality detection frame from visible object detection frames marked in the visible light video frame;
step S714, detecting key points of the target object detection frame to obtain visible light coordinate information containing a local object region; the local object area is a forehead area or a wrist area;
the local object area refers to a forehead area or a wrist area exposed outside without shielding clothes, and the object is a face, so in this embodiment, the forehead area is taken as an example, and the visible light coordinate information of the forehead area of the first object can be extracted from the target object detection frame by using a pre-trained face key point detection model.
The face key point detection model is obtained by training a neural network by using a sample training set of marked face key points, and the position of a local object region in a video frame can be accurately obtained by using the face key point detection model.
In step S716, a sub-detection area is determined in the high-quality detection frame based on the visible light coordinate information, and a temperature corresponding to the sub-detection area is determined as the temperature of the first object.
According to the visible light coordinate information of the forehead region obtained in step S714, and the projective transformation matrix obtained in the above embodiment, the position of the sub-detection region of the forehead region in the high-quality detection frame can be calculated, where the sub-detection region is a detection region only including the forehead region, and the temperature corresponding to the sub-detection region can be determined as the temperature of the first target object.
If the quality detection frame is one, the temperature corresponding to the sub detection area determined by the quality detection frame can be directly determined as the temperature of the first object.
If there are multiple quality detection frames, for example, the determined quality detection frames are respectively in the infrared video frames with frame identifiers of 3 and 4, the temperatures corresponding to the sub-detection regions obtained based on the visible light coordinate information and the projective transformation matrix are respectively 36.5 ℃ and 36.8 ℃, the highest temperature corresponding to the sub-detection regions can be determined as the temperature of the first object, or the temperature value obtained by weighting or averaging the multiple temperatures can be used as the temperature of the first object, or the temperature of the first object can be determined by using a fractional mode, which is not limited.
The obtained temperature of the first object is the temperature of the local object area, the mapping relation between the local object area and the body temperature can be obtained through a trained neural network mapping model, and the human body temperature is obtained through the mapping relation after the temperature of the local object area is obtained; alternatively, the temperature of the local target region is directly recorded and stored as the human body temperature.
According to the human body temperature measurement method provided by the embodiment of the invention, the obtained target object detection frame can be subjected to key point detection to obtain the visible light coordinate information containing the local object region, the sub detection region is determined in the high-quality detection frame based on the visible light coordinate information, the temperature corresponding to the sub detection region is determined as the temperature of the first object, and the appointed local detection region can be determined in the high-quality detection frame through the visible light coordinate information of the key point, so that the temperature detection of the appointed region of the first object is effectively realized.
In order to store the temperature of the first object in real time, the method further comprises: based on the characteristic information of the target object detection frame, searching the identity information of the first object in a pre-stored portrait database; and determining the corresponding relation between the retrieved identity information and the temperature of the first object.
The feature information of the target object detection frame may be image information of the target object detection frame; if image information matching the first object is retrieved from a pre-stored portrait database, the identity information of the first object, for example, an ID (Identity document, identification number) number, is queried, the identity information of the first object is associated with the detected temperature, and specifically, the acquired temperature of the first object may be stored in a folder under the ID number.
If the image information matched with the first object is not searched in the pre-stored portrait database, a track ID number is built based on the visible light track information of the first object, the acquired temperature of the first object is stored in a folder under the ID number based on the built track ID number, and the storage mode is convenient for recording the information of the detected person and checking the temperature abnormal person, so that the action track of the abnormal temperature person can be tracked quickly, and the tracking efficiency and accuracy of the abnormal temperature person are improved.
Further, in order to fully understand the above-mentioned human body temperature measurement method, fig. 8 shows a flowchart of another human body temperature measurement method, and as shown in fig. 8, taking a human face object as a target object as an example, the human body temperature measurement method includes the following steps:
Step S800, for the same visual field visible light camera to collect the visible light video frame sequence, the infrared camera collects the infrared video frame sequence, and then step S801 and step S802 are executed respectively;
the method comprises the steps of installing a visible light camera and an infrared camera in an aligned mode, and specifically, arranging the visible light camera and the infrared camera side by side in parallel or vertically arranging the visible light camera and the infrared camera side by side to collect video frame sequences in the same field of view area so as to achieve airspace alignment of the visible light video frame sequences and the infrared video frame sequences; because the frame rates of the visible light camera and the infrared camera are different, the number of the video frames contained in the acquired visible light video frame sequence and the infrared video frame sequence is different, and in order to realize time domain alignment, the two video sequences can be processed by a video frame copying method or by utilizing an acquisition time filtering method so as to realize time domain alignment of the two videos.
Step S801, face frame information in an infrared video frame sequence is determined, and then step S803 is executed;
and detecting the face object of the temperature video frame sequence through a pre-trained infrared face detection model to obtain an infrared video frame with the face object, and marking the face object in the infrared video frame with the face object in a box form.
Step S802, face frame information in a visible light video frame sequence is determined, and then step S804 is executed;
and detecting the human face object through a pre-trained visible light human face detection model to the visible light video frame sequence so as to obtain a visible light video frame with the human face object, and marking the human face object in the visible light video frame with the human face object in a box form.
Step S803, acquiring infrared track information of the face object in the infrared video frame sequence, and then executing step S805;
step S804, obtaining visible light track information of the face object in the visible light video frame sequence, and then executing step S806;
in this embodiment, an IOU matching tracking algorithm may be used to obtain frame identification of an infrared video frame where a detection frame of the same face object is located in the infrared video frame sequence, and infrared track information such as coordinate information of the detection frame of the face object; likewise, the frame identification of the visible light video frame where the detection frame of the same face object is located and the coordinate information of the face object detection frame can be determined in the visible light video frame sequence through the IOU matching tracking algorithm.
Step S805, selecting a high-quality detection frame corresponding to the first object based on each infrared object detection frame corresponding to the infrared track information of the first object, and then executing step S806;
In order to achieve temperature measurement of multiple objects, in this embodiment, an object corresponding to each piece of infrared track information may be used as a first object to perform temperature detection, and in order to achieve accurate temperature measurement, a pre-trained infrared detection frame quality evaluation model may be used to select a high-quality detection frame with a suitable size, small shielding, small gesture change, and reasonable temperature distribution of an infrared object detection frame from the infrared track information corresponding to the first object.
Step S806, searching target visible light track information matched with the infrared track information of the first object from the visible light track information of each same object;
because each object corresponds to a piece of visible light track information, in order to obtain the target visible light track information matched with the infrared track information of the first object, the visible light track information with the highest matching degree scoring value can be used as the target visible light track information matched with the infrared track information of the first object in a way of scoring the matching degree of the track information by using the IOU matching tracking algorithm.
Step S807, selecting a target object detection frame from the visible object detection frames corresponding to the target visible light track information;
and calculating the IOU value of each visible object detection frame corresponding to the high-quality detection frame and the target visible light track information by using the IOU matching tracking algorithm, and determining the visible object detection frame corresponding to the calculated maximum IOU value as the target object detection frame.
Step S808, detecting key points of the target object detection frame to obtain visible light coordinate information containing a local object region; the local object area is a forehead area or a wrist area;
in this embodiment, a pre-trained face key point detection model may be used to extract a pre-set temperature measurement local area, such as a forehead area, from the target object detection frame.
Step S809, determining a sub-detection area in the high-quality detection frame based on the visible light coordinate information;
according to the visible light coordinate information of the forehead region in the visible light video frame and the projection transformation matrix, the position of the forehead region can be determined in the high-quality detection frame, and then the forehead temperature corresponding to the forehead region can be obtained.
Step S810, obtaining the temperature of the human body through the temperature corresponding to the sub-detection area;
the mapping relation between the local object area of each face object and the body temperature is obtained through a pre-trained neural network mapping model, and the body temperature is obtained through the mapping relation after the temperature of the local object area corresponding to the face object is obtained.
In order to further improve the accuracy of temperature measurement, the method can further comprise temperature compensation for the human body temperature; in general, since the infrared camera is affected by internal and external environments during working, such as air conditioner and ambient temperature disturbance of people stream, a certain error may exist between an actually collected temperature map and an actually collected temperature, and in order to correct the temperature map collected by the infrared camera, temperature compensation can be performed on the infrared video stream based on the ambient temperature corresponding to the same field of view.
The infrared camera can capture infrared rays emitted by an object, wherein the infrared rays are the most extensive electromagnetic wave radiation existing in nature, the infrared camera is based on the fact that any object can generate irregular motions of molecules and atoms of the object under a conventional environment and continuously emits thermal infrared energy, in real life, the infrared camera can convert infrared rays emitted by an object to be detected into different gray scales to display, different representative temperatures of the gray scales are different, different gray scale images correspond to different temperatures, and each frame of infrared video frame in the infrared video sequence is a gray scale image.
In this embodiment, when an infrared camera is used, a reference "blackbody radiation source" is introduced for temperature compensation. In short, the blackbody radiation source is a constant temperature target and is used for correcting the infrared camera continuously, and the specific process is as follows: setting a fixed temperature by using a blackbody radiation source as a standard source; the thermal infrared camera collects infrared data of a target object and a blackbody radiation source and calibrates the blackbody radiation source; the infrared data are converted into different gray scales to be displayed; and obtaining a final temperature value through standard source temperature and collected gray mapping correction of the corresponding region.
The temperature compensation can be performed after the temperature of the first object is determined based on the target object detection frame and the high-quality detection frame, and the temperature compensation can be performed when the infrared original video stream is acquired, so that the influence of the infrared camera or the external environment on the temperature of the actual temperature measurement object is reduced, the accuracy of temperature measurement based on the video frame is ensured, the temperature measurement mode can be used for simultaneously tracking the track of a plurality of face objects in the same view field by combining the visible video frame with the infrared video frame, the human body temperature is determined in the infrared target frame, the temperature measurement efficiency is improved to a certain extent, in addition, the temperature measurement mode does not need a worker to be close to the measured target temperature measurement, and the labor cost is reduced to a certain extent.
It should be noted that, the foregoing method embodiments are all described in a progressive manner, and each embodiment focuses on the differences from the other embodiments, and the same similar parts between the embodiments are all mutually referred to.
Corresponding to the above method embodiment, the embodiment of the present invention provides a human body temperature measuring device, which is applied to the above electronic apparatus, and fig. 9 shows a schematic structural diagram of a human body temperature measuring device, and as shown in fig. 9, the human body temperature measuring device includes:
The acquisition module 902 is configured to acquire a visible light video frame sequence and an infrared video frame sequence corresponding to the same field of view; the visible light video frame sequence and the infrared video frame sequence are aligned in a time domain and a space domain;
the detection module 904 is configured to perform object detection on the sequence of visible light video frames to obtain visible light video frames marked with a visible object detection frame; wherein the object comprises a human face and/or a human body;
the detection tracking module 906 is configured to detect and track an object in the infrared video frame sequence, so as to obtain an infrared video frame marked with an infrared object detection frame and infrared track information of the same object; the infrared track information is an infrared video frame sub-sequence containing the same detection frame identification;
the execution module 910 is configured to take, as first objects, objects corresponding to each piece of infrared track information, respectively, and for each first object, perform the following operations:
a selection module 912, configured to select a high-quality detection frame corresponding to the first object based on each infrared object detection frame corresponding to the infrared track information of the first object;
a searching module 914, configured to search a target object detection frame matched with the high-quality detection frame from the visible object detection frames marked in the visible light video frame;
A first determination module 916 for determining a temperature of the first object based on the target object detection box and the quality detection box.
The human body temperature measuring device provided by the invention can acquire the visible light video frame sequence and the infrared video frame sequence corresponding to the same view field; performing object detection on the visible light video frame sequence to obtain a visible light video frame marked with a visible object detection frame, and performing object detection and tracking on the infrared video frame sequence to obtain an infrared video frame marked with an infrared object detection frame and infrared track information of the same object; and selecting a high-quality detection frame corresponding to the first object based on the infrared track information of the first object, and searching out a target object detection frame matched with the high-quality detection frame from the visible object detection frames marked in the visible light video frame to determine the temperature of the first object. The temperature measurement mode can track the same target object in the same area by combining the visible light video frame and the infrared video frame, and the temperature of the same object is determined based on track information obtained by tracking and a high-quality detection frame of the same object in the infrared video frame and a target object detection frame in the visible light video frame, so that the accuracy of temperature measurement is ensured.
The acquiring module 902 is further configured to acquire a video stream from the same field of view through a visible light camera and an infrared camera, so as to obtain a visible light original video stream and an infrared original video stream; and performing time domain and space domain alignment on the visible light original video stream and the infrared original video stream to obtain a visible light video frame sequence and an infrared video frame sequence.
The acquiring module 902 is further configured to determine a time domain alignment mode based on frame rates of the visible light camera and the infrared camera; determining an airspace alignment mode based on space coordinates corresponding to positions of the visible light camera and the infrared camera; and performing time domain and space domain alignment on the visible light original video stream and the infrared original video stream according to the determined time domain alignment mode and space domain alignment mode to obtain a visible light video frame sequence and an infrared video frame sequence.
The detection module 904 is further configured to perform object detection on the sequence of visible light video frames by using a pre-trained first detection model, so as to obtain visible light video frames marked with a visible object detection frame.
The detection tracking module 906 is further configured to perform object detection on the infrared video frame sequence through a pre-trained second detection model, so as to obtain an infrared video frame marked with an infrared object detection frame; and tracking each infrared video frame to obtain infrared track information of the same object.
The selection module 912 is further configured to evaluate each infrared object detection frame corresponding to the infrared track information of the first object through a pre-trained infrared detection frame quality evaluation model, and determine a high-quality detection frame corresponding to the first object based on the evaluation score.
The selecting module 912 is further configured to use the infrared object detection frame with the highest evaluation score as the high-quality detection frame corresponding to the first object; or taking the infrared object detection frame with the evaluation score larger than the preset quality threshold as the high-quality detection frame corresponding to the first object.
The searching module 914 is further configured to determine a first frame identifier of an infrared video frame where the high-quality detection frame is located; searching a target visible light video frame with a frame identifier being a first frame identifier from the visible light video frames; calculating IOU values of the high-quality detection frames and the detection frames of all visible objects in the target visible light video frame; and taking the visible object detection frame corresponding to the maximum IOU value as a target object detection frame.
Based on the above-mentioned human body temperature measuring device, the embodiment of the present invention further provides another human body temperature measuring device, referring to the schematic structural diagram of the human body temperature measuring device shown in fig. 10, where the human body temperature measuring device includes a tracking module 1002 connected to both the detection module 904 and the detection tracking module 906, and tracks each visible light video frame to obtain visible light track information of the same object; the visible light track information is a visible light video frame sub-sequence containing the same detection frame identification.
The searching module 914 is further configured to search for target visible light track information matching with the infrared track information of the first object from the visible light track information of each same object; and selecting a target object detection frame from the visible object detection frames corresponding to the target visible light track information.
The searching module 914 is further configured to score a matching degree between a visible object detection frame corresponding to the visible light track information of each identical object and an infrared object detection frame corresponding to the infrared track information of the first object; and taking the visible light track information with the highest matching degree scoring value as target visible light track information matched with the infrared track information of the first object.
The human body temperature measuring device further comprises a correction module 1004 connected to the selection module 912 and the search module 914, and configured to determine a spatial correction relationship between the visible light video frame sequence and the infrared video frame sequence based on the visible light track information and the infrared track information of the first object.
The searching module 914 is further configured to correct each visible object detection frame corresponding to the target visible light track information according to the spatial correction relationship, so as to obtain a visible object correction detection frame group; calculating IOU values of the high-quality detection frames and the visible object correction detection frames in the visible object correction detection frame group; and taking the visible object correction detection frame corresponding to the maximum IOU value as a target object detection frame.
The first determining module 916 is further configured to perform key point detection on the target object detection frame to obtain visible light coordinate information including the local object region; the local object area is a forehead area or a wrist area; and determining a sub-detection area in the high-quality detection frame based on the visible light coordinate information, and determining the temperature corresponding to the sub-detection area as the temperature of the first object.
The human body temperature measuring device further comprises a retrieving module 1006 connected to the first determining module 916, for retrieving the identity information of the first object from the pre-stored portrait database based on the feature information of the target object detection frame.
The human body temperature measuring device further comprises a second determining module 1008 connected to the retrieving module 1006, and configured to determine a correspondence between the retrieved identity information and the temperature of the first object.
The embodiment of the invention also provides a human body temperature measurement system, referring to a schematic structural diagram of a human body temperature measurement system shown in fig. 11, as shown in fig. 11, the human body temperature measurement system 1104 includes a server 1103 and a camera set 1100, where the camera set 1100 includes a visible light camera 1101 and an infrared camera 1102; the camera set 1100 is configured to obtain a visible light video sequence and an infrared video sequence corresponding to the same field of view through the visible light camera 1101 and the infrared camera 1102, respectively; the server 1103 includes a memory, a processor, and a computer program stored on the memory and executable on the processor, which when executed implements the steps of the human body temperature measurement method described above.
Specifically, the above-mentioned visible light camera 1101 and infrared camera 1102 may be placed side by side in parallel or placed side by side vertically on the same position to implement airspace alignment, and the server 1103 is in communication connection with the camera set 1100 to receive a visible light video sequence collected by the visible light camera and an infrared video sequence collected by the infrared camera, and perform time domain alignment on the received visible light video sequence and infrared video sequence, and perform the steps of the above-mentioned human body temperature measurement method by using the combination application of the two video sequences aligned in the time domain and airspace, so as to implement temperature measurement on multiple target objects in the same field of view. The present embodiment also provides a computer readable storage medium, on which a computer program is stored, which when executed by a processing device performs the steps of the human body temperature measurement method described above.
The embodiment of the invention provides a method and a device for measuring temperature of a human body and a computer program product of electronic equipment, which comprise a computer readable storage medium storing program codes, wherein the program codes comprise instructions for executing the method described in the embodiment of the method, and specific implementation can be seen in the embodiment of the method and is not repeated herein.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
Finally, it should be noted that: the above examples are only specific embodiments of the present invention, and are not intended to limit the scope of the present invention, but it should be understood by those skilled in the art that the present invention is not limited thereto, and that the present invention is described in detail with reference to the foregoing examples: any person skilled in the art may modify or easily conceive of the technical solution described in the foregoing embodiments, or perform equivalent substitution of some of the technical features, while remaining within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention, and are intended to be included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims (16)

1. A method for measuring temperature of a human body, the method being applied to an electronic device and comprising:
obtaining a visible light video frame sequence and an infrared video frame sequence corresponding to the same view field; wherein the visible light video frame sequence and the infrared video frame sequence are aligned in a time domain and a space domain;
performing object detection on the visible light video frame sequence to obtain a visible light video frame marked with a visible object detection frame; wherein the object comprises a human face and/or a human body;
the object detection and tracking are carried out on the infrared video frame sequence, so that an infrared video frame marked with an infrared object detection frame and infrared track information of the same object are obtained; the infrared track information is an infrared video frame sub-sequence containing the same detection frame identification;
taking the object corresponding to each piece of infrared track information as a first object respectively, and executing the following operations for each first object:
selecting a high-quality detection frame corresponding to the first object based on each infrared object detection frame corresponding to the infrared track information of the first object;
searching a target object detection frame matched with the high-quality detection frame from visible object detection frames marked in the visible light video frame;
A temperature of the first object is determined based on the target object detection box and the quality detection box.
2. The method of claim 1, wherein the step of acquiring the sequence of visible video frames and the sequence of infrared video frames corresponding to the same field of view comprises:
the video stream is acquired from the same view field through a visible light camera and an infrared camera, so that a visible light original video stream and an infrared original video stream are obtained;
and performing time domain and space domain alignment on the visible light original video stream and the infrared original video stream to obtain a visible light video frame sequence and an infrared video frame sequence.
3. The method of claim 2, wherein the step of time-domain and space-domain aligning the visible original video stream and the infrared original video frame sequence to obtain a visible video frame sequence and an infrared video frame sequence comprises:
determining a time domain alignment mode based on the frame rates of the visible light camera and the infrared camera;
determining an airspace alignment mode based on space coordinates corresponding to positions of the visible light camera and the infrared camera;
and performing time domain and space domain alignment on the visible light original video stream and the infrared original video stream according to the determined time domain alignment mode and the determined space domain alignment mode to obtain a visible light video frame sequence and an infrared video frame sequence.
4. The method of claim 1, wherein the step of performing object detection on the sequence of visible light video frames to obtain visible light video frames tagged with a visible object detection frame comprises:
performing object detection on the visible light video frame sequence through a pre-trained first detection model to obtain a visible light video frame marked with a visible object detection frame;
the step of carrying out object detection and tracking on the infrared video frame sequence to obtain the infrared video frame marked with the infrared object detection frame and the infrared track information of the same object comprises the following steps:
performing object detection on the infrared video frame sequence through a pre-trained second detection model to obtain an infrared video frame marked with an infrared object detection frame;
and tracking each infrared video frame to obtain infrared track information of the same object.
5. The method of claim 1, wherein the step of selecting a quality inspection box corresponding to the first object based on each infrared object inspection box corresponding to the infrared track information of the first object comprises:
and evaluating each infrared object detection frame corresponding to the infrared track information of the first object through a pre-trained infrared detection frame quality evaluation model, and determining a high-quality detection frame corresponding to the first object based on an evaluation score.
6. The method of claim 5, wherein the step of determining a quality inspection box corresponding to the first object based on the evaluation score comprises:
taking the infrared object detection frame with the highest evaluation score as a high-quality detection frame corresponding to the first object; or,
and taking the infrared object detection frame with the evaluation score larger than the preset quality threshold as a high-quality detection frame corresponding to the first object.
7. The method of claim 1, wherein the step of finding a target object detection box matching the quality detection box from the visible object detection boxes marked in the visible video frame comprises:
determining a first frame identification of an infrared video frame where the high-quality detection frame is located;
searching a target visible light video frame with a frame identifier being the first frame identifier from the visible light video frame;
calculating IOU values of the high-quality detection frames and each visible object detection frame in the target visible light video frame;
and taking the visible object detection frame corresponding to the maximum IOU value as a target object detection frame.
8. The method according to claim 1, wherein the method further comprises: tracking each visible light video frame to obtain visible light track information of the same object; the visible light track information is a visible light video frame sub-sequence containing the same detection frame mark;
The step of searching the target object detection frame matched with the high-quality detection frame from the visible object detection frames marked in the visible light video frame comprises the following steps:
searching target visible light track information matched with the infrared track information of the first object from the visible light track information of each same object;
and selecting a target object detection frame from the visible object detection frames corresponding to the target visible light track information.
9. The method of claim 8, wherein the target visible light track information matching the infrared track information of the first object is found from the visible light track information of each of the same objects; comprises the steps of:
scoring the matching degree of a visible object detection frame corresponding to the visible light track information of each identical object and an infrared object detection frame corresponding to the infrared track information of the first object;
and taking the visible light track information with the highest matching degree scoring value as target visible light track information matched with the infrared track information of the first object.
10. The method of claim 8, wherein the method further comprises:
determining a spatial correction relationship of the visible video frame sequence and the infrared video frame sequence based on the visible light track information and the infrared track information of the first object;
The step of selecting a target object detection frame from the visible object detection frames corresponding to the target visible light track information comprises the following steps:
correcting each visible object detection frame corresponding to the target visible light track information according to the space correction relation to obtain a visible object correction detection frame group;
calculating IOU values of the high-quality detection frames and each visible object correction detection frame in the visible object correction detection frame group;
and taking the visible object correction detection frame corresponding to the maximum IOU value as a target object detection frame.
11. The method of claim 1, wherein determining the temperature of the first object based on the target object detection box and the quality detection box comprises:
performing key point detection on the target object detection frame to obtain visible light coordinate information containing a local object region; wherein the local object area is a forehead area or a wrist area;
and determining a sub-detection area in the high-quality detection frame based on the visible light coordinate information, and determining the temperature corresponding to the sub-detection area as the temperature of the first object.
12. The method according to claim 1, wherein the method further comprises:
Based on the characteristic information of the target object detection frame, searching the identity information of the first object in a pre-stored portrait database;
and determining the corresponding relation between the retrieved identity information and the temperature of the first object.
13. A human body temperature measurement device, wherein the device is applied to electronic equipment and comprises:
the acquisition module is used for acquiring a visible light video frame sequence and an infrared video frame sequence corresponding to the same view field; wherein the visible light video frame sequence and the infrared video frame sequence are aligned in a time domain and a space domain;
the detection module is used for carrying out object detection on the visible light video frame sequence to obtain a visible light video frame marked with a visible object detection frame; wherein the object comprises a human face and/or a human body;
the detection tracking module is used for detecting and tracking the infrared video frame sequence to obtain an infrared video frame marked with an infrared object detection frame and infrared track information of the same object; the infrared track information is an infrared video frame sub-sequence containing the same detection frame identification;
the execution module is used for taking the object corresponding to each piece of infrared track information as a first object respectively, and executing the following operations for each first object:
The selection module is used for selecting a high-quality detection frame corresponding to the first object based on each infrared object detection frame corresponding to the infrared track information of the first object;
the searching module is used for searching a target object detection frame matched with the high-quality detection frame from the visible object detection frames marked in the visible light video frame;
and the first determining module is used for determining the temperature of the first object based on the target object detection frame and the high-quality detection frame.
14. The human body temperature measurement system is characterized by comprising a server and a camera set, wherein the camera set comprises a visible light camera and an infrared camera;
the camera set is used for respectively acquiring a visible light video frame sequence and an infrared video frame sequence corresponding to the same view field through the visible light camera and the infrared camera;
the server comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the human body thermometry method of any one of the preceding claims 1 to 12 when the computer program is executed.
15. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the human body thermometry method of any one of the preceding claims 1 to 12 when the computer program is executed by the processor.
16. A computer readable storage medium having stored thereon a computer program, characterized in that the computer program when run by a processing device performs the steps of the human body thermometry method according to any of claims 1 to 12.
CN202010583724.9A 2020-06-23 2020-06-23 Human body temperature measurement method, device, system and electronic equipment Active CN111914635B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010583724.9A CN111914635B (en) 2020-06-23 2020-06-23 Human body temperature measurement method, device, system and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010583724.9A CN111914635B (en) 2020-06-23 2020-06-23 Human body temperature measurement method, device, system and electronic equipment

Publications (2)

Publication Number Publication Date
CN111914635A CN111914635A (en) 2020-11-10
CN111914635B true CN111914635B (en) 2023-12-26

Family

ID=73226426

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010583724.9A Active CN111914635B (en) 2020-06-23 2020-06-23 Human body temperature measurement method, device, system and electronic equipment

Country Status (1)

Country Link
CN (1) CN111914635B (en)

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112378521B (en) * 2020-11-13 2023-05-05 深圳市科瑞康实业有限公司 Temperature measurement system
CN112818866B (en) * 2021-02-02 2023-11-07 苏州挚途科技有限公司 Vehicle positioning method and device and electronic equipment
CN112816073B (en) * 2021-02-07 2021-12-28 深圳市今视通数码科技有限公司 Temperature measurement method and system based on face recognition and temperature measurement all-in-one machine and storage medium
CN112906600A (en) * 2021-03-04 2021-06-04 联想(北京)有限公司 Object information monitoring method and device and electronic equipment
CN113065528A (en) * 2021-05-08 2021-07-02 南京四维向量科技有限公司 Embedded visual computing system for face recognition, counting and temperature measurement
CN113420782A (en) * 2021-05-27 2021-09-21 南京四维向量科技有限公司 Atlas-based edge vision computing system for face recognition
CN113420629B (en) * 2021-06-17 2023-04-28 浙江大华技术股份有限公司 Image processing method, device, equipment and medium
CN113432720A (en) * 2021-06-25 2021-09-24 深圳市迈斯泰克电子有限公司 Temperature detection method and device based on human body recognition and temperature detection instrument
CN113390515B (en) * 2021-07-06 2023-03-28 新疆爱华盈通信息技术有限公司 Multi-person mobile temperature measurement method based on double cameras
CN113379772B (en) * 2021-07-06 2022-10-11 新疆爱华盈通信息技术有限公司 Mobile temperature measurement method based on background elimination and tracking algorithm in complex environment
CN113503972B (en) * 2021-09-08 2022-01-18 四川大学 Local dynamic target temperature measurement system based on low-pixel infrared camera

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2017156796A (en) * 2016-02-29 2017-09-07 日本放送協会 Object tracking system, object tracking device and program, as well as physical object with location display body
WO2018133666A1 (en) * 2017-01-17 2018-07-26 腾讯科技（深圳）有限公司 Method and apparatus for tracking video target
CN109919007A (en) * 2019-01-23 2019-06-21 绵阳慧视光电技术有限责任公司 A method of generating infrared image markup information
CN110060272A (en) * 2018-01-18 2019-07-26 杭州海康威视数字技术股份有限公司 Determination method, apparatus, electronic equipment and the storage medium of human face region

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2017156796A (en) * 2016-02-29 2017-09-07 日本放送協会 Object tracking system, object tracking device and program, as well as physical object with location display body
WO2018133666A1 (en) * 2017-01-17 2018-07-26 腾讯科技（深圳）有限公司 Method and apparatus for tracking video target
CN110060272A (en) * 2018-01-18 2019-07-26 杭州海康威视数字技术股份有限公司 Determination method, apparatus, electronic equipment and the storage medium of human face region
CN109919007A (en) * 2019-01-23 2019-06-21 绵阳慧视光电技术有限责任公司 A method of generating infrared image markup information

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
目标运动轨迹匹配式的红外-可见光视频自动配准;王洪庆;许廷发;孙兴龙;李相民;刘太辉;;光学精密工程(第06期);全文 *
面向室内环境控制的人员信息检测系统的设计与实现;张文利;郭向;杨堃;王佳琪;朱清宇;;北京工业大学学报(第05期);全文 *

Also Published As

Publication number Publication date
CN111914635A (en) 2020-11-10

Similar Documents

Publication Publication Date Title
CN111914635B (en) Human body temperature measurement method, device, system and electronic equipment
JP4970195B2 (en) Person tracking system, person tracking apparatus, and person tracking program
Milan et al. Challenges of ground truth evaluation of multi-target tracking
JP6428266B2 (en) COLOR CORRECTION DEVICE, COLOR CORRECTION METHOD, AND COLOR CORRECTION PROGRAM
Tang et al. Cross-camera knowledge transfer for multiview people counting
JP2019075156A (en) Method, circuit, device, and system for registering and tracking multifactorial image characteristic and code executable by related computer
WO2019225547A1 (en) Object tracking device, object tracking method, and object tracking program
JP5554726B2 (en) Method and apparatus for data association
JP2014182480A (en) Person recognition device and method
JP2023015989A (en) Item identification and tracking system
GB2430735A (en) Object detection
CN103501688A (en) Method and apparatus for gaze point mapping
GB2430736A (en) Image processing
KR102144394B1 (en) Apparatus and method for alignment of images
JPWO2014010174A1 (en) Angle of view variation detection device, angle of view variation detection method, and field angle variation detection program
CN111327788A (en) Synchronization method, temperature measurement method and device of camera set and electronic system
CN108710841A (en) A kind of face living body detection device and method based on MEMs infrared sensor arrays
CN111488775A (en) Device and method for judging degree of fixation
CN111652314A (en) Temperature detection method and device, computer equipment and storage medium
CN112541403B (en) Indoor personnel falling detection method by utilizing infrared camera
JP5047658B2 (en) Camera device
CN113033266A (en) Personnel motion trajectory tracking method, device and system and electronic equipment
TW202242803A (en) Positioning method and apparatus, electronic device and storage medium
CN106971381B (en) A kind of wide angle camera visual field line of demarcation generation method with the overlapping ken
CN112513870A (en) System and method for detecting, tracking and counting human objects of interest with improved altimetry

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant