CN111970414A - Picture synchronization method and device based on real-time temperature measurement system and storage medium - Google Patents
Picture synchronization method and device based on real-time temperature measurement system and storage medium Download PDFInfo
- Publication number
- CN111970414A CN111970414A CN202010699254.2A CN202010699254A CN111970414A CN 111970414 A CN111970414 A CN 111970414A CN 202010699254 A CN202010699254 A CN 202010699254A CN 111970414 A CN111970414 A CN 111970414A
- Authority
- CN
- China
- Prior art keywords
- picture
- visible light
- infrared
- temperature
- frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000009529 body temperature measurement Methods 0.000 title claims abstract description 77
- 238000000034 method Methods 0.000 title claims abstract description 61
- 230000001360 synchronised effect Effects 0.000 claims abstract description 22
- 230000015654 memory Effects 0.000 claims description 20
- 238000012545 processing Methods 0.000 claims description 14
- 238000002372 labelling Methods 0.000 claims description 12
- 238000005516 engineering process Methods 0.000 claims description 10
- 238000000605 extraction Methods 0.000 claims description 9
- 238000004364 calculation method Methods 0.000 claims description 8
- 238000004422 calculation algorithm Methods 0.000 claims description 7
- 238000004891 communication Methods 0.000 claims description 7
- 238000001914 filtration Methods 0.000 claims description 7
- 238000004861 thermometry Methods 0.000 claims description 6
- 238000010606 normalization Methods 0.000 claims description 5
- 238000009432 framing Methods 0.000 claims 1
- 238000010586 diagram Methods 0.000 description 6
- 230000003111 delayed effect Effects 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 238000004590 computer program Methods 0.000 description 4
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 241001465754 Metazoa Species 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002035 prolonged effect Effects 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/04—Synchronising
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J5/00—Radiation pyrometry, e.g. infrared or optical thermometry
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/76—Television signal recording
- H04N5/91—Television signal processing therefor
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J5/00—Radiation pyrometry, e.g. infrared or optical thermometry
- G01J2005/0077—Imaging
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Studio Devices (AREA)
Abstract
The application discloses a picture synchronization method and device based on a real-time temperature measurement system and a storage medium, and relates to the technical field of Internet of things. The specific implementation scheme is as follows: correspondingly generating an infrared picture cache queue and a visible light picture cache queue respectively based on a first video stream output by an infrared camera and a second video stream output by a visible light camera; based on a pre-configured time delay parameter, acquiring a visible light picture and an infrared picture from an infrared picture cache queue and a visible light picture cache queue so as to synchronously display the infrared picture and the visible light picture; and pushing the infrared picture and the visible light picture to the front end so that the front end can synchronously display the infrared picture and the visible light picture. According to the method and the device, when the time stamp is not output in the picture, the infrared picture and the visible light picture in the real-time temperature measurement system can be still synchronized, the temperature measurement accuracy of the real-time temperature measurement system can be effectively improved, and meanwhile the real-time performance of the picture can be guaranteed.
Description
Technical Field
The application relates to the technical field of image processing, in particular to the technical field of Internet of things, and specifically relates to a picture synchronization method and device based on a real-time temperature measurement system, and a storage medium.
Background
In the existing real-time temperature measurement system, a visible light picture acquired by a visible light camera and an infrared picture acquired by an infrared camera are overlapped together, so that the temperature of a person or an object in the visible light picture is measured in real time. However, due to the hardware device itself and the reason that the picture obtaining mode is different, the problem that the infrared picture and the visible light picture are not synchronous usually occurs, and further the problems that the pictures are misplaced and the temperature cannot be accurately measured are caused.
In the prior art, the time stamps of all pictures output by a camera can be used for matching visible light pictures and infrared pictures with the same time stamp, so that the infrared pictures and the visible light pictures are synchronized. However, many hardware devices do not support outputting a timestamp for each frame of a picture, so synchronization of infrared and visible light pictures cannot be achieved with the same timestamp. Therefore, it is desirable to provide a frame synchronization scheme based on a real-time temperature measurement system.
Disclosure of Invention
In order to solve the technical problem, the application provides a picture synchronization method and device based on a real-time temperature measurement system and a storage medium.
According to an aspect of the present application, there is provided a picture synchronization method based on a real-time temperature measurement system, wherein the method includes:
correspondingly generating an infrared picture cache queue and a visible light picture cache queue respectively based on a first video stream output by an infrared camera and a second video stream output by a visible light camera;
based on a pre-configured time delay parameter, acquiring a visible light picture and an infrared picture from the infrared picture cache queue and the visible light picture cache queue so as to enable the infrared picture and the visible light picture to be displayed synchronously;
and pushing the infrared picture and the visible light picture to the front end so that the front end can synchronously display the infrared picture and the visible light picture.
According to another aspect of the present application, there is provided a picture synchronization apparatus based on a real-time temperature measurement system, wherein the apparatus includes:
the generating module is used for correspondingly generating an infrared picture cache queue and a visible light picture cache queue respectively based on a first video stream output by the infrared camera and a second video stream output by the visible light camera;
the acquisition module is used for acquiring a visible light picture and an infrared picture from the infrared picture cache queue and the visible light picture cache queue based on a preconfigured time delay parameter so as to enable the infrared picture and the visible light picture to be displayed synchronously;
and the pushing module is used for pushing the infrared picture and the visible light picture to the front end so that the front end can synchronously display the infrared picture and the visible light picture.
According to still another aspect of the present application, there is provided a real-time temperature measurement system, including: the system comprises a visible light camera, an infrared camera and the picture synchronization device based on the real-time temperature measurement system; the picture synchronization device based on the real-time temperature measurement system is in communication connection with the visible light camera and the infrared camera respectively, and realizes synchronization processing of visible light pictures of the visible light camera and infrared pictures of the infrared camera.
According to yet another aspect of the present application, there is provided an electronic device including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method as described above.
According to yet another aspect of the present application, there is provided a non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method as described above.
According to the technology of the application, the synchronization of the infrared picture and the visible light picture in the real-time temperature measurement system can be still realized when the timestamp is not output in the picture, the temperature measurement accuracy of the real-time temperature measurement system can be effectively improved, and meanwhile the real-time performance of the picture can be ensured.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.
Drawings
The drawings are included to provide a better understanding of the present solution and are not intended to limit the present application. Wherein:
FIG. 1 is a schematic diagram according to a first embodiment of the present application;
FIG. 2 is a schematic diagram according to a second embodiment of the present application;
FIG. 3 is a schematic illustration according to a third embodiment of the present application;
FIG. 4 is a schematic illustration according to a fourth embodiment of the present application;
FIG. 5 is a schematic illustration according to a fifth embodiment of the present application;
fig. 6 is a block diagram of an electronic device for implementing the above-described method of an embodiment of the present application.
Detailed Description
The following description of the exemplary embodiments of the present application, taken in conjunction with the accompanying drawings, includes various details of the embodiments of the application for the understanding of the same, which are to be considered exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
FIG. 1 is a schematic diagram according to a first embodiment of the present application; as shown in fig. 1, this embodiment provides a method for synchronizing pictures based on a real-time temperature measurement system, which specifically includes the following steps:
s101, correspondingly generating an infrared picture buffer queue and a visible light picture buffer queue respectively based on a first video stream output by an infrared camera and a second video stream output by a visible light camera;
the execution main body of the picture synchronization method based on the real-time temperature measurement system can be a picture synchronization device based on the real-time temperature measurement system, and the device can be arranged in the real-time temperature measurement system to realize the synchronous processing of the visible light picture and the infrared picture in the real-time temperature measurement system.
The real-time temperature measurement system of the embodiment at least comprises an infrared camera and a visible light camera. Wherein visible light camera and infrared camera can set up to a whole, adopt a binocular camera to realize. Or may be implemented using two separate cameras. However, in the real-time temperature measurement system, the directions of the visible light camera and the infrared camera are consistent, and the same picture is acquired, so that the content information of the picture acquired by the visible light camera, such as various object information of people, animals and the like, can be acquired, and the temperature information in the picture is acquired by the infrared camera. When the infrared light picture and the visible light face are overlapped and displayed together, temperature identification can be carried out on objects of the picture collected by the visible light camera based on temperature information in the picture collected by the infrared camera, and therefore real-time temperature measurement is achieved.
However, due to different parameters of the cameras or other reasons, the infrared picture in the first video stream output by the infrared camera and the visible light picture in the second video stream output by the visible light camera may not be synchronized, thereby causing inaccurate real-time temperature measurement. Based on the problem, the technical scheme of the application is provided.
First, in this embodiment, an infrared picture buffer queue and a visible light picture buffer queue are correspondingly generated based on a first video stream output by an infrared camera and a second video stream output by a visible light camera, respectively. For example, the first video stream output by the infrared camera of the present embodiment may be a video stream in Software Development Kit (SDK) format, and the second video stream output by the visible light camera may be a video stream in SDK format or may also be a video stream in Real Time Streaming Protocol (RTSP) format. In this embodiment, the infrared picture buffer queue may be generated correspondingly based on a first video stream output by the infrared camera, and the visible light picture buffer queue may be generated correspondingly based on a second video stream output by the visible light camera. That is, a plurality of frames of infrared pictures are cached in the infrared picture cache queue, and a plurality of frames of visible light pictures are cached in the visible light picture cache queue.
S102, acquiring a visible light picture and an infrared picture from an infrared picture cache queue and a visible light picture cache queue based on a preconfigured time delay parameter so that the infrared picture and the visible light picture can be synchronously displayed;
s103, pushing the infrared picture and the visible light picture to the front end so that the front end can synchronously display the infrared picture and the visible light picture.
In this embodiment, the infrared picture buffer queue and the visible light picture buffer queue are generated, so that the synchronization of the infrared picture and the visible light picture can be conveniently controlled subsequently. For example, the visible light picture and the infrared picture can be acquired from the infrared picture buffer queue and the visible light picture buffer queue according to the preconfigured time delay parameter, so that the infrared picture and the visible light picture can be synchronously displayed. At this time, the acquired infrared picture and visible light picture are pushed from the front end, and the infrared picture and the visible light picture can be synchronously displayed at the front end.
In the picture synchronization method based on the real-time temperature measurement system, an infrared picture cache queue and a visible light picture cache queue are correspondingly generated based on a first video stream output by an infrared camera and a second video stream output by a visible light camera respectively, and a visible light picture and an infrared picture are acquired from the infrared picture cache queue and the visible light picture cache queue based on a pre-configured time delay parameter, so that the infrared picture and the visible light picture can be synchronously displayed; the infrared picture and the visible light picture are pushed to the front end, so that the front end can synchronously display the infrared picture and the visible light picture, the infrared picture and the visible light picture in the real-time temperature measurement system can be still synchronized when the timestamp is not output in the picture, the temperature measurement accuracy of the real-time temperature measurement system can be effectively improved, and meanwhile, the real-time performance of the picture can be ensured.
FIG. 2 is a schematic diagram according to a second embodiment of the present application; as shown in fig. 2, the method for frame synchronization based on a real-time temperature measurement system according to this embodiment further introduces the technical solution of the present application in more detail on the basis of the technical solution of the embodiment shown in fig. 1. As shown in fig. 2, the picture synchronization method based on the real-time temperature measurement system of this embodiment may specifically include the following steps:
s201, decoding a first video stream output by an infrared camera to obtain a first picture frame sequence;
for example, the first video stream output by the infrared camera may be in an SDK format, and by decoding the first video stream, consecutive frames of infrared pictures of one frame included in the first video stream may be obtained, and the consecutive frames of infrared pictures form a first picture frame sequence according to the order in the first video stream.
S202, sequentially storing multiple frames of infrared pictures in the first picture frame sequence into a queue to generate an infrared picture buffer queue;
the infrared frame buffer queue of the embodiment has the characteristics of a queue, such as complying with the first-in first-out rule. Specifically, multiple frames of infrared pictures in the first picture frame sequence may be sequentially stored in a queue, that is, when the infrared pictures are stored in the queue, the sequence of the infrared pictures in the first video stream is maintained, and then an infrared picture buffer queue is generated.
S203, decoding a second video stream output by the visible light camera to obtain a second picture frame sequence;
for example, the first video stream output by the visible light camera may be in an SDK format or an RTSP stream format, and the second video stream is decoded to obtain visible light pictures of consecutive multiple frames of one frame included in the second video stream, and the visible light pictures of the consecutive multiple frames form a second picture frame sequence according to the order in the second video stream.
S204, performing frame extraction operation on a plurality of continuous image frames in the second image frame sequence to enable a plurality of visible light images obtained after frame extraction to have the same frame rate as a plurality of infrared images in the first image frame sequence;
where the frame rate is the number of frames per second (fps), which can be understood as the number of frames of a picture drawn or included per second.
In this embodiment, when performing frame extraction on a plurality of continuous image frames in the second image frame sequence, the frame rate of the infrared image in the first image frame sequence needs to be referred to, so that the frame rate of a plurality of frames of visible light images obtained after frame extraction is the same as the frame rate of the infrared image in the first image frame sequence.
In addition, optionally, since the real-time thermometry system is generally deployed on the edge device, in order to relieve the stress on the device, the frame extraction operation may also be performed on both the first frame sequence and the second frame sequence. Regardless of whether the frame rates of the original first frame sequence and the second frame sequence are the same or not, after the frames are respectively extracted, it is necessary to ensure that the frame rates of a plurality of frames of visible light images and a plurality of frames of infrared images obtained after the frames are extracted are the same.
S205, sequentially storing the obtained multiple frames of visible light pictures into a queue to generate a visible light picture buffer queue;
in order to implement synchronization of the infrared picture and the visible light picture, in this embodiment, by adopting the above steps, frame rates of the visible light picture and the infrared picture included in the visible light picture buffer queue and the infrared picture buffer queue can be the same, and an effective guarantee is provided for synchronization of the subsequent visible light picture and the infrared picture. And by generating the infrared picture buffer queue and the visible light picture buffer queue, synchronous infrared pictures and visible light pictures can be conveniently acquired subsequently, and necessary conditions are provided for picture synchronization.
The above steps S201 to S205 are an implementation manner of the step S101 in the embodiment shown in fig. 1. Steps S201-S202 are used for generating an infrared image buffer queue; steps S203-S205 are for generating a visible light picture buffer queue.
S206, acquiring a visible light picture and an infrared picture from the infrared picture cache queue and the visible light picture cache queue based on a preconfigured time delay parameter so that the visible light picture and the infrared picture can be synchronously displayed;
it should be noted that the time delay parameter configured in this embodiment may be a pair, for example, the time delay for acquiring the infrared image is a positive integer other than 0, and the time delay for acquiring the visible light image may be 0; or, the time delay for acquiring the infrared image is 0, and the time delay for acquiring the visible light image may be a positive integer other than 0. For example, the step S206 may specifically include the following two cases:
in the first case, the infrared picture needs to be delayed, while the visible light picture does not need to be delayed; at the moment, acquiring an infrared picture from an infrared picture cache queue according to a preset time delay parameter; acquiring a visible light picture from the visible light picture cache queue so as to synchronously display the infrared picture and the visible light picture;
in the second case, the visible light frames need to be delayed, while the infrared frames do not need to be delayed; at the moment, according to a preset time delay parameter, acquiring a visible light picture from a visible light picture cache queue; and acquiring the infrared picture from the infrared picture buffer queue so as to synchronously display the infrared picture and the visible light picture.
In practical applications, which picture is ahead, which picture needs to be delayed, and is affected by hardware devices such as a camera. By configuring the time delay parameter, the infrared picture and the visible light picture obtained from the infrared picture buffer queue and the visible light picture buffer queue can be effectively ensured to be synchronous.
The delay parameter of this embodiment may be preconfigured according to specific requirements, and may be, for example, 200ms, 100ms, or other time lengths.
For example, optionally, before the step S206, a configuration of the delay parameter may be further included, and for example, the following steps may be specifically included:
(a) acquiring a visible light picture and an infrared picture from a visible light picture cache queue and an infrared picture cache queue;
at this time, correspondingly, neither the visible light picture nor the infrared light picture is acquired with time delay.
(b) Based on the obtained visible light picture and the infrared picture, carrying out picture pushing on the front end so as to enable the front end to display the visible light picture and the infrared picture;
(c) acquiring display feedback information;
because the visible light picture and the infrared picture acquired by the method have no time delay, the visible light picture and the infrared picture are simultaneously pushed to the back of the front end, a tester or a user at the front end can see whether the two pictures are synchronous, and when the two pictures are not synchronous, which picture is forward and which picture is backward can be observed by naked eyes. For example, a synchronization button, a selection button with an infrared screen ahead, and a selection button with a visible light screen ahead may be provided in the front end. If the user has seen the picture synchronization, the synchronization button is selected. And if the user sees that the pictures are asynchronous, the user selects the corresponding button before which picture is asynchronous, which indicates that the reason of the asynchronous is that which picture is asynchronous, and feeds the asynchronous back to the real-time temperature measuring system, so that the real-time temperature measuring system can acquire display feedback information.
(d) And configuring a time delay parameter for the picture before the display deviation according to the display feedback information, so that the visible light picture and the infrared picture are synchronous.
At this time, a delay parameter may be configured for the picture before the display, and it should be noted that, the configuration process is configured step by step, and a minimum step length, such as 50ms, 60ms, 80ms, or other time length, may be set. When configuring, firstly configuring the time delay parameter equal to the minimum step length, and after possible one-time configuration, continuing to detect according to the above mode, so as to determine that the visible light picture and the infrared picture are synchronous. And if the display feedback information is detected to be unchanged, namely the picture before the display is deviated, the time delay parameter is increased, namely the minimum step length is increased by 2 times, then the detection is continued, and the like is performed until the infrared picture and the visible light picture which are marked in the display feedback information are synchronized. In this embodiment, by the above adjustment method, the accuracy of the configured delay parameter can be determined, and further, the synchronization of the subsequent infrared picture and the subsequent visible light picture can be ensured.
After the time delay parameters are determined according to the above method, because the frame rates in the infrared image cache queue and the visible light image cache queue are the same, the time stamps of the infrared image and the visible light image acquired according to the time delay parameters are synchronous, and further, the infrared image and the visible light image are ensured to be also synchronous when displayed at the front end.
S207, carrying out face recognition in a visible light picture;
optionally, in this embodiment, the recognition of the face in the visible light image may specifically adopt a preset template or a pre-trained neural network model for recognition.
S208, marking the identified face frame in the visible light picture and/or the infrared picture;
for example, when the real-time temperature measurement system is applied to doorways of public places such as office buildings, shopping malls, supermarkets and the like, the real-time temperature measurement system is mainly used for measuring the temperature of pedestrians, and at the moment, in order to identify the temperature of each person, face recognition can be performed in a visible light picture. And further labeling a human face frame in the visible light picture. Due to the real-time temperature measuring system of the embodiment, the sizes of the infrared picture and the visible light picture displayed at the front end are consistent. The coordinate points of the two synchronous pictures can be shared, for example, the temperature of the (x, y) point in the infrared picture is the temperature of the object at the (x, y) point in the visible light picture. Therefore, the boundary information of the human face, that is, the coordinate information of the human face frame, can be obtained according to the human face recognized in the visible light picture, and then the human face frame is labeled in the visible light picture. And the face frame can be marked in the infrared picture based on the coordinate information of the face frame. The face frame marked in the embodiment may be a rectangular frame, a circle, or other forms.
In practical application, in order to avoid repetition of labeled face frames, the identified face frames can be selected to be labeled in a visible light picture or an infrared picture in an alternative mode.
S209, calculating the temperature of the face frame by adopting a filtering algorithm based on the temperature data of each pixel point in the infrared image, and storing the temperature in an inference result cache module;
for example, if the temperature data of each pixel point in the infrared image collected by the infrared camera has a burr, the burr can be smoothed by using a filtering algorithm and then output. For example, the temperature data of each pixel point in the infrared image may be normalized first; and calculating the temperature of the face frame based on the temperature data of each pixel point after normalization processing. Further, when calculating the temperature of the face frame based on the temperature data of each pixel point after the normalization processing, a point with the highest temperature in the face frame may be detected first, and then the average temperature of a plurality of points, such as 9 points, nearest to the point with the highest temperature may be taken as the temperature of the face frame. The method only aims at the temperature data in the current infrared picture, and calculates the temperature of the human face frame in the current infrared picture or the visible light picture. It should be noted that, in order to facilitate passing management of the temperature information of the face frame in the infrared picture and the visible light picture of each frame, in this embodiment, the calculated temperature of the face frame needs to be stored in the inference result cache module.
In addition, optionally, the temperature of the face frame in each frame of the infrared picture can be calculated in the above manner. Furthermore, the temperature of each face frame in continuous multiframe historical infrared pictures before the infrared picture stored in the inferred result cache module can be obtained through a face tracking technology; and the temperature of the face frame in the infrared picture can be updated based on the temperature of each face frame in the continuous multi-frame historical infrared picture and the temperature of the face frame in the infrared picture. The face tracking technology may refer to related corresponding technologies, which are not described herein again. For example, for each face frame, an average value of the face frame temperatures may be calculated according to the face frame temperature in each frame of historical infrared pictures in the consecutive multiple frames of historical infrared pictures and the face frame temperature in the current infrared picture, and the average value is used as the updated face frame temperature of the current infrared picture.
For example, according to the above manner, a face frame may be labeled in the infrared image, and in order to facilitate execution of the face tracking technology, the face frame may be identified, for example, by using numbers, letters, and combinations thereof. For example, by using a face tracking technology, it can be tracked that the face frame appears in N consecutive frames before the current infrared picture, and at this time, the temperature of the face frame in each of the N consecutive frames needs to be obtained, and then the temperature of the face frame in the N consecutive frames before and the temperature of the face frame in the current infrared picture are averaged together to be used as the temperature of the face frame in the current infrared picture.
Or alternatively, the face tracking technology may be executed in the visible light picture, and by performing face tracking in the continuous visible light picture, it may be tracked that a certain face frame appears in the continuous multi-frame visible light picture until the current visible light picture also appears. And counting, wherein N continuous frames of pictures appear before the current visible light picture. Thus, by referring to the corresponding relationship between each frame of visible light picture and the infrared picture, the temperature of the face frame in the corresponding infrared picture can be calculated based on the face frame in each frame of visible light picture. According to the method, the temperature of the face frame in the current infrared picture and the temperature of the face frame in each frame of infrared pictures in N continuous frames of infrared pictures before the current infrared picture can be calculated, and further the average of the temperature of the face frame in the N continuous frames before and the temperature of the face frame in the current infrared picture can be taken as the temperature of the face frame in the current infrared picture.
The calculation mode of the temperature of the face frame can effectively ensure the accuracy of the calculated face frame temperature.
S210, marking the temperature of the human face frame in a visible light picture and/or an infrared picture;
in practical application, the temperature of the face frame can be marked in each frame of visible light picture and/or infrared picture. However, this may result in the temperature data of the labeled face frame being too dense, so alternatively, the temperature of the face frame may also be labeled according to the frame interval parameter. For example, the picture interval parameter may be the temperature of the face frame marked once every 100ms or other interval duration according to the frame rate.
Optionally, the step S210 takes the temperature of the face frame that needs to be marked in the visible light picture and/or the infrared picture as an example, and optionally before the step S210, the method may further include: and determining the temperature of the face frame to be marked in the visible light picture and/or the infrared picture according to the picture interval parameter, the picture interval between the visible light picture which is the nearest neighbor before the temperature of the face frame and the current visible light picture, and/or the picture interval between the infrared picture which is the nearest neighbor before the temperature of the face frame and the current infrared picture.
The inter-screen interval parameter of the present embodiment can be set based on the user's demand and the performance of the apparatus. For example, the performance of the device may refer to the indexes of a Central Processing Unit (CPU), a disk Input/Output (I/O), and the like of the device in which the real-time temperature measurement system operates, and if the index is too high, the frame interval parameter may be increased. In this embodiment, after the delay parameter and the picture interval parameter are adjusted to appropriate values, the picture synchronizer based on the real-time temperature measurement system can perform synchronous processing on the picture according to the above embodiment, so that the real-time temperature measurement system can operate stably.
It should be noted that, in step S210, the temperature of the face frame marked in the visible light picture and/or the infrared picture may be calculated based on the temperature data of each pixel point in the current infrared picture in step S209. However, this causes a delay in pushing the picture. Alternatively, the temperature of the same face frame in two consecutive frames is theoretically the same, considering that the time difference between two consecutive frames is small. Therefore, step S210 may be: labeling the temperature of the face frame in the previous frame of infrared picture stored in the inference result cache module in the visible light picture and/or the infrared picture; and the temperature of the face frame in the previous frame of infrared picture is also calculated by using step S209. Therefore, the temperature of the face frame calculated in the previous frame can be asynchronously called as the temperature of the face frame in the current frame, the decoupling of the calculation of the face frame temperature and the picture display is realized, the picture delay is avoided, and the picture display has better fluency and real-time performance; and each module can also perform its own function, the pressure of a CPU and a disk is dispersed, the computational requirement of the equipment is reduced, the cost is saved, and the service life of the equipment is prolonged.
The above steps S207 to S210 are the inference service of this embodiment, and on the premise of ensuring the synchronization of the visible light picture and the infrared picture, some services may be further extended to the synchronized visible light picture and infrared picture, such as identifying the temperature of the face in the picture, specifically identifying the temperature of the face by calculating the temperature of the face frame, so as to provide a very convenient real-time temperature display service for the real-time temperature measurement system, and enhance the practicability of the real-time temperature measurement system.
S211, pushing the infrared picture and the visible light picture to the front end so that the front end can synchronously display the infrared picture and the visible light picture.
The infrared picture and the visible light picture acquired in the mode can have synchronism, the infrared picture and the visible light picture are pushed to the front end at the moment, the front end can synchronously display the infrared picture and the visible light picture, the temperature of the face frame is marked in the visible light picture and/or the infrared picture at the moment, and then accurate display of the temperature of the face in the picture can be guaranteed.
By adopting the above scheme, the picture synchronization method based on the real-time temperature measurement system of the embodiment can still realize synchronization of the infrared picture and the visible light picture in the real-time temperature measurement system when the timestamp is not output in the picture, can effectively improve the temperature measurement accuracy of the real-time temperature measurement system, and can also ensure the real-time performance of the picture. In addition, the technical scheme of the embodiment can also display the temperature of the face in the picture in real time, and enhance the practicability of the real-time temperature measurement system; meanwhile, the temperature calculation service of the picture and the face frame can be decoupled, so that the picture display has better fluency and real-time performance.
FIG. 3 is a schematic illustration according to a third embodiment of the present application; as shown in fig. 3, the present embodiment provides a picture synchronization apparatus 300 based on a real-time temperature measurement system, which includes:
the generation module 301 generates an infrared picture buffer queue and a visible light picture buffer queue correspondingly based on a first video stream output by the infrared camera and a second video stream output by the visible light camera, respectively;
the obtaining module 302 obtains the visible light picture and the infrared picture from the infrared picture buffer queue and the visible light picture buffer queue based on the preconfigured time delay parameter, so that the infrared picture and the visible light picture can be synchronously displayed;
the pushing module 303 pushes the infrared picture and the visible light picture to the front end, so that the front end can synchronously display the infrared picture and the visible light picture.
The picture synchronization apparatus 300 based on the real-time temperature measurement system of this embodiment implements the picture synchronization based on the real-time temperature measurement system by using the modules, and the implementation principle and the technical effect are the same as the implementation of the related method embodiment, and reference may be made to the description of the related method embodiment in detail, which is not described herein again.
FIG. 4 is a schematic illustration according to a fourth embodiment of the present application; as shown in fig. 4, the picture synchronization apparatus 300 based on the real-time temperature measurement system of the present embodiment further describes the technical solution of the present application in more detail on the basis of the technical solution of the embodiment shown in fig. 3.
As shown in fig. 4, in the picture synchronization apparatus 300 based on the real-time temperature measurement system of the present embodiment, the generating module 301 includes:
a decoding unit 3011, configured to decode a first video stream output by an infrared camera to obtain a first picture frame sequence;
the storage unit 3012 is configured to store multiple frames of infrared pictures in the first picture frame sequence in a queue in sequence, and generate an infrared picture buffer queue.
Further optionally, the generating module 301 further includes a frame extracting unit 3013;
for example, the decoding unit 3011 is further configured to decode a second video stream output by the visible light camera to obtain a second picture frame sequence;
a frame extracting unit 3013, configured to perform frame extraction on multiple consecutive frame sequences in the second frame sequence, so that multiple frames of visible light frames obtained after frame extraction have the same frame rate as multiple frames of infrared frames in the first frame sequence;
the storage unit 3012 is further configured to store the obtained multiple frames of visible light frames in a queue in sequence, so as to generate a visible light frame buffer queue.
Further optionally, in the picture synchronization apparatus 300 based on the real-time temperature measurement system of the embodiment, the obtaining module 302 is specifically configured to:
acquiring an infrared picture from an infrared picture cache queue according to a preset time delay parameter; acquiring a visible light picture from the visible light picture cache queue so as to synchronously display the infrared picture and the visible light picture; or
Acquiring a visible light picture from a visible light picture cache queue according to a preset time delay parameter; and acquiring the infrared picture from the infrared picture buffer queue so as to synchronously display the infrared picture and the visible light picture.
Further optionally, the picture synchronization apparatus 300 based on the real-time temperature measurement system of the present embodiment further includes a configuration module 304;
the obtaining module 302 is further configured to obtain a visible light picture and an infrared picture from the visible light picture buffer queue and the infrared picture buffer queue;
the pushing module 303 is further configured to perform image pushing to the front end based on the acquired visible light image and infrared image, so that the front end displays the visible light image and the infrared image;
the obtaining module 302 is further configured to obtain display feedback information;
the configuration module 304 is configured to configure a time delay parameter for the pre-display picture according to the display feedback information, so that the visible light picture and the infrared picture are synchronized.
Further optionally, in the picture synchronization apparatus 300 based on the real-time temperature measurement system according to this embodiment, the method further includes:
a face recognition module 305, configured to perform face recognition in a visible light image;
a face labeling module 306, configured to label the identified face frame in the visible light frame and/or the infrared frame;
the temperature calculation module 307 is configured to calculate the temperature of the face frame by using a filtering algorithm based on the temperature data of each pixel point in the infrared image, and store the temperature in the inference result cache module;
the temperature labeling module 308 is further configured to label the temperature of the face frame in the visible light frame and/or the infrared frame.
Further optionally, the temperature calculating module 307 is configured to:
normalizing the temperature data of each pixel point in the infrared picture; and calculating the temperature of the face frame based on the temperature data of each pixel point after normalization processing.
Further optionally, the temperature calculating module 307 is further configured to:
acquiring the temperature of each face frame in continuous multiframe historical infrared pictures before the infrared pictures stored in the inference result cache module by a face tracking technology;
and updating the temperature of the face frame in the infrared picture based on the temperature of each face frame in the continuous multi-frame historical infrared picture and the temperature of the face frame in the infrared picture.
Further optionally, in the picture synchronization apparatus 300 based on the real-time temperature measurement system according to this embodiment, the method further includes:
a determining module 309, configured to determine, according to the picture interval parameter, and a picture interval between a visible light picture immediately before and a current visible light picture, and/or a picture interval between an infrared picture immediately before and a current infrared picture, where the temperature of the face frame is labeled, the temperature of the face frame to be labeled in the visible light picture and/or the infrared picture;
further, the temperature labeling module 308 is specifically configured to:
and labeling the temperature of the face frame in the previous frame of infrared picture stored in the inference result cache module in the visible light picture and/or the infrared picture.
The picture synchronization apparatus 300 based on the real-time temperature measurement system of this embodiment implements the picture synchronization based on the real-time temperature measurement system by using the modules, and the implementation principle and the technical effect are the same as the implementation of the related method embodiment, and reference may be made to the description of the related method embodiment in detail, which is not described herein again.
FIG. 5 is a schematic illustration according to a fifth embodiment of the present application; as shown in fig. 5, the real-time temperature measurement system 500 of the present embodiment includes: a visible light camera 501, an infrared camera 502 and a picture synchronization device 503 based on a real-time temperature measurement system; the picture synchronization device 503 based on the real-time temperature measurement system is respectively in communication connection with the visible light camera 501 and the infrared camera 502, so that the visible light picture of the visible light camera 501 and the infrared picture of the infrared camera 502 are synchronously processed. Specifically, the frame synchronization device 503 based on the real-time temperature measurement system may specifically adopt the frame synchronization device based on the real-time temperature measurement system shown in fig. 3 or fig. 4, and adopt the frame synchronization method based on the real-time temperature measurement system shown in fig. 1 or fig. 2, so as to implement frame synchronization processing.
According to an embodiment of the present application, an electronic device and a readable storage medium are also provided.
Fig. 6 is a block diagram of an electronic device implementing the method according to the embodiment of the present application. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application that are described and/or claimed herein.
As shown in fig. 6, the electronic apparatus includes: one or more processors 601, memory 602, and interfaces for connecting the various components, including a high-speed interface and a low-speed interface. The various components are interconnected using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions for execution within the electronic device, including instructions stored in or on the memory to display graphical information of a GUI on an external input/output apparatus (such as a display device coupled to the interface). In other embodiments, multiple processors and/or multiple buses may be used, along with multiple memories and multiple memories, as desired. Also, multiple electronic devices may be connected, with each device providing portions of the necessary operations (e.g., as a server array, a group of blade servers, or a multi-processor system). In fig. 6, one processor 601 is taken as an example.
The memory 602 is a non-transitory computer readable storage medium as provided herein. The memory stores instructions executable by at least one processor, so that the at least one processor executes the picture synchronization method based on the real-time temperature measurement system provided by the application. The non-transitory computer readable storage medium of the present application stores computer instructions for causing a computer to perform the real-time thermometry system based picture synchronization method provided herein.
The memory 602, serving as a non-transitory computer readable storage medium, may be used to store non-transitory software programs, non-transitory computer executable programs, and modules, such as program instructions/modules (e.g., related modules shown in fig. 3 and 4) corresponding to the frame synchronization method based on the real-time thermometry system in the embodiments of the present application. The processor 601 executes various functional applications and data processing of the server by running non-transitory software programs, instructions and modules stored in the memory 602, that is, implements the picture synchronization method based on the real-time thermometry system in the above method embodiment.
The memory 602 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to use of the electronic device implementing the picture synchronization method based on the real-time temperature measurement system, and the like. Further, the memory 602 may include high speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory 602 optionally includes memory remotely located from the processor 601, and these remote memories may be connected over a network to an electronic device implementing a real-time thermometry system based picture synchronization method. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The electronic device for implementing the picture synchronization method based on the real-time temperature measurement system may further include: an input device 603 and an output device 604. The processor 601, the memory 602, the input device 603 and the output device 604 may be connected by a bus or other means, and fig. 6 illustrates the connection by a bus as an example.
The input device 603 may receive input numeric or character information and generate key signal inputs related to user settings and function control of the electronic device implementing the picture synchronization method based on the real-time temperature measurement system, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, a pointing stick, one or more mouse buttons, a track ball, a joystick, or other input devices. The output devices 604 may include a display device, auxiliary lighting devices (e.g., LEDs), and tactile feedback devices (e.g., vibrating motors), among others. The display device may include, but is not limited to, a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display, and a plasma display. In some implementations, the display device can be a touch screen.
Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, application specific ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
These computer programs (also known as programs, software applications, or code) include machine instructions for a programmable processor, and may be implemented using high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), the internet, and blockchain networks.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
According to the technical scheme of the embodiment of the application, an infrared picture buffer queue and a visible light picture buffer queue are correspondingly generated based on a first video stream output by an infrared camera and a second video stream output by a visible light camera respectively, and a visible light picture and an infrared picture are obtained from the infrared picture buffer queue and the visible light picture buffer queue based on a pre-configured time delay parameter, so that the infrared picture and the visible light picture can be synchronously displayed; the infrared picture and the visible light picture are pushed to the front end, so that the front end can synchronously display the infrared picture and the visible light picture, the infrared picture and the visible light picture in the real-time temperature measurement system can be still synchronized when the timestamp is not output in the picture, the temperature measurement accuracy of the real-time temperature measurement system can be effectively improved, and meanwhile, the real-time performance of the picture can be ensured.
According to the technical scheme of the embodiment of the application, by adopting the scheme, the synchronization of the infrared picture and the visible light picture in the real-time temperature measurement system can be still realized when the timestamp is not output in the picture, the temperature measurement accuracy of the real-time temperature measurement system can be effectively improved, and meanwhile, the real-time performance of the picture can be ensured. In addition, the technical scheme of the embodiment can also display the temperature of the face in the picture in real time, and enhance the practicability of the real-time temperature measurement system; meanwhile, the temperature calculation service of the picture and the face frame can be decoupled, so that the picture display has better fluency and real-time performance.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present application may be executed in parallel, sequentially, or in different orders, and the present invention is not limited thereto as long as the desired results of the technical solutions disclosed in the present application can be achieved.
The above-described embodiments should not be construed as limiting the scope of the present application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims (21)
1. A picture synchronization method based on a real-time temperature measurement system is disclosed, wherein the method comprises the following steps:
correspondingly generating an infrared picture cache queue and a visible light picture cache queue respectively based on a first video stream output by an infrared camera and a second video stream output by a visible light camera;
based on a pre-configured time delay parameter, acquiring a visible light picture and an infrared picture from the infrared picture cache queue and the visible light picture cache queue so as to enable the infrared picture and the visible light picture to be displayed synchronously;
and pushing the infrared picture and the visible light picture to the front end so that the front end can synchronously display the infrared picture and the visible light picture.
2. The method of claim 1, wherein correspondingly generating an infrared picture buffer queue based on the first video stream output by the infrared camera comprises:
decoding the first video stream output by the infrared camera to obtain a first picture frame sequence;
and sequentially storing multiple frames of infrared pictures in the first picture frame sequence into a queue to generate the infrared picture buffer queue.
3. The method of claim 2, wherein correspondingly generating a visible light picture buffer queue based on the second video stream output by the visible light camera comprises:
decoding the second video stream output by the visible light camera to obtain a second picture frame sequence;
performing frame extraction on a plurality of continuous image frames in the second image frame sequence to enable a plurality of visible light images obtained after frame extraction to have the same frame rate as a plurality of infrared images in the first image frame sequence;
and sequentially storing the obtained multiple frames of visible light pictures into a queue to generate the visible light picture buffer queue.
4. The method of claim 1, wherein acquiring a visible light picture and an infrared picture from the infrared picture buffer queue and the visible light picture buffer queue based on a preconfigured delay parameter so that the infrared picture and the visible light picture can be displayed synchronously comprises:
acquiring the infrared picture from the infrared picture cache queue according to the preset time delay parameter; acquiring the visible light picture from the visible light picture cache queue so as to synchronously display the infrared picture and the visible light picture; or
Acquiring the visible light picture from the visible light picture cache queue according to the preset time delay parameter; and acquiring the infrared picture from the infrared picture buffer queue so as to synchronously display the infrared picture and the visible light picture.
5. The method of claim 1, wherein before acquiring the visible light picture and the infrared picture from the infrared picture buffer queue and the visible light picture buffer queue based on a preconfigured delay parameter so that the infrared picture and the visible light picture can be displayed synchronously, the method comprises:
acquiring a visible light picture and an infrared picture from the visible light picture cache queue and the infrared picture cache queue;
based on the obtained visible light picture and the infrared picture, carrying out picture pushing on a front end so as to enable the front end to display the visible light picture and the infrared picture;
acquiring display feedback information;
and configuring the time delay parameter for the picture before the display according to the display feedback information, so that the visible light picture and the infrared picture are synchronous.
6. The method according to any one of claims 1 to 5, wherein visible light pictures and infrared pictures are acquired from the infrared picture buffer queue and the visible light picture buffer queue based on a preconfigured time delay parameter, so that after the infrared pictures and the visible light pictures can be displayed synchronously, before the infrared pictures and the visible light pictures are pushed to a front end, the method further comprises:
carrying out face recognition in the visible light picture;
marking the identified face frame in the visible light picture and/or the infrared picture;
calculating the temperature of the face frame by adopting a filtering algorithm based on the temperature data of each pixel point in the infrared image, and storing the temperature in an inference result cache module;
and marking the temperature of the human face frame in the visible light picture and/or the infrared picture.
7. The method of claim 6, wherein calculating the temperature of the face frame using a filtering algorithm based on the temperature data of each pixel in the infrared image comprises:
normalizing the temperature data of each pixel point in the infrared picture; and calculating the temperature of the face frame based on the temperature data of each pixel point after normalization processing.
8. The method of claim 7, wherein calculating the temperature of the face frame using a filtering algorithm based on the temperature data of each pixel in the infrared image further comprises:
acquiring the temperature of each face frame in continuous multi-frame historical infrared pictures before the infrared picture stored in the inference result cache module by a face tracking technology;
and updating the temperature of the face frame in the infrared picture based on the temperature of each face frame in the continuous multi-frame historical infrared picture and the temperature of the face frame in the infrared picture.
9. The method of claim 6, wherein prior to labeling the temperature of the face box in the visible light frame and/or the infrared frame, the method further comprises:
determining the temperature of the face frame to be marked in the visible light picture and/or the infrared picture according to picture interval parameters, the picture interval between the visible light picture which is the nearest neighbor of the temperature of the face frame and the current visible light picture, and/or the picture interval between the infrared picture which is the nearest neighbor of the temperature of the face frame and the current infrared picture;
further, the marking the temperature of the face frame in the visible light picture and/or the infrared picture includes:
and labeling the temperature of the face frame in the previous frame of infrared picture stored in the inference result cache module in the visible light picture and/or the infrared picture.
10. A picture synchronization device based on a real-time temperature measurement system, wherein the device comprises:
the generating module is used for correspondingly generating an infrared picture cache queue and a visible light picture cache queue respectively based on a first video stream output by the infrared camera and a second video stream output by the visible light camera;
the acquisition module is used for acquiring a visible light picture and an infrared picture from the infrared picture cache queue and the visible light picture cache queue based on a preconfigured time delay parameter so as to enable the infrared picture and the visible light picture to be displayed synchronously;
and the pushing module is used for pushing the infrared picture and the visible light picture to the front end so that the front end can synchronously display the infrared picture and the visible light picture.
11. The apparatus of claim 10, wherein the generating means comprises:
a decoding unit, configured to decode the first video stream output by the infrared camera to obtain a first picture frame sequence;
and the storage unit is used for sequentially storing multiple frames of infrared pictures in the first picture frame sequence into a queue to generate the infrared picture buffer queue.
12. The apparatus of claim 11, wherein the generating means further comprises a framing unit;
the decoding unit is further configured to decode the second video stream output by the visible light camera to obtain a second picture frame sequence;
the frame extracting unit is used for performing frame extracting operation on a plurality of continuous image frames in the second image frame sequence, so that a plurality of visible light images obtained after frame extracting have the same frame rate as a plurality of infrared images in the first image frame sequence;
the storage unit is further configured to sequentially store the obtained multiple frames of visible light frames in a queue, and generate the visible light frame buffer queue.
13. The apparatus according to claim 10, wherein the obtaining module is specifically configured to:
acquiring the infrared picture from the infrared picture cache queue according to the preset time delay parameter; acquiring the visible light picture from the visible light picture cache queue so as to synchronously display the infrared picture and the visible light picture; or
Acquiring the visible light picture from the visible light picture cache queue according to the preset time delay parameter; and acquiring the infrared picture from the infrared picture buffer queue so as to synchronously display the infrared picture and the visible light picture.
14. The apparatus of claim 10, wherein the apparatus further comprises a configuration module;
the acquisition module is further used for acquiring the visible light picture and the infrared picture from the visible light picture cache queue and the infrared picture cache queue;
the pushing module is further configured to push a picture to a front end based on the acquired visible light picture and the acquired infrared picture, so that the front end can display the visible light picture and the infrared picture;
the acquisition module is also used for acquiring display feedback information;
and the configuration module is used for configuring the time delay parameter for the picture before the display deviation according to the display feedback information so as to enable the visible light picture and the infrared picture to be synchronous.
15. The apparatus of any of claims 10-14, wherein the apparatus further comprises:
the face recognition module is used for carrying out face recognition in the visible light picture;
the human face labeling module is used for labeling the recognized human face frame in the visible light picture and/or the infrared picture;
the temperature calculation module is used for calculating the temperature of the face frame by adopting a filtering algorithm based on the temperature data of each pixel point in the infrared image and storing the temperature in the inference result cache module;
and the temperature marking module is also used for marking the temperature of the human face frame in the visible light picture and/or the infrared picture.
16. The apparatus of claim 15, wherein the temperature calculation module is to:
normalizing the temperature data of each pixel point in the infrared picture; and calculating the temperature of the face frame based on the temperature data of each pixel point after normalization processing.
17. The apparatus of claim 16, wherein the temperature calculation module is further configured to:
acquiring the temperature of each face frame in continuous multi-frame historical infrared pictures before the infrared picture stored in the inference result cache module by a face tracking technology;
and updating the temperature of the face frame in the infrared picture based on the temperature of each face frame in the continuous multi-frame historical infrared picture and the temperature of the face frame in the infrared picture.
18. The apparatus of claim 15, wherein the apparatus further comprises:
the determining module is used for determining the temperature of the face frame to be marked in the visible light picture and/or the infrared picture according to picture interval parameters, and the picture interval between the visible light picture which is the nearest neighbor of the temperature of the face frame and the current visible light picture, and/or the picture interval between the infrared picture which is the nearest neighbor of the temperature of the face frame and the current infrared picture;
further, the temperature labeling module is specifically configured to:
and labeling the temperature of the face frame in the previous frame of infrared picture stored in the inference result cache module in the visible light picture and/or the infrared picture.
19. A real-time thermometry system, comprising: a visible light camera, an infrared camera and a picture synchronization device based on a real-time temperature measurement system as claimed in any one of the above claims 10-18; the picture synchronization device based on the real-time temperature measurement system is in communication connection with the visible light camera and the infrared camera respectively, and realizes synchronization processing of visible light pictures of the visible light camera and infrared pictures of the infrared camera.
20. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-9.
21. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010699254.2A CN111970414A (en) | 2020-07-20 | 2020-07-20 | Picture synchronization method and device based on real-time temperature measurement system and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010699254.2A CN111970414A (en) | 2020-07-20 | 2020-07-20 | Picture synchronization method and device based on real-time temperature measurement system and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111970414A true CN111970414A (en) | 2020-11-20 |
Family
ID=73361699
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010699254.2A Pending CN111970414A (en) | 2020-07-20 | 2020-07-20 | Picture synchronization method and device based on real-time temperature measurement system and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111970414A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022156721A1 (en) * | 2021-01-20 | 2022-07-28 | 华为技术有限公司 | Photographing method and electronic device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120086568A1 (en) * | 2010-10-06 | 2012-04-12 | Microsoft Corporation | Inferring Building Metadata From Distributed Sensors |
| CN102714708A (en) * | 2010-02-12 | 2012-10-03 | 汤姆森特许公司 | Method for synchronized content playback |
| CN107770570A (en) * | 2017-09-13 | 2018-03-06 | 深圳天珑无线科技有限公司 | Audio video synchronization player method, system and computer-readable recording medium |
| US20190082519A1 (en) * | 2017-09-13 | 2019-03-14 | Essential Products, Inc. | Display and a light sensor operable as an infrared emitter and infrared receiver |
| CN110830678A (en) * | 2019-11-14 | 2020-02-21 | 威创集团股份有限公司 | Multi-channel video signal synchronous output method, device, system and medium |
| CN111327788A (en) * | 2020-02-28 | 2020-06-23 | 北京迈格威科技有限公司 | Synchronization method, temperature measurement method and device of camera set and electronic system |
-
2020
- 2020-07-20 CN CN202010699254.2A patent/CN111970414A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102714708A (en) * | 2010-02-12 | 2012-10-03 | 汤姆森特许公司 | Method for synchronized content playback |
| US20120086568A1 (en) * | 2010-10-06 | 2012-04-12 | Microsoft Corporation | Inferring Building Metadata From Distributed Sensors |
| CN107770570A (en) * | 2017-09-13 | 2018-03-06 | 深圳天珑无线科技有限公司 | Audio video synchronization player method, system and computer-readable recording medium |
| US20190082519A1 (en) * | 2017-09-13 | 2019-03-14 | Essential Products, Inc. | Display and a light sensor operable as an infrared emitter and infrared receiver |
| CN110830678A (en) * | 2019-11-14 | 2020-02-21 | 威创集团股份有限公司 | Multi-channel video signal synchronous output method, device, system and medium |
| CN111327788A (en) * | 2020-02-28 | 2020-06-23 | 北京迈格威科技有限公司 | Synchronization method, temperature measurement method and device of camera set and electronic system |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022156721A1 (en) * | 2021-01-20 | 2022-07-28 | 华为技术有限公司 | Photographing method and electronic device |
| EP4254927A4 (en) * | 2021-01-20 | 2024-05-08 | Huawei Technologies Co., Ltd. | Photographing method and electronic device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109716769B (en) | System and method for providing object-oriented scaling in multimedia information | |
| CN111327788B (en) | Synchronization method, temperature measurement method and device of camera set and electronic system | |
| CN113163260B (en) | Video frame output control method and device and electronic equipment | |
| CN110659600B (en) | Object detection method, device and equipment | |
| JP6986187B2 (en) | Person identification methods, devices, electronic devices, storage media, and programs | |
| CN112584077B (en) | Video frame interpolation method and device and electronic equipment | |
| CN112584076B (en) | Video frame interpolation method and device and electronic equipment | |
| CN111524166A (en) | Video frame processing method and device | |
| CN111998959B (en) | Temperature calibration method and device based on real-time temperature measurement system and storage medium | |
| KR102496334B1 (en) | Method and device for detecting body temperature, electronic apparatus and storage medium | |
| CN112530021A (en) | Method, apparatus, device and storage medium for processing data | |
| WO2023103329A1 (en) | Data labeling method, apparatus, and system, device, and storage medium | |
| CN111601013A (en) | Method and apparatus for processing video frames | |
| CN111970414A (en) | Picture synchronization method and device based on real-time temperature measurement system and storage medium | |
| CN111008305B (en) | Visual search method and device and electronic equipment | |
| CN114187392B (en) | Virtual even image generation method and device and electronic equipment | |
| CN114554110A (en) | Video generation method and device, electronic equipment and storage medium | |
| CN111860163A (en) | Image processing method and device, electronic equipment and readable storage medium | |
| CN111696134A (en) | Target detection method and device and electronic equipment | |
| CN112270303A (en) | Image recognition method and device and electronic equipment | |
| CN110809041A (en) | Data synchronization method and device, electronic equipment and storage medium | |
| CN110477866A (en) | Detect method, apparatus, electronic equipment and the storage medium of sleep quality | |
| US20230298143A1 (en) | Object removal during video conferencing | |
| CN112015951B (en) | Video monitoring method, device, electronic equipment and computer readable medium | |
| CN115022722A (en) | Video monitoring method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20201120 |