CN115876334B - Infrared thermal image indoor temperature measurement method, system and computer readable storage medium - Google Patents
Infrared thermal image indoor temperature measurement method, system and computer readable storage medium Download PDFInfo
- Publication number
- CN115876334B CN115876334B CN202310148242.4A CN202310148242A CN115876334B CN 115876334 B CN115876334 B CN 115876334B CN 202310148242 A CN202310148242 A CN 202310148242A CN 115876334 B CN115876334 B CN 115876334B
- Authority
- CN
- China
- Prior art keywords
- feature map
- decoding
- module
- temperature detection
- thermal image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Radiation Pyrometers (AREA)
Abstract
The invention discloses an infrared thermal image indoor temperature measurement method, an infrared thermal image indoor temperature measurement system and a computer readable storage medium. Wherein the method comprises the following steps: acquiring a real-time thermal image and a real-time high-definition image outside a unit building; inputting the real-time high-definition image into a pre-trained target detection deep learning network to perform window positioning, and obtaining window position information; dividing the real-time thermal image according to the window position information to obtain a window thermal image; and inputting the window thermal image into a pre-trained first temperature detection model to perform temperature identification, and obtaining an indoor temperature detection result, wherein the first temperature detection model is obtained by distillation training of a second temperature detection model. According to the method, the temperature detection is carried out on the obtained thermal image of the outer window of the unit building, so that the real-time detection of the indoor temperature is realized.
Description
Technical Field
The present invention relates to the field of image processing, and in particular, to an infrared thermal imaging indoor temperature measurement method, system and computer readable storage medium.
Background
In a central heating system, heat waste is reduced while indoor temperature is guaranteed for heat supply in a building, and heat supply and heat preservation are at a certain dynamic balance degree, so that heat loss is low and daily unaffected of users is guaranteed. In view of this requirement, real-time monitoring of the indoor temperature of the user is required to ensure that the indoor temperature can be maintained at a temperature suitable for the user under the condition of changing the external temperature, so that monitoring of the indoor temperature is an important task in heat supply engineering. The infrared thermal imaging detection has the characteristics of long distance, non-contact, high precision, quick dynamic response, visual image and the like, so that the infrared thermal imaging detection is applied to indoor temperature monitoring, more area observation can be carried out on the room temperature of a user, and abnormal loss conditions of the room temperature of the user can be observed, thereby dynamically adjusting a heating strategy. In the related art, a method for rapidly acquiring indoor air temperature from outdoor in batches through infrared thermal imaging is used for acquiring thermal images of a user window area, thermal image analysis and calculation are carried out according to different window opening degrees and according to a set threshold value and an outdoor temperature measurement result through a heat transfer model calculation formula, so that the indoor temperature is calculated, but different buildings have different convection heat exchange coefficients, so that multiplexing efficiency is low in each region, and a certain error exists in the actually measured temperature.
Disclosure of Invention
The present invention aims to solve at least one of the technical problems in the related art to some extent.
Therefore, a first object of the present invention is to provide an infrared thermal image indoor temperature measurement method, which realizes real-time detection of indoor temperature by detecting the temperature of an obtained thermal image of an exterior window of a unit building.
A second object of the present invention is to propose a computer readable storage medium.
The third purpose of the invention is to provide an infrared thermal imaging indoor temperature measurement system.
In order to achieve the above object, an embodiment of a first aspect of the present invention provides an infrared thermal imaging indoor temperature measurement method, the method including: acquiring a real-time thermal image and a real-time high-definition image outside a unit building; inputting the real-time high-definition image into a pre-trained target detection deep learning network to perform window positioning, and obtaining window position information; dividing the real-time thermal image according to the window position information to obtain a window thermal image; and inputting the window thermal image into a pre-trained first temperature detection model to perform temperature identification, and obtaining an indoor temperature detection result, wherein the first temperature detection model is obtained by distillation training of a second temperature detection model.
According to one embodiment of the present invention, the frame position information includes a coordinate set of each frame, where the dividing the real-time thermal image according to the frame position information includes:
and cutting the real-time thermal image according to the coordinate set of each window so as to obtain a window thermal image corresponding to each window of the unit building.
According to one embodiment of the invention, before training the first temperature detection model, the method further comprises: acquiring a thermal image sample image, a high-definition sample image and temperature measurement sample data; inputting the Gao Qingyang image into the target detection deep learning network to perform window positioning to obtain positioning coordinates; cutting the thermal image sample image according to the positioning coordinates to obtain a preliminary window thermal image; and screening the preliminary window thermal image according to the temperature measurement sample data to obtain training sample data.
According to one embodiment of the invention, after obtaining the training sample data, the method further comprises:
the training sample data are sent to an automatic training server, wherein the automatic training server trains the second temperature detection model according to the training sample data, and the trained knowledge of the second temperature detection model is taught to the first temperature detection model in a distillation learning mode so as to train the first temperature detection model; and receiving a trained first temperature detection model fed back by the automatic training server.
According to one embodiment of the invention, the first temperature detection model is an SRAnet-student deep learning network model, and the second temperature detection model is an SRAnet-teacher deep learning network model.
According to one embodiment of the present invention, the first temperature detection model includes a first convolution module, a first spatial cyclic neural module, a second spatial cyclic neural module, a third spatial cyclic neural module, a fourth spatial cyclic neural module, a first linking module, a first decoding module, and a second decoding module, where inputting the window thermal image map into a pre-trained first temperature detection model performs temperature identification, and includes: performing feature extraction processing on the window thermal image graph through the first convolution module to obtain a first feature graph;
the first space circulation nerve module, the second space circulation nerve module, the third space circulation nerve module and the fourth space circulation nerve module are used for extracting features of the first feature map in different directions respectively to obtain a first direction feature map, a second direction feature map, a third direction feature map and a fourth direction feature map; the first link module is used for carrying out link processing on the first feature map, the first direction feature map, the second direction feature map, the third direction feature map and the fourth direction feature map to obtain an encoding feature map; decoding the coding feature map through the first decoding module to obtain a first decoding intermediate feature map; and decoding the first decoding intermediate feature map through the second decoding module to obtain the indoor temperature detection result.
According to one embodiment of the present invention, the first decoding module includes a first multi-head self-attention layer, a first normalization layer, a first feed-forward network layer, and a second normalization layer that are sequentially connected, where decoding the encoded feature map by the first decoding module includes: performing feature extraction on the coding feature map through the first multi-head self-attention layer to obtain a first decoding feature map; normalizing the coding feature map and the first decoding feature map through the first normalization layer to obtain a second decoding feature map; processing the second decoding feature map through the first feedforward network layer to obtain a third decoding feature map; and normalizing the second decoding feature map and the third decoding feature map through the second normalization layer to obtain the first decoding intermediate feature map.
According to an embodiment of the present invention, the second decoding module includes a second multi-head self-attention layer, a third normalization layer, a second feedforward network layer, and a fourth normalization layer connected in sequence, where decoding the first decoding intermediate feature map by the second decoding module includes: performing feature extraction on the first decoding intermediate feature map through the second multi-head self-attention layer to obtain a fourth decoding feature map; normalizing the first decoding intermediate feature map and the fourth decoding feature map through the third normalization layer to obtain a fifth decoding feature map; processing the fifth decoding feature map through the second feedforward network layer to obtain a sixth decoding feature map; and carrying out normalization processing on the fifth decoding feature map and the sixth decoding feature map through the fourth normalization layer to obtain the indoor temperature detection result.
According to the infrared thermal image indoor temperature measurement method, a real-time thermal image and a real-time high-definition image outside a unit building are obtained; inputting the real-time high-definition image into a pre-trained target detection deep learning network to perform window positioning, and obtaining window position information; dividing the real-time thermal image according to the window position information to obtain a window thermal image; and inputting the window thermal image into a pre-trained first temperature detection model to perform temperature identification, and obtaining an indoor temperature detection result, so that the internal relation of the thermal image on spatial distribution and the internal relation of the characteristics of the thermal image are extracted, the indoor temperature is predicted by combining the characteristic distribution conditions, and the efficiency of real-time prediction of the indoor temperature is realized by further lightening the first temperature detection model through a distillation learning method.
To achieve the above object, an embodiment of a second aspect of the present invention provides a computer readable storage medium having a computer program stored thereon, where the computer program when executed by a processor implements an infrared thermal image indoor temperature measurement method according to the embodiment of the first aspect of the present invention.
In order to achieve the above embodiments, an embodiment of a third aspect of the present invention provides an infrared thermal imaging indoor temperature measurement system, including: the binocular thermal imaging system comprises a binocular thermal imaging system and a 3D (three-dimensional) cradle head, wherein the 3D cradle head is used for carrying the binocular thermal imaging system so as to shoot images outside a unit building and obtain a real-time thermal image and a real-time high-definition image; the miniature embedded control analysis device is used for inputting the real-time high-definition image into a pre-trained target detection deep learning network to perform window positioning, obtaining window position information, dividing the real-time thermal image according to the window position information to obtain a window thermal image, inputting the window thermal image into a pre-trained first temperature detection model to perform temperature identification, and obtaining an indoor temperature detection result, wherein the first temperature detection model is obtained by distillation training of a second temperature detection model.
Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Drawings
FIG. 1 is a flow chart of an infrared thermography indoor temperature measurement method according to one embodiment of the present invention;
FIG. 2 is a flow chart of an infrared thermography indoor temperature measurement method according to a first embodiment of the present invention;
FIG. 3 is a schematic diagram of a first temperature detection model distillation learning a second temperature detection model according to one embodiment of the present invention.
FIG. 4 is a schematic diagram of the structure of a first temperature detection model according to one embodiment of the present invention;
FIG. 5 is a schematic structural diagram of a first temperature detection model according to one embodiment of the present application;
fig. 6 is a schematic diagram of a first decoding module according to an embodiment of the present invention;
fig. 7 is a schematic diagram of a second decoding module according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of a second temperature detection model according to an embodiment of the present invention;
FIG. 9 is a schematic diagram of the structure of a computer-readable storage medium according to one embodiment of the invention;
FIG. 10 is a schematic diagram of an infrared thermographic indoor temperature measurement system according to an embodiment of the present invention;
FIG. 11 is a schematic workflow diagram of an automatic training server in accordance with one embodiment of the present invention.
Description of the reference numerals:
410. a first convolution module; 420. a first spatially recurrent neural module; 430. a second spatially recurrent neural module; 440. a third spatially recurrent neural module; 450. a fourth spatially recurrent neural module; 460. a first linking module; 470. a first decoding module; 480. a second decoding module; 610. a first multi-headed self-attention layer; 620. a first normalization layer; 630. a first feed forward network layer; 640. a second normalization layer; 710. a second multi-headed self-attention layer; 720. a third normalization layer; 730. a second feed forward network layer; 740. a fourth normalization layer; 900. a computer-readable storage medium; 901. a computer program; 1000. an infrared thermal image indoor temperature measurement system; 1010. a binocular thermal imager; 1020. 3D cradle head; 1030. miniature embedded control analysis device.
Detailed Description
Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are illustrative and intended to explain the present invention and should not be construed as limiting the invention.
An infrared thermal imaging indoor temperature measurement method, system and computer readable storage medium according to embodiments of the present invention are described below with reference to fig. 1-11.
FIG. 1 is a flow chart of an infrared thermal imaging indoor temperature measurement method according to an embodiment of the invention.
As shown in FIG. 1, the method for measuring the temperature in the infrared thermal image room can comprise the following steps:
s110, acquiring a real-time thermal image and a real-time high-definition image outside the unit building.
The shooting device can be placed outside the unit building to obtain a real-time thermal image and a real-time high-definition image, wherein the shooting device can comprise a high-definition camera and an infrared thermal image camera or an infrared dual-view image shooting device, the high-definition camera is used for shooting the real-time high-definition image outside the unit building, the infrared thermal image camera is used for shooting the real-time thermal image outside the unit building, the positions of the high-definition camera and the infrared camera are arranged at the same position outside the unit building, and the high-definition camera and the infrared camera can simultaneously obtain the real-time thermal image and the real-time high-definition image of the same shooting angle under the same scene in a camera calibration mode.
For example, the photographing devices of the real-time thermal image and the real-time high-definition image may be disposed at intermediate positions outside the two unit buildings, and the real-time thermal image and the real-time high-definition image of the two unit buildings are obtained by controlling the photographing devices to rotate.
S120, inputting the real-time high-definition image into a pre-trained target detection deep learning network to perform window positioning, and obtaining window position information.
S130, dividing the real-time thermal image according to the window position information to obtain a window thermal image.
Further, detecting the position of the window in the real-time high-definition image through the target detection deep learning network to obtain the position information of the window in the real-time high-definition image. And dividing the real-time thermal image according to the window position information, and dividing the window part in the real-time thermal image to obtain a window thermal image.
It can be understood that the real-time thermal image and the real-time high-definition image are taken as images outside the unit building of the same time, the same angle and the same scene, so that the part of the window can be segmented from the real-time thermal image according to the position information of the window in the real-time high-definition image.
The target detection deep learning network may be a target detection network model such as YOLOv5, SSD, R-CNN, SPP-net, fast R-CNN, etc., which is not particularly limited in this regard.
S140, inputting the window thermal image into a pre-trained first temperature detection model for temperature identification, and obtaining an indoor temperature detection result, wherein the first temperature detection model is obtained by distillation training of a second temperature detection model.
Specifically, the segmented window thermal image is input into a first temperature detection model trained in advance to predict the temperature of the window, wherein the temperature of the window can represent the indoor temperature, and an indoor temperature detection result is further obtained through the prediction of the temperature of the window.
In some embodiments of the present invention, the frame position information includes a coordinate set of each frame, where the dividing the real-time thermal image according to the frame position information includes: and cutting the real-time thermal image according to the coordinate set of each window so as to obtain a window thermal image corresponding to each window of the unit building.
It can be understood that the obtained real-time high-definition image outside the unit building comprises a plurality of frames, the position of each frame in the real-time high-definition image is detected through the target detection deep learning network, the coordinate position of each frame position in the real-time high-definition image can be obtained, and the real-time thermal image is segmented according to the coordinate set of each frame position, so that the window thermal image corresponding to each frame of the unit building is obtained.
FIG. 2 is a flow chart of an infrared thermal imaging indoor temperature measurement method according to a first embodiment of the invention.
As shown in fig. 2, before training the first temperature detection model, the following steps may be included:
s210, acquiring a thermal image, a high-definition sample image and temperature measurement sample data.
S220, inputting the high-definition sample image into a target detection deep learning network to perform window positioning, and obtaining positioning coordinates.
S230, cutting the thermal image sample image according to the positioning coordinates to obtain a preliminary window thermal image.
S240, screening the preliminary window thermal image according to the temperature measurement sample data to obtain training sample data.
For example, the thermometric sample data may be obtained from thermometric data of a thermometric sensor disposed indoors. Further positioning the position of a window in the high-definition sample image through a target detection deep learning network to obtain positioning coordinates; cutting out all window parts in the thermal image sample image according to the positioning coordinates of the window, and attaching coordinate information of the window to obtain a preliminary window thermal image. And screening the preliminary window thermal image according to window coordinates close to the temperature sensor, and screening the window thermal image close to the temperature sensor to obtain training sample data of the window thermal image related to temperature measurement data corresponding to the window.
After obtaining the training sample data in some embodiments of the invention, the method further comprises: the training sample data are sent to an automatic training server, wherein the automatic training server trains the second temperature detection model according to the training sample data, and transmits knowledge learned by the trained second temperature detection model to the first temperature detection model in a distillation learning mode so as to train the first temperature detection model; and receiving a trained first temperature detection model fed back by the automatic training server.
As one example, the first temperature detection model is an SRAnet-student deep learning network model and the second temperature detection model is an SRAnet-teacher deep learning network model.
Illustratively, FIG. 3 is a schematic diagram of a first temperature detection model distill learning a second temperature detection model in accordance with one embodiment of the present invention. As shown in fig. 3, the second temperature detection model SRAnet-teacher is a deep learning network model after training, when the second temperature detection model is SRAnet-teacher, loss is reduced and converged to a stable range, the second temperature detection model is tested by using a test data set, after the accuracy of the test result is greater than or equal to a preset accuracy, knowledge learned by the second temperature detection model SRAnet-teacher is transmitted to the first temperature detection model SRAnet-student by means of distillation learning, so that the first temperature detection model SRAnet-student is trained by means of the training data set, and after the loss of the first temperature detection model SRAnet-student is reduced and converged to a stable range, and the accuracy of the test result is greater than or equal to the preset accuracy, the first temperature detection model SRAnet-student which is pre-trained is obtained, namely the light first temperature detection model is obtained, thereby further improving the temperature detection efficiency.
As a possible implementation manner, fig. 4 is a schematic structural diagram of a first temperature detection model according to an embodiment of the present invention.
As shown in fig. 4, the first temperature detection model includes a first convolution module 410, a first spatial circulation nerve module 420, a second spatial circulation nerve module 430, a third spatial circulation nerve module 440, a fourth spatial circulation nerve module 450, a first linking module 460, a first decoding module 470, and a second decoding module 480.
The method for identifying the temperature by inputting the window thermal image into a pre-trained first temperature detection model comprises the following steps: performing feature extraction processing on the window thermal image by using a first convolution module 410 to obtain a first feature image; the first space circulation nerve module 420, the second space circulation nerve module 430, the third space circulation nerve module 440 and the fourth space circulation nerve module 450 are used for respectively extracting features of the first feature map in different directions to obtain a first direction feature map, a second direction feature map, a third direction feature map and a fourth direction feature map; linking the first feature map, the first direction feature map, the second direction feature map, the third direction feature map and the fourth direction feature map through a first linking module 460 to obtain an encoding feature map; decoding the encoded feature map by a first decoding module 470 to obtain a first decoded intermediate feature map; the first decoded intermediate feature map is decoded by the second decoding module 480 to obtain an indoor temperature detection result.
Illustratively, fig. 5 is a schematic structural diagram of a first temperature detection model according to a specific embodiment of the present application. As shown in fig. 5, the window thermal image is input into a first lightweight temperature detection model, spatial information of the thermal image outside the window in four directions is obtained through a spatial cyclic neural module, the spatial information is linked with a characteristic result obtained by a first layer convolution module, a coding characteristic image containing spatial correlation characteristic information is output, and a decoding module decodes the coding characteristic image to obtain an indoor temperature detection result.
Fig. 6 is a schematic diagram of a first decoding module according to an embodiment of the present invention.
As shown in fig. 6, the first decoding module 470 includes a first multi-headed self-attention layer 610, a first normalization layer 620, a first feed-forward network layer 630, and a second normalization layer 640 connected in sequence, where decoding the encoded feature map by the first decoding module 470 may include: performing feature extraction on the coding feature map through the first multi-head self-attention layer 610 to obtain a first decoding feature map; normalizing the coding feature map and the first decoding feature map through a first normalization layer 620 to obtain a second decoding feature map; processing the second decoded feature map through the first feed forward network layer 630 to obtain a third decoded feature map; and normalizing the second decoding feature map and the third decoding feature map through a second normalization layer 640 to obtain a first decoding intermediate feature map.
Specifically, the first multi-head self-focusing layer 610 is configured to focus on information of different subspaces at different positions in the window thermal image map, through the first normalization layer 620, reduce difficulty of fitting and risk of over-fitting by losing a part of unimportant complex information, thereby accelerating convergence of the model, through the first feedforward network layer 630, enhance expression capability of image features by activating functions, and further through the second normalization layer 640, obtain a first decoding intermediate feature map.
Fig. 7 is a schematic diagram of a structure of a second decoding module according to an embodiment of the present invention.
As shown in fig. 7, the second decoding module 480 includes a second multi-headed self-attention layer 710, a third normalization layer 720, a second feed-forward network layer 730, and a fourth normalization layer 740 connected in sequence, wherein decoding the first decoded intermediate feature map by the second decoding module 480 includes: feature extraction is performed on the first decoding intermediate feature map through the second multi-head self-attention layer 710 to obtain a fourth decoding feature map; normalizing the first decoding intermediate feature map and the fourth decoding feature map through a third normalization layer 720 to obtain a fifth decoding feature map; processing the fifth decoding feature map through the second feedforward network layer 730 to obtain a sixth decoding feature map; and normalizing the fifth decoding feature map and the sixth decoding feature map through a fourth normalization layer 740 to obtain an indoor temperature detection result.
In this embodiment, the second decoding module 480 has the same structure as the first decoding module 470, the input of the second decoding module 480 is the output of the first decoding module 470, and the temperature prediction is performed on the window thermal image by performing decoding processing on the first decoding intermediate feature map, so as to obtain the indoor temperature detection result.
As an example, fig. 8 is a schematic structural view of a second temperature detection model according to an embodiment of the present invention. As shown in fig. 8, features of an outside-window image are initially extracted through a layer of convolution module, then spatial correlation information is extracted through four spatial cyclic neural modules in different directions respectively, meanwhile, in order to enable a network to pay attention to effective information, a distribution weight w is obtained through an attention module, the obtained spatial correlation information feature images in different directions are multiplied by the attention distribution weight, then, the feature images initially extracted through a first layer of convolution module are supplemented and linked to form an encoding feature image, each unit feature in the feature image contains spatial correlation features in the position, the upper direction, the lower direction, the left direction and the right direction of the image, different complex thermal image distribution situations can be fully associated in the encoding process, the encoding feature image is input into a decoding layer formed by a 6-layer decoding module decoder block for feature information decoding, and the decoding module is formed by a self-attention structure, so that an indoor temperature detection result is obtained.
It may be appreciated that the first temperature detection model includes two decoding modules, namely, the first decoding module 470 and the second decoding module 480, and the second temperature detection model includes six decoding modules, where the first decoding module 470 and the second decoding module 480 learn network parameters of any two decoding modules in the second temperature detection model through distillation to obtain network parameters of the first decoding module 470 and the second decoding module 480, the process of distillation learning is exemplarily shown in fig. 3, so as to reduce the burden of real-time prediction of the first temperature detection model, the first temperature detection model cuts the attention module in the spatial attention circulating neural network of the second temperature detection model to obtain a structure of the first temperature detection model, because the characteristic distribution condition of network concentration is obtained through distillation information during distillation, four characteristic maps with spatial correlation information are output through the spatial circulating neural network, and are linked with the characteristic maps output by the first layer convolution module, so as to output coded characteristic maps with spatial correlation information, and input the coded characteristic maps with spatial correlation information into the quantized coded characteristic maps, the decoded indoor temperature prediction layer is obtained, the first temperature detection model includes spatial correlation information, and the spatial correlation information is extracted from the spatial correlation information through the two temperature prediction layers through the spatial correlation modules, and the spatial correlation information is extracted from the spatial correlation information layer through the spatial correlation prediction module.
According to the infrared thermal image indoor temperature measurement method, a real-time thermal image and a real-time high-definition image outside a unit building are obtained; inputting the real-time high-definition image into a pre-trained target detection deep learning network to perform window positioning, and obtaining window position information; dividing the real-time thermal image according to the window position information to obtain a window thermal image; and inputting the window thermal image into a pre-trained first temperature detection model to perform temperature identification, and obtaining an indoor temperature detection result, so that the internal relation of the thermal image on spatial distribution and the internal relation of the characteristics of the thermal image are extracted, the indoor temperature is predicted by combining the characteristic distribution conditions, and the efficiency of real-time prediction of the indoor temperature is realized by further lightening the first temperature detection model through a distillation learning method.
To achieve the above embodiments, the present invention further proposes a computer readable storage medium, and fig. 9 is a schematic structural diagram of a computer readable storage medium according to an embodiment of the present invention.
As shown in fig. 9, the computer readable storage medium 900 has a computer program 901 stored thereon, and the computer program 901 implements the infrared thermal image indoor temperature measurement method according to the above embodiment of the present invention when executed by a processor.
In order to implement the above embodiment, the present invention further provides an infrared thermal image indoor temperature measurement system, and fig. 10 is a schematic structural diagram of the infrared thermal image indoor temperature measurement system according to an embodiment of the present invention.
As shown in fig. 10, the infrared thermal imaging indoor temperature measurement system 1000 may include: the binocular thermal imager 1010 and the 3D cradle head 1020,3D cradle head 1020 are used for carrying the binocular thermal imager 1010 so as to shoot images outside the unit building and obtain a real-time thermal image and a real-time high-definition image; the micro embedded control analysis device 1030 is configured to input a real-time high-definition image into a pre-trained target detection deep learning network to perform window positioning, obtain window position information, segment a real-time thermal image according to the window position information, obtain a window thermal image, and input the window thermal image into a pre-trained first temperature detection model to perform temperature identification, so as to obtain an indoor temperature detection result, where the first temperature detection model is obtained by distillation training of a second temperature detection model.
The infrared thermal image indoor temperature measurement system 1000 may acquire a real-time thermal image and a real-time high-definition image through the binocular thermal imager 1010 and the 3D cradle head 1020, cut the position of a window in the real-time thermal image through the micro embedded control analysis device 1030, input the window thermal image and the corresponding temperature information thereof to the automatic training server for training, distill and train the first temperature detection model through the second temperature detection model, thereby obtaining a pre-trained second temperature detection model, push the trained first temperature detection model to the micro embedded control analysis device 1030, input the cut window thermal image to the first target detection model distilled and learned in the micro embedded control analysis device 1030 when performing real-time indoor temperature prediction, thereby obtaining the indoor temperature corresponding to the window, and push the predicted indoor temperature, the corresponding real-time thermal image video and the real-time high-definition video to the high-definition network transmission and cloud platform system, so as to realize the real-time monitoring of the indoor temperature. As an example, an image of each frame in a video preset time may be obtained from a real-time thermal image video and a real-time high-definition video according to a preset interval time as a real-time thermal image and a real-time high-definition image.
As an example, fig. 11 is a workflow diagram of an automatic training server in accordance with one embodiment of the present invention. As shown in fig. 11, each region may package and push a training set including the window thermal image map and the window temperature information to the automatic training server, and perform training of the first temperature detection model, so as to monitor the indoor temperature of each region in real time.
Illustratively, the binocular thermal imager 1010 may be a grid-based image information and information YRH600D infrared binocular camera binocular system, the infrared resolution is 384×288, the visible resolution is 2560×1440, the binocular thermal imager is mounted on the 3D cradle 1020, and the miniature embedded control analysis device 1030 is configured, and may predict an indoor temperature by using a target detection network YOLOv5n to cut a real-time thermal image, push training data to an automatic training server and push a prediction result and a thermal image high-definition image or video to a high-definition network transmission radio and a cloud direct broadcast system, the automatic training server runs an automatic reinforcement training service, trains a second temperature detection model and a distillation training first temperature detection model, and is responsible for sending the updated first temperature detection model back to the miniature embedded control analysis device 1030. And the cloud live broadcast system displays the predicted indoor temperature condition in real time, and the high-definition network transmission radio station sends the packaged indoor temperature information of the user to the heat supply network balance server to serve as a basis for regulating and controlling heat supply of the user by the heat supply network.
It should be noted that the logic and/or steps represented in the flowcharts or otherwise described herein, for example, may be considered as a ordered listing of executable instructions for implementing logical functions, and may be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program is printed, as the program may be electronically captured, via, for instance, optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.
It is to be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, may be implemented using any one or combination of the following techniques, as is well known in the art: discrete logic circuits having logic gates for implementing logic functions on data signals, application specific integrated circuits having suitable combinational logic gates, programmable Gate Arrays (PGAs), field Programmable Gate Arrays (FPGAs), and the like.
In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings are merely for convenience in describing the present invention and simplifying the description, and do not indicate or imply that the device or element being referred to must have a specific orientation, be configured and operated in a specific orientation, and therefore should not be construed as limiting the present invention.
Furthermore, the terms "first," "second," and the like, as used in embodiments of the present invention, are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or as implying any particular number of features in the present embodiment. Thus, a feature of an embodiment of the invention that is defined by terms such as "first," "second," etc., may explicitly or implicitly indicate that at least one such feature is included in the embodiment. In the description of the present invention, the word "plurality" means at least two or more, for example, two, three, four, etc., unless explicitly defined otherwise in the embodiments.
In the present invention, unless explicitly stated or limited otherwise in the examples, the terms "mounted," "connected," and "fixed" as used in the examples should be interpreted broadly, e.g., the connection may be a fixed connection, may be a removable connection, or may be integral, and it may be understood that the connection may also be a mechanical connection, an electrical connection, etc.; of course, it may be directly connected, or indirectly connected through an intermediate medium, or may be in communication with each other, or in interaction with each other. The specific meaning of the above terms in the present invention can be understood by those of ordinary skill in the art according to specific embodiments.
In the present invention, unless expressly stated or limited otherwise, a first feature "up" or "down" a second feature may be the first and second features in direct contact, or the first and second features in indirect contact via an intervening medium. Moreover, a first feature being "above," "over" and "on" a second feature may be a first feature being directly above or obliquely above the second feature, or simply indicating that the first feature is level higher than the second feature. The first feature being "under", "below" and "beneath" the second feature may be the first feature being directly under or obliquely below the second feature, or simply indicating that the first feature is less level than the second feature.
While embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and not to be construed as limiting the invention, and that variations, modifications, alternatives and variations may be made to the above embodiments by one of ordinary skill in the art within the scope of the invention.
Claims (9)
1. An infrared thermal imaging indoor temperature measurement method, which is characterized by comprising the following steps:
acquiring a real-time thermal image and a real-time high-definition image outside a unit building;
inputting the real-time high-definition image into a pre-trained target detection deep learning network to perform window positioning, and obtaining window position information;
dividing the real-time thermal image according to the window position information to obtain a window thermal image;
inputting the window thermal image map into a pre-trained first temperature detection model for temperature identification to obtain an indoor temperature detection result, wherein the first temperature detection model is obtained by distillation training of a second temperature detection model, the first temperature detection model comprises a first convolution module, a first space circulation nerve module, a second space circulation nerve module, a third space circulation nerve module, a fourth space circulation nerve module, a first link module, a first decoding module and a second decoding module, and the window thermal image map is input into the pre-trained first temperature detection model for temperature identification, and the method comprises the following steps:
Performing feature extraction processing on the window thermal image graph through the first convolution module to obtain a first feature graph;
the first space circulation nerve module, the second space circulation nerve module, the third space circulation nerve module and the fourth space circulation nerve module are used for extracting features of the first feature map in different directions respectively to obtain a first direction feature map, a second direction feature map, a third direction feature map and a fourth direction feature map;
the first link module is used for carrying out link processing on the first feature map, the first direction feature map, the second direction feature map, the third direction feature map and the fourth direction feature map to obtain an encoding feature map;
decoding the coding feature map through the first decoding module to obtain a first decoding intermediate feature map;
and decoding the first decoding intermediate feature map through the second decoding module to obtain the indoor temperature detection result.
2. The method of claim 1, wherein the frame position information includes a set of coordinates of each frame, and wherein the dividing the real-time thermal image according to the frame position information includes:
And cutting the real-time thermal image according to the coordinate set of each window so as to obtain a window thermal image corresponding to each window of the unit building.
3. The infrared thermal imaging indoor thermometry method of claim 1 or 2, further comprising, prior to training the first temperature detection model:
acquiring a thermal image sample image, a high-definition sample image and temperature measurement sample data;
inputting the Gao Qingyang image into the target detection deep learning network to perform window positioning to obtain positioning coordinates;
cutting the thermal image sample image according to the positioning coordinates to obtain a preliminary window thermal image;
and screening the preliminary window thermal image according to the temperature measurement sample data to obtain training sample data.
4. The infrared thermography indoor temperature measurement method of claim 3, wherein after obtaining the training sample data, the method further comprises:
the training sample data are sent to an automatic training server, wherein the automatic training server trains the second temperature detection model according to the training sample data, and the trained knowledge of the second temperature detection model is taught to the first temperature detection model in a distillation learning mode so as to train the first temperature detection model;
And receiving a trained first temperature detection model fed back by the automatic training server.
5. The method for indoor temperature measurement by infrared thermal imaging according to claim 4, wherein the first temperature detection model is an SRAnet-student deep learning network model, and the second temperature detection model is an SRAnet-teacher deep learning network model.
6. The method of claim 1, wherein the first decoding module includes a first multi-headed self-attention layer, a first normalization layer, a first feed-forward network layer, and a second normalization layer connected in sequence, wherein decoding the encoded signature by the first decoding module includes:
performing feature extraction on the coding feature map through the first multi-head self-attention layer to obtain a first decoding feature map;
normalizing the coding feature map and the first decoding feature map through the first normalization layer to obtain a second decoding feature map;
processing the second decoding feature map through the first feedforward network layer to obtain a third decoding feature map;
and normalizing the second decoding feature map and the third decoding feature map through the second normalization layer to obtain the first decoding intermediate feature map.
7. The method of infrared thermal imaging indoor temperature measurement according to claim 6, wherein the second decoding module includes a second multi-headed self-focusing layer, a third normalization layer, a second feed-forward network layer, and a fourth normalization layer connected in sequence, wherein decoding the first decoded intermediate feature map by the second decoding module includes:
performing feature extraction on the first decoding intermediate feature map through the second multi-head self-attention layer to obtain a fourth decoding feature map;
normalizing the first decoding intermediate feature map and the fourth decoding feature map through the third normalization layer to obtain a fifth decoding feature map;
processing the fifth decoding feature map through the second feedforward network layer to obtain a sixth decoding feature map;
and carrying out normalization processing on the fifth decoding feature map and the sixth decoding feature map through the fourth normalization layer to obtain the indoor temperature detection result.
8. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the infrared thermographic indoor temperature measurement method according to any of claims 1-7.
9. An infrared thermal imaging indoor temperature measurement system, comprising:
the binocular thermal imaging system comprises a binocular thermal imaging system and a 3D (three-dimensional) cradle head, wherein the 3D cradle head is used for carrying the binocular thermal imaging system so as to shoot images outside a unit building and obtain a real-time thermal image and a real-time high-definition image;
the miniature embedded control analysis device is used for inputting the real-time high-definition image into a pre-trained target detection deep learning network to perform window positioning, obtaining window position information, dividing the real-time thermal image according to the window position information, obtaining a window thermal image, inputting the window thermal image into a pre-trained first temperature detection model to perform temperature identification, and obtaining an indoor temperature detection result, wherein the first temperature detection model is obtained by distillation training of a second temperature detection model, the first temperature detection model comprises a first convolution module, a first space circulation nerve module, a second space circulation nerve module, a third space circulation nerve module, a fourth space circulation nerve module, a first link module, a first decoding module and a second decoding module, and the window thermal image is input into the pre-trained first temperature detection model to perform temperature identification, and the miniature embedded control analysis device comprises:
Performing feature extraction processing on the window thermal image graph through the first convolution module to obtain a first feature graph;
the first space circulation nerve module, the second space circulation nerve module, the third space circulation nerve module and the fourth space circulation nerve module are used for extracting features of the first feature map in different directions respectively to obtain a first direction feature map, a second direction feature map, a third direction feature map and a fourth direction feature map;
the first link module is used for carrying out link processing on the first feature map, the first direction feature map, the second direction feature map, the third direction feature map and the fourth direction feature map to obtain an encoding feature map;
decoding the coding feature map through the first decoding module to obtain a first decoding intermediate feature map;
and decoding the first decoding intermediate feature map through the second decoding module to obtain the indoor temperature detection result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310148242.4A CN115876334B (en) | 2023-02-22 | 2023-02-22 | Infrared thermal image indoor temperature measurement method, system and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310148242.4A CN115876334B (en) | 2023-02-22 | 2023-02-22 | Infrared thermal image indoor temperature measurement method, system and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115876334A CN115876334A (en) | 2023-03-31 |
| CN115876334B true CN115876334B (en) | 2023-05-30 |
Family
ID=85761494
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310148242.4A Active CN115876334B (en) | 2023-02-22 | 2023-02-22 | Infrared thermal image indoor temperature measurement method, system and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115876334B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN214309201U (en) * | 2021-03-24 | 2021-09-28 | 瑞纳智能设备股份有限公司 | Room temperature collector with multiple temperature sensors |
| CN115118928A (en) * | 2022-06-14 | 2022-09-27 | 江苏国信协联能源有限公司 | Heat supply pipeline video intelligent vision perception alarm system |
| CN115661016A (en) * | 2022-12-08 | 2023-01-31 | 瑞纳智能设备股份有限公司 | Fault monitoring method and system of heat exchange station and embedded image diagnosis control platform |
Family Cites Families (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB0724165D0 (en) * | 2007-12-11 | 2008-01-23 | Irt Surveys Ltd | Quantification of energy loss from buildings |
| CN105701495B (en) * | 2016-01-05 | 2022-08-16 | 贵州大学 | Image texture feature extraction method |
| KR102045804B1 (en) * | 2018-03-13 | 2019-11-18 | 상명대학교 산학협력단 | How to judge the insulation index of a building by measuring the outside temperature |
| CN111561732B (en) * | 2020-05-18 | 2021-02-09 | 瑞纳智能设备股份有限公司 | Heat exchange station heat supply adjusting method and system based on artificial intelligence |
| CN112013966B (en) * | 2020-08-24 | 2021-07-06 | 电子科技大学 | Power equipment infrared image processing method based on measured temperature |
| CN112964370B (en) * | 2021-03-30 | 2022-03-29 | 清华大学 | Method for rapidly acquiring indoor air temperature from outdoor in batch through infrared thermal imaging |
| CN113267258B (en) * | 2021-05-18 | 2023-02-17 | 烟台艾睿光电科技有限公司 | Infrared temperature measurement method, device, equipment, intelligent inspection robot and storage medium |
| CN113449680B (en) * | 2021-07-15 | 2022-08-30 | 北京理工大学 | Knowledge distillation-based multimode small target detection method |
| CN113705788B (en) * | 2021-08-27 | 2023-09-22 | 齐鲁工业大学 | Infrared image temperature estimation method and system based on full convolution neural network |
| CN113888737A (en) * | 2021-09-16 | 2022-01-04 | 杭州英集动力科技有限公司 | Hot user room temperature measuring method and system based on temperature measurement image shot by intelligent equipment |
| CN114298973A (en) * | 2021-12-01 | 2022-04-08 | 哈尔滨工业大学 | Intelligent heat supply monitoring method based on infrared image segmentation |
| CN115077717A (en) * | 2022-06-14 | 2022-09-20 | 深圳弘洁凯电子科技有限公司 | Millimeter wave infrared temperature measurement method and device, electronic equipment and storage medium |
| CN115063418B (en) * | 2022-08-10 | 2022-11-01 | 北京航空航天大学 | Power battery temperature detection method based on image recognition |
| CN115410003A (en) * | 2022-08-31 | 2022-11-29 | 广东电网有限责任公司广州供电局 | Power distribution room temperature early warning method, system and medium based on deep learning and infrared detection |
| CN115541030B (en) * | 2022-09-21 | 2024-02-27 | 中冶南方工程技术有限公司 | Method and device for identifying blast furnace top charge level temperature distribution and storage medium |
| CN115359338A (en) * | 2022-10-20 | 2022-11-18 | 南京信息工程大学 | Sea surface temperature prediction method and system based on hybrid learning model |
-
2023
- 2023-02-22 CN CN202310148242.4A patent/CN115876334B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN214309201U (en) * | 2021-03-24 | 2021-09-28 | 瑞纳智能设备股份有限公司 | Room temperature collector with multiple temperature sensors |
| CN115118928A (en) * | 2022-06-14 | 2022-09-27 | 江苏国信协联能源有限公司 | Heat supply pipeline video intelligent vision perception alarm system |
| CN115661016A (en) * | 2022-12-08 | 2023-01-31 | 瑞纳智能设备股份有限公司 | Fault monitoring method and system of heat exchange station and embedded image diagnosis control platform |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115876334A (en) | 2023-03-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109978820B (en) | Unmanned aerial vehicle route acquisition method, system and equipment based on laser point cloud | |
| WO2021184254A1 (en) | Infrared thermal imaging temperature measurement method, electronic device, unmanned aerial vehicle and storage medium | |
| CN111751003B (en) | Thermal imager temperature correction system and method and thermal imager | |
| US11594060B2 (en) | Animal information management system and animal information management method | |
| JP2020531014A (en) | Plant health monitoring methods and equipment | |
| CN115314609B (en) | Automatic acquisition method and device for fire hole video of aluminum electrolysis cell | |
| CN108133189B (en) | Hospital waiting information display method | |
| CN107146257B (en) | Underwater camera calibration device capable of self-adapting to water quality | |
| CN115876334B (en) | Infrared thermal image indoor temperature measurement method, system and computer readable storage medium | |
| CN113554781A (en) | Remote inspection method and device for overhead transmission line and computer equipment | |
| WO2024199547A1 (en) | Passive human body detection method and apparatus, and device and medium | |
| CN115965579A (en) | Method and system for identifying and positioning three-dimensional defects in transformer substation inspection | |
| CN110889847A (en) | Nuclear radiation damage assessment system and method based on infrared imaging | |
| CN117241142A (en) | Dynamic correction method and device for pitch angle of pan-tilt camera, equipment and storage medium | |
| CN117075042A (en) | Operation safety detection method, system and storage medium for electric power operation site | |
| CN116224884A (en) | Poultry house environment and poultry temperature sensing control system and method based on AI vision | |
| CN116999026A (en) | Traditional Chinese medicine physique identification method and equipment combining edge calculation and infrared imaging | |
| CN110400333A (en) | Coach's formula binocular stereo vision device and High Precision Stereo visual pattern acquisition methods | |
| CN115205714A (en) | Fire point positioning method and device, electronic equipment and storage medium | |
| CN111582157B (en) | Human body identification method, device, equipment and computer readable storage medium | |
| CN112601021B (en) | Method and system for processing monitoring video of network camera | |
| WO2021253247A1 (en) | Inspection method and apparatus for movable platform, and movable platform and storage medium | |
| CN115147573A (en) | Three-dimensional attitude estimation method and system, and computer-readable storage medium | |
| CN111317462A (en) | Blood flow imaging method and device based on U-net neural network | |
| KR20160027482A (en) | Method and apparatus for measuring fish body of swimming fish species based on laser beam |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |