CN110633682A - Infrared image anomaly monitoring method, device and equipment based on double-light fusion - Google Patents
Infrared image anomaly monitoring method, device and equipment based on double-light fusion Download PDFInfo
- Publication number
- CN110633682A CN110633682A CN201910887106.0A CN201910887106A CN110633682A CN 110633682 A CN110633682 A CN 110633682A CN 201910887106 A CN201910887106 A CN 201910887106A CN 110633682 A CN110633682 A CN 110633682A
- Authority
- CN
- China
- Prior art keywords
- image
- infrared image
- infrared
- visible light
- gray
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 55
- 230000004927 fusion Effects 0.000 title claims abstract description 49
- 238000000034 method Methods 0.000 title claims abstract description 46
- 230000002159 abnormal effect Effects 0.000 claims abstract description 36
- 230000005856 abnormality Effects 0.000 claims abstract description 26
- 238000013528 artificial neural network Methods 0.000 claims abstract description 17
- 230000009977 dual effect Effects 0.000 claims description 20
- 238000012806 monitoring device Methods 0.000 claims description 14
- 230000003287 optical effect Effects 0.000 claims description 8
- 238000004590 computer program Methods 0.000 claims description 6
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 238000001514 detection method Methods 0.000 abstract description 6
- 230000008859 change Effects 0.000 description 12
- 238000010586 diagram Methods 0.000 description 4
- 230000005855 radiation Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 238000003331 infrared imaging Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000000295 complement effect Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 229910044991 metal oxide Inorganic materials 0.000 description 1
- 150000004706 metal oxides Chemical class 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J5/00—Radiation pyrometry, e.g. infrared or optical thermometry
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J5/00—Radiation pyrometry, e.g. infrared or optical thermometry
- G01J2005/0077—Imaging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10048—Infrared image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30232—Surveillance
Abstract
The application discloses an anomaly monitoring method, device, equipment and system of infrared images based on double-light fusion, which comprises the following steps: acquiring a visible light image and an infrared image which have consistent image display contents and are synchronously acquired; inputting the visible light image into a pre-trained neural network, and outputting a converted infrared image; respectively obtaining gray level difference values of the converted infrared image and the same area in the infrared image; judging whether the gray difference value is larger than a preset threshold value or not; if so, determining that the area corresponding to the gray difference value is abnormal. The converted infrared image is converted from the visible light image, the gray value of each area is irrelevant to the temperature, when the temperature of a certain area is abnormal, the gray value in the infrared image changes, the gray difference value of the same area in the two images is obtained, whether the abnormality occurs is determined according to the relation between the gray difference value and a preset threshold value, the abnormality detection of all the areas is realized, the infrared image is not limited to the area with the highest temperature, and the abnormal area can be detected more effectively.
Description
Technical Field
The application relates to the technical field of image detection, in particular to an infrared image anomaly monitoring method, device, equipment and system based on double-light fusion.
Background
The infrared imaging technology is based on the difference imaging of the infrared radiation of an object, and because different objects or different parts of the same object usually have different thermal radiation characteristics, such as temperature difference, emissivity and the like, after thermal infrared imaging is carried out, the objects in the infrared image are distinguished due to the difference of the thermal radiation.
Currently, when a target object in an infrared image is monitored according to an infrared image, a preset monitoring area (an area with the highest temperature in the infrared image) is mainly monitored. The thermal imager searches and marks the region with the highest temperature in the current infrared image, the region is set as a monitoring region, when the temperature of the region is monitored to be increased, the target object in the region is abnormal, the monitoring effect on the region outside the monitoring region cannot be achieved, and the abnormal conditions of other regions in the infrared image cannot be found in time.
Therefore, how to monitor all regions in the infrared image is a major concern to those skilled in the art.
Disclosure of Invention
The application aims to provide a method, a device, equipment and a system for monitoring the abnormity of an infrared image based on double-light fusion so as to monitor all areas in the infrared image.
In order to solve the above technical problem, the present application provides a method for monitoring an abnormality of an infrared image based on dual-light fusion, including:
acquiring a visible light image and an infrared image which have consistent image display contents and are synchronously acquired;
inputting the visible light image into a pre-trained neural network, and outputting a converted infrared image;
respectively acquiring gray level difference values of the converted infrared image and the same area in the infrared image;
judging whether the gray difference value is larger than a preset threshold value or not;
and if so, determining that the area corresponding to the gray difference value is abnormal.
Optionally, the acquiring the visible light image and the infrared image which have consistent image display contents and are synchronously acquired includes:
acquiring an initial visible light image and an initial infrared image which are synchronously acquired;
setting one image of the initial visible light image and the initial infrared image as a standard image and the other image as an image to be registered;
and registering the display content in the image to be registered by taking the image display content of the standard image as a standard, so that the image display content of the registered image to be registered is consistent with the standard image, and the visible light image and the infrared image are obtained.
Optionally, the registering the display content in the image to be registered includes:
and determining the characteristic points in the standard image by using an SIFT algorithm, and registering the display content in the image to be registered according to the characteristic points.
Optionally, after determining that the region corresponding to the gray scale difference value is abnormal, the method further includes:
and sending alarm information to a preset terminal.
Optionally, the alarm information is sound alarm information and/or text alarm information.
The application also provides an anomaly monitoring device based on the infrared image of two light fusions, includes:
the first acquisition module is used for acquiring a visible light image and an infrared image which are consistent in image display content and synchronously acquired;
the conversion module is used for inputting the visible light image into a pre-trained neural network and outputting a converted infrared image;
the second acquisition module is used for respectively acquiring gray level difference values of the converted infrared image and the same area in the infrared image;
the judging module is used for judging whether the gray difference value is larger than a preset threshold value or not;
and the determining module is used for determining that the area corresponding to the gray difference value is abnormal when the gray difference value is larger than the preset threshold value.
Optionally, the first obtaining module specifically includes:
the acquisition unit is used for acquiring an initial visible light image and an initial infrared image which are synchronously acquired;
the setting unit is used for setting either one of the initial visible light image and the initial infrared image as a standard image and the other one as an image to be registered;
and the registration unit is used for registering the display content in the image to be registered by taking the image display content of the standard image as a standard, so that the image display content of the registered image to be registered is consistent with the standard image, and the visible light image and the infrared image are obtained.
Optionally, the method further includes:
and the sending module is used for sending the alarm information to a preset terminal.
The application also provides an anomaly monitoring device based on the infrared image of two light fusions, includes:
a memory for storing a computer program;
and the processor is used for realizing the steps of any one of the above two-light fusion based infrared image abnormity monitoring methods when executing the computer program.
The application also provides an anomaly monitoring system based on the infrared image of two light fusion, includes:
the image acquisition equipment is used for acquiring a visible light image;
the thermal infrared imager is used for acquiring infrared images;
the anomaly monitoring equipment based on the infrared image of the double-light fusion is described above.
The application provides an anomaly monitoring method of an infrared image based on double-light fusion, which comprises the following steps: acquiring a visible light image and an infrared image which have consistent image display contents and are synchronously acquired; inputting the visible light image into a pre-trained neural network, and outputting a converted infrared image; respectively acquiring gray level difference values of the converted infrared image and the same area in the infrared image; judging whether the gray difference value is larger than a preset threshold value or not; and if so, determining that the area corresponding to the gray difference value is abnormal.
Therefore, the anomaly monitoring method in the application obtains the converted infrared image corresponding to the visible image by inputting the visible image which is consistent with the image display content of the infrared image and is synchronously acquired into the pre-trained neural network, namely the display content of the converted infrared image is consistent with that of the infrared image, because the converted infrared image is converted from the visible image, the gray value of each area in the converted infrared image is irrelevant to the temperature change, the gray value of each area cannot change, when the temperature of a certain area is abnormal, the gray value of the area in the infrared image can change, the gray difference value of the same area in the two images of the converted infrared image and the infrared image is obtained, and whether the area corresponding to the gray difference value is abnormal or not can be determined according to the relation between the gray difference value and the preset threshold value, the method and the device realize the abnormal detection of all areas in the infrared image, are not limited to monitoring only the area with the highest temperature in the infrared image, and can more effectively detect the area with the abnormality. In addition, the application also provides an abnormality monitoring device, equipment and system with the advantages.
Drawings
For a clearer explanation of the embodiments or technical solutions of the prior art of the present application, the drawings needed for the description of the embodiments or prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
Fig. 1 is a flowchart of an anomaly monitoring method for infrared images based on dual optical fusion according to an embodiment of the present application;
fig. 2 is a flowchart of another anomaly monitoring method based on dual-optical fusion infrared images according to an embodiment of the present application;
fig. 3 is a block diagram of an anomaly monitoring device based on a dual-optical fusion infrared image according to an embodiment of the present application;
fig. 4 is a block diagram of an anomaly monitoring device based on a dual-optical fusion infrared image according to an embodiment of the present application.
Detailed Description
In order that those skilled in the art will better understand the disclosure, the following detailed description will be given with reference to the accompanying drawings. It is to be understood that the embodiments described are only a few embodiments of the present application and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, but the present invention may be practiced in other ways than those specifically described and will be readily apparent to those of ordinary skill in the art without departing from the spirit of the present invention, and therefore the present invention is not limited to the specific embodiments disclosed below.
As described in the background section, currently, when monitoring a target object in an infrared image according to an infrared image, monitoring is mainly performed through a preset monitoring area. The thermal imager searches and marks the region with the highest temperature in the current infrared image, the region is set as a monitoring region, when the temperature of the region is monitored to be increased, the target object in the region is abnormal, the monitoring effect on the region outside the monitoring region cannot be achieved, and the abnormal conditions of other regions in the infrared image cannot be found in time.
In view of the above, the present application provides a method for monitoring an abnormality of an infrared image based on dual optical fusion, please refer to fig. 1, where fig. 1 is a flowchart of an embodiment of the method for monitoring an abnormality of an infrared image based on dual optical fusion, where the method includes:
step S101: and acquiring a visible light image and an infrared image which have consistent image display contents and are synchronously acquired.
It should be noted that the consistency of the image display contents of the visible light image and the infrared image means that the images displayed in the two images are the same, and the difference points are only that one image is the visible light image and the other image is the infrared image. The infrared image can display the temperature information of objects in different areas in the infrared image, and the visible light image does not contain the temperature information.
Step S102: and inputting the visible light image into a pre-trained neural network, and outputting the converted infrared image.
Specifically, the pre-trained neural network is obtained by training a neural network in a deep learning manner by using a visible light image and an infrared image which are prepared in advance, have consistent image display contents and are synchronously acquired as training sample images, wherein the training sample images comprise images acquired by the user and images collected from the network. And comparing the converted infrared image with the corresponding infrared image, and gradually reducing the error between the converted infrared image and the infrared image by training parameters of the neural network. The specific training process of neural networks is well known to those skilled in the art and will not be described in detail herein.
Step S103: and respectively acquiring gray level difference values of the converted infrared image and the same area in the infrared image.
It should be noted that the gray-scale difference is the difference between the gray-scale values of the converted infrared image and the same area in the infrared image, and is a positive number greater than or equal to zero.
It can be understood that temperature information of different areas is displayed in the infrared image, the higher the temperature of a certain area is, the brighter the area display is, and the gray values of different temperature areas are different; the converted infrared image is converted from the visible light image, the converted infrared image does not contain temperature information because the visible light image does not contain temperature information, and the gray value in the converted infrared image does not change along with the change of the temperature, so that the difference of the temperature can be represented according to the gray difference value of the converted infrared image and the same area in the infrared image.
Further, since the gray scale difference represents the temperature difference, the present embodiment is not limited to effectively detect the region with increased temperature, and can also be monitored when the temperature value of a certain region is decreased.
Specifically, different areas in the infrared image represent different monitoring ranges, and all areas displayed in the infrared image are monitored by respectively obtaining the gray level difference value of the converted infrared image and the same area in the infrared image.
Step S104: and judging whether the gray difference value is larger than a preset threshold value.
It should be noted that, in the present embodiment, the preset threshold is not specifically limited, and as the case may be, for example, the preset threshold may be 10, or 20, etc.
Step S105: and if so, determining that the area corresponding to the gray difference value is abnormal.
Specifically, when the grayscale difference is greater than the preset threshold, the temperature increase or decrease amplitude of the region corresponding to the grayscale difference exceeds the normal amplitude, and an abnormality occurs.
It can be understood that, when the gray scale difference value is not greater than the preset threshold, it indicates that the temperature change of the region corresponding to the gray scale difference value is normal.
In the anomaly monitoring method in this embodiment, the converted infrared image corresponding to the visible light image is obtained by inputting the visible light image, which is consistent with the image display content of the infrared image and is synchronously acquired, into the pre-trained neural network, i.e., the display content of the converted infrared image is also consistent with that of the infrared image, because the converted infrared image is converted from the visible light image, the gray value of each region in the converted infrared image is irrelevant to the temperature change, the gray value of each region does not change, when the temperature of a certain region is abnormal, the gray value of the region in the infrared image changes, the gray difference value of the same region in the two images of the converted infrared image and the infrared image is obtained, and whether the region corresponding to the gray difference value is abnormal or not can be determined according to the relationship between the gray difference value and the preset threshold value, the method and the device have the advantages that the abnormity detection of all areas in the infrared image is realized, the method and the device are not limited to monitoring only the area with the highest temperature in the infrared image, and the abnormal temperature cannot be detected when the temperature in the abnormal state is lower than the highest temperature value in the infrared image, so that the abnormal area can be detected more effectively.
Referring to fig. 2, fig. 2 is a flowchart of another anomaly monitoring method based on dual-light fusion infrared images according to an embodiment of the present application, where the method includes:
step S201: acquiring an initial visible light image and an initial infrared image which are synchronously acquired;
specifically, in this embodiment, the display contents in the initial visible light image and the initial infrared image are not consistent.
Step S202: and setting one of the initial visible light image and the initial infrared image as a standard image and the other one as an image to be registered.
Specifically, when the initial infrared image is a standard image, the initial visible light image is an image to be registered; and when the initial visible light image is the standard image, the initial infrared image is the image to be registered.
Step S203: and registering the display content in the image to be registered by taking the image display content of the standard image as a standard, so that the image display content of the registered image to be registered is consistent with the standard image, and the visible light image and the infrared image are obtained.
Step S204: and inputting the visible light image into a pre-trained neural network, and outputting the converted infrared image.
Step S205: and respectively acquiring gray level difference values of the converted infrared image and the same area in the infrared image.
Step S206: and judging whether the gray difference value is larger than a preset threshold value.
Step S207: and if so, determining that the area corresponding to the gray difference value is abnormal.
Step S204 to step S207 are not described in detail herein, and refer to the above embodiments specifically.
Optionally, in an embodiment of the present application, the registering display content in the image to be registered includes: and determining Feature points in the standard image by using a Scale Invariant Feature Transform (SIFT) algorithm, and registering display contents in the image to be registered according to the Feature points. However, the present application is not limited to this, and in other embodiments of the present application, registration may also be performed by manually marking feature points, and a position that is obvious in both the initial visible light image and the initial infrared image, such as a position of a corner of an object, is selected for registration. The specific registration process is well known to those skilled in the art and will not be described in detail herein.
It can be understood that parameters, positions, lenses and the like of the visible light sensor and the infrared sensor in the image acquisition device are fixed, the parameters are adjusted, and after one-time manual registration, the images acquired later can be used for registration of the initial visible light image and the initial infrared image by using the set of parameters.
On the basis of any of the foregoing embodiments, in an embodiment of the present application, after determining that an area corresponding to the gray scale difference value is abnormal, the method further includes:
and sending alarm information to a preset terminal so that the worker can know the occurrence of abnormal conditions.
It should be noted that, in this embodiment, the preset terminal is not specifically limited, and may be set by itself. For example, the default terminal may be a computer, a mobile phone, an iPad, or the like.
It should be noted that, in this embodiment, the alarm information is not specifically limited, and may be set by itself. For example, the alarm information is sound alarm information and/or text alarm information.
The anomaly monitoring device based on the infrared image with dual light fusion provided by the embodiment of the application is introduced below, and the anomaly monitoring device based on the infrared image with dual light fusion described below and the anomaly monitoring method based on the infrared image with dual light fusion described above can be referred to correspondingly.
Fig. 3 is a block diagram of an anomaly monitoring apparatus based on a dual-optical fusion infrared image according to an embodiment of the present application, where referring to fig. 3, the anomaly monitoring apparatus based on a dual-optical fusion infrared image may include:
the first acquisition module 100 is configured to acquire a visible light image and an infrared image which are consistent in image display content and are acquired synchronously;
the conversion module 200 is configured to input the visible light image into a pre-trained neural network, and output a converted infrared image;
a second obtaining module 300, configured to obtain gray level difference values of the converted infrared image and the same region in the infrared image respectively;
a judging module 400, configured to judge whether the grayscale difference is greater than a preset threshold;
a determining module 500, configured to determine that an area corresponding to the grayscale difference is abnormal when the grayscale difference is greater than the preset threshold.
The abnormality monitoring device based on the infrared image with dual light fusion of the present embodiment is used for implementing the foregoing abnormality monitoring method based on the infrared image with dual light fusion, and therefore, a specific implementation manner of the abnormality monitoring device based on the infrared image with dual light fusion of the present embodiment can be seen in the foregoing example portions of the abnormality monitoring method based on the infrared image with dual light fusion, for example, the first obtaining module 100, the transforming module 200, the second obtaining module 300, the determining module 400, and the determining module 500 are respectively used for implementing steps S101, S102, S103, S104, and S105 in the foregoing abnormality monitoring method based on the infrared image with dual light fusion, so that the specific implementation manner thereof may refer to descriptions of corresponding respective partial examples, and details thereof are not repeated.
The anomaly monitoring device in this embodiment obtains a converted infrared image corresponding to the visible image by inputting the visible image, which is consistent with the image display content of the infrared image and is synchronously acquired, into a pre-trained neural network, that is, the display content of the converted infrared image is also consistent with that of the infrared image, because the converted infrared image is converted from the visible image, the gray value of each region in the converted infrared image is irrelevant to the temperature change, the gray value of each region does not change, and when the temperature of a certain region is abnormal, the gray value of the region in the infrared image changes, the gray value difference of the same region in the two images of the converted infrared image and the infrared image is obtained, and whether the region corresponding to the gray value difference is abnormal or not can be determined according to the relationship between the gray value difference and a preset threshold value, the method and the device have the advantages that the abnormity detection of all areas in the infrared image is realized, the method and the device are not limited to monitoring only the area with the highest temperature in the infrared image, and the abnormal temperature cannot be detected when the temperature in the abnormal state is lower than the highest temperature value in the infrared image, so that the abnormal area can be detected more effectively.
In an embodiment of the present application, the first obtaining module 100 specifically includes:
the acquisition unit is used for acquiring an initial visible light image and an initial infrared image which are synchronously acquired;
the setting unit is used for setting either one of the initial visible light image and the initial infrared image as a standard image and the other one as an image to be registered;
and the registration unit is used for registering the display content in the image to be registered by taking the image display content of the standard image as a standard, so that the image display content of the registered image to be registered is consistent with the standard image, and the visible light image and the infrared image are obtained.
On the basis of any one of the above embodiments, in an embodiment of the present application, the abnormality monitoring apparatus based on the infrared image with dual optical fusion further includes:
and the sending module is used for sending the alarm information to a preset terminal.
The anomaly monitoring device based on the infrared image with dual light fusion provided by the embodiment of the application is introduced below, and the anomaly monitoring device based on the infrared image with dual light fusion described below and the anomaly monitoring method based on the infrared image with dual light fusion described above can be referred to correspondingly.
Fig. 4 is a block diagram of an anomaly monitoring apparatus based on a dual-optical fusion infrared image according to an embodiment of the present application, where the apparatus includes:
a memory 11 for storing a computer program;
a processor 12, configured to implement the steps of any one of the above-mentioned methods for monitoring an abnormality based on a dual optical fusion infrared image when executing the computer program.
The anomaly monitoring device in this embodiment obtains a converted infrared image corresponding to the visible image by inputting the visible image, which is consistent with the image display content of the infrared image and is synchronously acquired, into a pre-trained neural network, that is, the display content of the converted infrared image is also consistent with that of the infrared image, because the converted infrared image is converted from the visible image, the gray value of each region in the converted infrared image is irrelevant to the temperature change, the gray value of each region does not change, and when the temperature of a certain region is abnormal, the gray value of the region in the infrared image changes, the gray value difference of the same region in the two images of the converted infrared image and the infrared image is obtained, and whether the region corresponding to the gray value difference is abnormal or not can be determined according to the relationship between the gray value difference and a preset threshold value, the method and the device have the advantages that the abnormity detection of all areas in the infrared image is realized, the method and the device are not limited to monitoring only the area with the highest temperature in the infrared image, and the abnormal temperature cannot be detected when the temperature in the abnormal state is lower than the highest temperature value in the infrared image, so that the abnormal area can be detected more effectively.
The application also provides an anomaly monitoring system based on the infrared image of two light fusion, and the system includes:
the image acquisition equipment is used for acquiring a visible light image;
the thermal infrared imager is used for acquiring infrared images;
the anomaly monitoring equipment based on the infrared image of the double-light fusion.
It should be noted that the type of the image capturing apparatus in the present embodiment is not particularly limited, and may be determined as appropriate. For example, the image capturing Device may be a CCD (Charge-coupled Device) image capturing Device, or a CMOS (Complementary Metal Oxide Semiconductor) image capturing Device.
The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.
The method, the device, the equipment and the system for monitoring the abnormality of the infrared image based on the double-light fusion are described in detail above. The principles and embodiments of the present application are explained herein using specific examples, which are provided only to help understand the method and the core idea of the present application. It should be noted that, for those skilled in the art, it is possible to make several improvements and modifications to the present application without departing from the principle of the present application, and such improvements and modifications also fall within the scope of the claims of the present application.
Claims (10)
1. A method for monitoring the abnormity of an infrared image based on double-light fusion is characterized by comprising the following steps:
acquiring a visible light image and an infrared image which have consistent image display contents and are synchronously acquired;
inputting the visible light image into a pre-trained neural network, and outputting a converted infrared image;
respectively acquiring gray level difference values of the converted infrared image and the same area in the infrared image;
judging whether the gray difference value is larger than a preset threshold value or not;
and if so, determining that the area corresponding to the gray difference value is abnormal.
2. The method for monitoring abnormality of infrared image based on dual optical fusion as claimed in claim 1, wherein said acquiring the visible light image and the infrared image whose image display contents are consistent and synchronously collected includes:
acquiring an initial visible light image and an initial infrared image which are synchronously acquired;
setting one image of the initial visible light image and the initial infrared image as a standard image and the other image as an image to be registered;
and registering the display content in the image to be registered by taking the image display content of the standard image as a standard, so that the image display content of the registered image to be registered is consistent with the standard image, and the visible light image and the infrared image are obtained.
3. The method for abnormality monitoring based on the infrared image with dual optical fusion of claim 2, wherein the registering the display content in the image to be registered includes:
and determining the characteristic points in the standard image by using an SIFT algorithm, and registering the display content in the image to be registered according to the characteristic points.
4. A method as claimed in any one of claims 1 to 3, wherein after determining that there is an abnormality in the region corresponding to the gray scale difference value, the method further comprises:
and sending alarm information to a preset terminal.
5. The abnormality monitoring method based on the infrared image with double light fusion of claim 4, characterized in that the alarm information is sound alarm information and/or text alarm information.
6. An abnormality monitoring device for an infrared image based on double light fusion, comprising:
the first acquisition module is used for acquiring a visible light image and an infrared image which are consistent in image display content and synchronously acquired;
the conversion module is used for inputting the visible light image into a pre-trained neural network and outputting a converted infrared image;
the second acquisition module is used for respectively acquiring gray level difference values of the converted infrared image and the same area in the infrared image;
the judging module is used for judging whether the gray difference value is larger than a preset threshold value or not;
and the determining module is used for determining that the area corresponding to the gray difference value is abnormal when the gray difference value is larger than the preset threshold value.
7. The apparatus for monitoring abnormality based on infrared image with dual optical fusion according to claim 6, wherein the first acquiring module specifically includes:
the acquisition unit is used for acquiring an initial visible light image and an initial infrared image which are synchronously acquired;
the setting unit is used for setting either one of the initial visible light image and the initial infrared image as a standard image and the other one as an image to be registered;
and the registration unit is used for registering the display content in the image to be registered by taking the image display content of the standard image as a standard, so that the image display content of the registered image to be registered is consistent with the standard image, and the visible light image and the infrared image are obtained.
8. A dual-optical fusion-based infrared image abnormality monitoring apparatus according to claim 6 or 7, further comprising:
and the sending module is used for sending the alarm information to a preset terminal.
9. An abnormality monitoring apparatus for infrared images based on two-light fusion, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the abnormality monitoring method based on the infrared image with dual light fusion according to any one of claims 1 to 5 when the computer program is executed.
10. An anomaly monitoring system based on infrared images of double light fusion is characterized by comprising:
the image acquisition equipment is used for acquiring a visible light image;
the thermal infrared imager is used for acquiring infrared images;
the abnormality monitoring apparatus based on the infrared image of the double light fusion of claim 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910887106.0A CN110633682B (en) | 2019-09-19 | 2019-09-19 | Infrared image anomaly monitoring method, device and equipment based on double-light fusion |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910887106.0A CN110633682B (en) | 2019-09-19 | 2019-09-19 | Infrared image anomaly monitoring method, device and equipment based on double-light fusion |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110633682A true CN110633682A (en) | 2019-12-31 |
| CN110633682B CN110633682B (en) | 2022-07-12 |
Family
ID=68971697
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910887106.0A Active CN110633682B (en) | 2019-09-19 | 2019-09-19 | Infrared image anomaly monitoring method, device and equipment based on double-light fusion |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110633682B (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111307289A (en) * | 2020-02-18 | 2020-06-19 | 浙江大华技术股份有限公司 | Body temperature abnormality identification method and device |
| CN111738132A (en) * | 2020-06-17 | 2020-10-02 | 银河水滴科技（北京）有限公司 | Method and device for measuring human body temperature, electronic equipment and readable storage medium |
| CN112528895A (en) * | 2020-12-17 | 2021-03-19 | 恒银金融科技股份有限公司 | Computing method for alteration detection of financial bill image ticket number |
| CN112577603A (en) * | 2020-10-09 | 2021-03-30 | 国网浙江宁波市奉化区供电有限公司 | Switch cabinet real-time monitoring method and system based on cable connector and ambient temperature thereof |
| CN112766481A (en) * | 2020-03-13 | 2021-05-07 | 腾讯科技（深圳）有限公司 | Neural network model training method and device and image detection method |
| CN113034915A (en) * | 2021-03-29 | 2021-06-25 | 北京卓视智通科技有限责任公司 | Double-spectrum traffic incident detection method and device |
| CN113095189A (en) * | 2021-04-01 | 2021-07-09 | 武汉一维路科技有限公司 | Machine vision and data analysis-based intelligent monitoring and management system for operating environment safety of power system transformer substation |
| CN113298744A (en) * | 2021-06-07 | 2021-08-24 | 长春理工大学 | End-to-end infrared and visible light image fusion method |
| CN113674212A (en) * | 2021-07-26 | 2021-11-19 | 宁波帅特龙集团有限公司 | Handle assembly detection method and device |
| CN114485947A (en) * | 2021-12-30 | 2022-05-13 | 北京联合永道软件股份有限公司 | Security monitoring method for outdoor electric equipment |
| CN114783141A (en) * | 2022-04-24 | 2022-07-22 | 王江湖 | Fire safety system |
| CN117238143A (en) * | 2023-09-15 | 2023-12-15 | 北京卓视智通科技有限责任公司 | Traffic data fusion method, system and device based on radar double-spectrum camera |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2011162417A1 (en) * | 2010-06-24 | 2011-12-29 | 有限会社西原電子 | Defective electrode detecting device |
| CN102661799A (en) * | 2012-05-16 | 2012-09-12 | 广东电网公司珠海供电局 | Fault positioning method and system |
| CN103399248A (en) * | 2013-07-18 | 2013-11-20 | 江苏物联网研究发展中心 | PCB (printed circuit board) defect mass detection system and method |
| US20140192184A1 (en) * | 2011-06-09 | 2014-07-10 | Guangzhou Sat Infrared Technology Co., Ltd. | Forest fire early-warning system and method based on infrared thermal imaging technology |
| CN104021548A (en) * | 2014-05-16 | 2014-09-03 | 中国科学院西安光学精密机械研究所 | Method for acquiring 4D scene information |
| US20150304612A1 (en) * | 2014-04-18 | 2015-10-22 | Flir Systems, Inc. | Multi-sensor monitoring systems and methods |
| CN107025648A (en) * | 2017-03-20 | 2017-08-08 | 中国人民解放军空军工程大学 | A kind of board failure infrared image automatic testing method |
| CN107253485A (en) * | 2017-05-16 | 2017-10-17 | 北京交通大学 | Foreign matter invades detection method and foreign matter intrusion detection means |
| US20180227509A1 (en) * | 2015-08-05 | 2018-08-09 | Wuhan Guide Infrared Co., Ltd. | Visible light image and infrared image fusion processing system and fusion method |
| CN109242815A (en) * | 2018-09-28 | 2019-01-18 | 合肥英睿系统技术有限公司 | A kind of infrared light image and visible light image fusion method and system |
| CN208443773U (en) * | 2018-07-11 | 2019-01-29 | 云南电网有限责任公司电力科学研究院 | A kind of unmanned plane line walking defect detecting device based on heat radiation |
| CN109472837A (en) * | 2018-10-24 | 2019-03-15 | 西安电子科技大学 | The photoelectric image conversion method of confrontation network is generated based on condition |
| CN109613401A (en) * | 2019-01-31 | 2019-04-12 | 福州大学 | A kind of cable arc fault early warning system based on the infrared thermovision of intelligence low cost |
| EP3534326A1 (en) * | 2016-10-31 | 2019-09-04 | Guangzhou SAT Infrared Technology Co., Ltd. | Method and apparatus for merging infrared image and visible light image |
-
2019
- 2019-09-19 CN CN201910887106.0A patent/CN110633682B/en active Active
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2011162417A1 (en) * | 2010-06-24 | 2011-12-29 | 有限会社西原電子 | Defective electrode detecting device |
| US20140192184A1 (en) * | 2011-06-09 | 2014-07-10 | Guangzhou Sat Infrared Technology Co., Ltd. | Forest fire early-warning system and method based on infrared thermal imaging technology |
| CN102661799A (en) * | 2012-05-16 | 2012-09-12 | 广东电网公司珠海供电局 | Fault positioning method and system |
| CN103399248A (en) * | 2013-07-18 | 2013-11-20 | 江苏物联网研究发展中心 | PCB (printed circuit board) defect mass detection system and method |
| US20150304612A1 (en) * | 2014-04-18 | 2015-10-22 | Flir Systems, Inc. | Multi-sensor monitoring systems and methods |
| CN104021548A (en) * | 2014-05-16 | 2014-09-03 | 中国科学院西安光学精密机械研究所 | Method for acquiring 4D scene information |
| US20180227509A1 (en) * | 2015-08-05 | 2018-08-09 | Wuhan Guide Infrared Co., Ltd. | Visible light image and infrared image fusion processing system and fusion method |
| EP3534326A1 (en) * | 2016-10-31 | 2019-09-04 | Guangzhou SAT Infrared Technology Co., Ltd. | Method and apparatus for merging infrared image and visible light image |
| CN107025648A (en) * | 2017-03-20 | 2017-08-08 | 中国人民解放军空军工程大学 | A kind of board failure infrared image automatic testing method |
| CN107253485A (en) * | 2017-05-16 | 2017-10-17 | 北京交通大学 | Foreign matter invades detection method and foreign matter intrusion detection means |
| CN208443773U (en) * | 2018-07-11 | 2019-01-29 | 云南电网有限责任公司电力科学研究院 | A kind of unmanned plane line walking defect detecting device based on heat radiation |
| CN109242815A (en) * | 2018-09-28 | 2019-01-18 | 合肥英睿系统技术有限公司 | A kind of infrared light image and visible light image fusion method and system |
| CN109472837A (en) * | 2018-10-24 | 2019-03-15 | 西安电子科技大学 | The photoelectric image conversion method of confrontation network is generated based on condition |
| CN109613401A (en) * | 2019-01-31 | 2019-04-12 | 福州大学 | A kind of cable arc fault early warning system based on the infrared thermovision of intelligence low cost |
Non-Patent Citations (5)
| Title |
|---|
| TIEZHU QIAO等: "Integrative binocular vision detection method based on infrared and visible light fusion for conveyor belts longitudinal tear", 《MEASUREMENT》 * |
| TIEZHU QIAO等: "Integrative binocular vision detection method based on infrared and visible light fusion for conveyor belts longitudinal tear", 《MEASUREMENT》, vol. 110, 29 June 2017 (2017-06-29), pages 192 - 201 * |
| 乔铁柱: "输送带纵向撕裂可见光与红外双目视觉在线检测系统研究", 《中国优秀博硕士学位论文全文数据库(博士)信息科技辑》 * |
| 乔铁柱: "输送带纵向撕裂可见光与红外双目视觉在线检测系统研究", 《中国优秀博硕士学位论文全文数据库(博士)信息科技辑》, no. 06, 15 June 2016 (2016-06-15), pages 138 - 14 * |
| 刘鹏: "基于双目成像技术的电力设备监测系统设计", 《中国优秀博硕士学位论文全文数据库(硕士)工程科技Ⅱ辑》, no. 2019, 15 June 2019 (2019-06-15), pages 042 - 470 * |
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111307289A (en) * | 2020-02-18 | 2020-06-19 | 浙江大华技术股份有限公司 | Body temperature abnormality identification method and device |
| CN111307289B (en) * | 2020-02-18 | 2021-08-27 | 浙江大华技术股份有限公司 | Body temperature abnormality identification method and device |
| CN112766481A (en) * | 2020-03-13 | 2021-05-07 | 腾讯科技（深圳）有限公司 | Neural network model training method and device and image detection method |
| CN112766481B (en) * | 2020-03-13 | 2023-11-24 | 腾讯科技（深圳）有限公司 | Training method and device for neural network model and image detection method |
| CN111738132A (en) * | 2020-06-17 | 2020-10-02 | 银河水滴科技（北京）有限公司 | Method and device for measuring human body temperature, electronic equipment and readable storage medium |
| CN111738132B (en) * | 2020-06-17 | 2024-03-05 | 银河水滴科技（北京）有限公司 | Method and device for measuring human body temperature, electronic equipment and readable storage medium |
| CN112577603A (en) * | 2020-10-09 | 2021-03-30 | 国网浙江宁波市奉化区供电有限公司 | Switch cabinet real-time monitoring method and system based on cable connector and ambient temperature thereof |
| CN112528895A (en) * | 2020-12-17 | 2021-03-19 | 恒银金融科技股份有限公司 | Computing method for alteration detection of financial bill image ticket number |
| CN112528895B (en) * | 2020-12-17 | 2023-09-12 | 恒银金融科技股份有限公司 | Calculation method for detecting change of financial bill image and bill number |
| CN113034915A (en) * | 2021-03-29 | 2021-06-25 | 北京卓视智通科技有限责任公司 | Double-spectrum traffic incident detection method and device |
| CN113095189A (en) * | 2021-04-01 | 2021-07-09 | 武汉一维路科技有限公司 | Machine vision and data analysis-based intelligent monitoring and management system for operating environment safety of power system transformer substation |
| CN113095189B (en) * | 2021-04-01 | 2022-01-14 | 广东履安实业有限公司 | Machine vision and data analysis-based intelligent monitoring and management system for operating environment safety of power system transformer substation |
| CN113298744A (en) * | 2021-06-07 | 2021-08-24 | 长春理工大学 | End-to-end infrared and visible light image fusion method |
| CN113674212A (en) * | 2021-07-26 | 2021-11-19 | 宁波帅特龙集团有限公司 | Handle assembly detection method and device |
| CN114485947A (en) * | 2021-12-30 | 2022-05-13 | 北京联合永道软件股份有限公司 | Security monitoring method for outdoor electric equipment |
| CN114485947B (en) * | 2021-12-30 | 2024-04-02 | 北京联合永道软件股份有限公司 | Security monitoring method for outdoor electric equipment |
| CN114783141A (en) * | 2022-04-24 | 2022-07-22 | 王江湖 | Fire safety system |
| CN117238143A (en) * | 2023-09-15 | 2023-12-15 | 北京卓视智通科技有限责任公司 | Traffic data fusion method, system and device based on radar double-spectrum camera |
| CN117238143B (en) * | 2023-09-15 | 2024-03-22 | 北京卓视智通科技有限责任公司 | Traffic data fusion method, system and device based on radar double-spectrum camera |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110633682B (en) | 2022-07-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110633682B (en) | Infrared image anomaly monitoring method, device and equipment based on double-light fusion | |
| CN107621932B (en) | Local amplification method and device for display image | |
| JP2012089929A (en) | Object detection apparatus, object detection method, monitoring camera system and program | |
| US20150071548A1 (en) | Image processing system, server device, image pickup device and image evaluation method | |
| CN107305695B (en) | Automatic image dead pixel correction device and method | |
| CN109698906B (en) | Image-based jitter processing method and device and video monitoring system | |
| JP5155110B2 (en) | Monitoring device | |
| JP7024713B2 (en) | Image processing device and image processing method | |
| CN111814776A (en) | Image processing method, device, server and storage medium | |
| JP5264457B2 (en) | Object detection device | |
| KR101428913B1 (en) | Surveillance system for overheating state utilizing micro-size image array sensor | |
| WO2019144262A1 (en) | Smudge detection method and apparatus and mobile electronic device | |
| JP2018063675A (en) | Image processor and control method | |
| CN111145234A (en) | Fire smoke detection method based on binocular vision | |
| CN113108919B (en) | Human body temperature detection method, device and storage medium | |
| JP7417426B2 (en) | Anomaly detection device and anomaly detection method | |
| CN110213457B (en) | Image transmission method and device | |
| JP5045449B2 (en) | Environment-adaptive intruder detection device by image processing | |
| US11272172B2 (en) | Image processing apparatus, failure detection method performed by image processing apparatus, and non-transitory recording medium | |
| JP5167614B2 (en) | Distance image generating apparatus, distance image generating method and program | |
| US10306138B2 (en) | Image output method and image capturing device | |
| JP2022158339A (en) | Detection device, detection method, and program | |
| CN110853127A (en) | Image processing method, device and equipment | |
| JP2020085829A (en) | Vibration detection method | |
| CN113706429B (en) | Image processing method, device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |