CN112365669A - Dual-band far infrared fusion-superposition imaging and fire early warning method and system - Google Patents
Dual-band far infrared fusion-superposition imaging and fire early warning method and system Download PDFInfo
- Publication number
- CN112365669A CN112365669A CN202011077708.9A CN202011077708A CN112365669A CN 112365669 A CN112365669 A CN 112365669A CN 202011077708 A CN202011077708 A CN 202011077708A CN 112365669 A CN112365669 A CN 112365669A
- Authority
- CN
- China
- Prior art keywords
- image
- video
- far infrared
- alarm
- fusion
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 26
- 238000003384 imaging method Methods 0.000 title claims abstract description 25
- 230000004927 fusion Effects 0.000 claims abstract description 42
- 238000012544 monitoring process Methods 0.000 claims abstract description 29
- 238000001514 detection method Methods 0.000 claims abstract description 28
- 238000009529 body temperature measurement Methods 0.000 claims abstract description 23
- 238000004458 analytical method Methods 0.000 claims abstract description 10
- 238000001931 thermography Methods 0.000 claims description 17
- 230000009466 transformation Effects 0.000 claims description 15
- 238000012545 processing Methods 0.000 claims description 14
- 238000007499 fusion processing Methods 0.000 claims description 10
- 230000009977 dual effect Effects 0.000 claims description 6
- 238000001914 filtration Methods 0.000 claims description 4
- 230000008569 process Effects 0.000 claims description 4
- 230000015572 biosynthetic process Effects 0.000 claims description 3
- 230000002708 enhancing effect Effects 0.000 claims description 3
- 238000005457 optimization Methods 0.000 claims description 3
- 238000007781 pre-processing Methods 0.000 claims description 3
- 238000012216 screening Methods 0.000 claims description 3
- 238000006243 chemical reaction Methods 0.000 claims description 2
- 238000002485 combustion reaction Methods 0.000 abstract description 7
- 238000004880 explosion Methods 0.000 abstract description 7
- 230000002269 spontaneous effect Effects 0.000 abstract description 6
- 230000005856 abnormality Effects 0.000 abstract description 2
- 230000006870 function Effects 0.000 description 10
- 238000005286 illumination Methods 0.000 description 7
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 230000002265 prevention Effects 0.000 description 3
- 230000005855 radiation Effects 0.000 description 3
- 230000006835 compression Effects 0.000 description 2
- 238000007906 compression Methods 0.000 description 2
- 238000005315 distribution function Methods 0.000 description 2
- 239000002360 explosive Substances 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 230000008447 perception Effects 0.000 description 2
- 239000000126 substance Substances 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- XHCLAFWTIXFWPH-UHFFFAOYSA-N [O-2].[O-2].[O-2].[O-2].[O-2].[V+5].[V+5] Chemical group [O-2].[O-2].[O-2].[O-2].[O-2].[V+5].[V+5] XHCLAFWTIXFWPH-UHFFFAOYSA-N 0.000 description 1
- 230000003321 amplification Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000002457 bidirectional effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000003750 conditioning effect Effects 0.000 description 1
- 238000001816 cooling Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000001186 cumulative effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- AJNVQOSZGJRYEI-UHFFFAOYSA-N digallium;oxygen(2-) Chemical compound [O-2].[O-2].[O-2].[Ga+3].[Ga+3] AJNVQOSZGJRYEI-UHFFFAOYSA-N 0.000 description 1
- 239000000428 dust Substances 0.000 description 1
- 238000010892 electric spark Methods 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 230000004907 flux Effects 0.000 description 1
- 238000009432 framing Methods 0.000 description 1
- 229910001195 gallium oxide Inorganic materials 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000004297 night vision Effects 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000000779 smoke Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 239000000758 substrate Substances 0.000 description 1
- 238000000825 ultraviolet detection Methods 0.000 description 1
- 229910001935 vanadium oxide Inorganic materials 0.000 description 1
- 230000016776 visual perception Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B17/00—Fire alarms; Alarms responsive to explosion
- G08B17/06—Electric actuation of the alarm, e.g. using a thermally-operated switch
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/40—Image enhancement or restoration using histogram techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B17/00—Fire alarms; Alarms responsive to explosion
- G08B17/12—Actuation by presence of radiation or particles, e.g. of infrared radiation or of ions
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B17/00—Fire alarms; Alarms responsive to explosion
- G08B17/12—Actuation by presence of radiation or particles, e.g. of infrared radiation or of ions
- G08B17/125—Actuation by presence of radiation or particles, e.g. of infrared radiation or of ions by using a video camera to detect fire or smoke
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B7/00—Signalling systems according to more than one of groups G08B3/00 - G08B6/00; Personal calling systems according to more than one of groups G08B3/00 - G08B6/00
- G08B7/06—Signalling systems according to more than one of groups G08B3/00 - G08B6/00; Personal calling systems according to more than one of groups G08B3/00 - G08B6/00 using electric transmission, e.g. involving audible and visible signalling through the use of sound and light sources
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10048—Infrared image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Business, Economics & Management (AREA)
- Emergency Management (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Closed-Circuit Television Systems (AREA)
- Fire-Detection Mechanisms (AREA)
Abstract
The application relates to a dual-band far infrared fusion and superposition imaging and fire early warning method and system. The method comprises the following steps: the temperature measurement acquisition module acquires visible light images/videos, far infrared light images/videos and temperatures of a monitored area; the image fusion and superposition module fuses and superposes the far infrared light image/video and the visible light image/video to obtain a fusion and superposition image/video; the temperature measurement analysis module sets an alarm temperature interval and analyzes whether the temperature enters the alarm temperature interval or not; the sound and light alarm module and the alarm display give an alarm when monitoring abnormality; the monitoring center monitors and outputs and displays the image/video, the temperature and the light signal, and controls the temperature measurement acquisition module to snapshot the image/video of the monitored area when giving an alarm. The application realizes automatic detection of disastrous accidents such as over-temperature spontaneous combustion, explosion and the like, reduces the occurrence of the accidents, combines automatic detection, identification and alarm modes, and realizes unattended operation and high-efficiency management.
Description
Technical Field
The application relates to the technical field of fire early warning, in particular to a dual-waveband far infrared fusion and superposition imaging and fire early warning method and system.
Background
In recent years, the economy of China continues to develop rapidly, the traffic is developed increasingly, the highway is gradually paid attention to the nation and people as an important component in the traffic field, with the rapid increase of the holding capacity of motor vehicles in China, the contradiction between the full-load application of the highway and the current situation of traffic management is further aggravated, and particularly, the management measures of the highway supervision department on large-scale passenger and cargo vehicles and special flammable and explosive chemical product transport vehicles are very important. In order to avoid tragedies of inflammable and explosive chemical warehouses occurring in the process of highway transportation, how to utilize advanced technological means to enhance the monitoring intensity of highways, prevent disasters caused by the inflammability of large vehicles, reduce the potential safety hazards of the vehicles in the transportation state, avoid the phenomena of fire, explosion and the like, prevent the hidden dangers from being unburnt and become an important subject of intelligent high-speed management, and the problem to be solved urgently by the highway management part is also solved.
In order to effectively and timely monitor the potential safety hazards, a real-time vehicle overtemperature early warning measurement and control system must be established. The traditional video monitoring mode can not realize the overtemperature pre-alarming function, can only carry out post-remedial treatment and emergency treatment even if spontaneous combustion, explosion and other conditions occur, and can not reduce or reduce the outbreak or generation of the disaster accidents of overtemperature or spontaneous combustion and dangerous goods explosion. In order to effectively solve the problems of prevention and treatment of such disaster accidents in advance by highway management departments, a mode which can realize combination of automatic detection, identification and video event alarm detection in advance is required to be provided, and unattended and efficient management is realized.
Disclosure of Invention
In order to facilitate automatic detection and early warning prevention of disastrous accidents such as over-temperature spontaneous combustion and explosion, and the like, so as to reduce the occurrence of the accidents, the application provides a dual-band far infrared fusion and superposition imaging and fire early warning method and system.
On the one hand, the application provides a double-waveband far infrared fusion and superposition imaging and fire early warning method, adopts following technical scheme:
a dual-band far infrared fusion imaging and fire early warning method comprises the following steps:
collecting visible light images/videos, far infrared light images/videos, temperatures and fire signals of a monitoring area;
fusing and overlapping the far infrared light image/video and the visible light image/video to obtain a fused and overlapped image/video;
setting an alarm temperature interval, and giving an alarm when the temperature enters the alarm temperature interval or a fire signal is detected;
when an alarm is given, the monitoring area is subjected to snapshot of visible light images/videos and far infrared light images/videos.
By adopting the technical scheme, all-weather real-time monitoring is carried out on the monitored area based on the visible light and far infrared light technologies. Visible light images are suitable for human visual perception due to their high spatial resolution and image contrast, but are highly susceptible to harsh conditioning. The far infrared image has better scene anti-interference capability, so the visible light image and the far infrared image fusion technology have complementarity, not only can meet the application link in the daytime, but also can not be influenced by rain, fog and dust at night. The alarm is timely sent out when high temperature or fire occurs, accidents such as over-temperature combustion, explosion and the like are effectively reduced, the combination of automatic detection, identification and alarm modes is realized in advance, and unattended and efficient management is realized.
Optionally, the far-infrared light image/video and the visible light image/video are fused and superimposed to obtain a fused and superimposed image/video, which specifically includes: and performing image denoising, image enhancement and image registration on the visible light image/video and the far infrared light image/video, and performing fusion processing on the processed visible light image/video and the far infrared light image/video.
By adopting the technical scheme, the far infrared light image and the visible light image are preprocessed, namely, image denoising, image enhancement and image registration are carried out, then image fusion is carried out, the advantages of the heat radiation information of the far infrared light image and the advantages of the detail texture information of the visible light image can be combined, and the fused image with abundant details and scene perception capability is obtained.
Optionally, the image denoising process specifically includes: denoising the visible light image/video and the far infrared light image/video by a Gaussian filtering mode, scanning each pixel point of the image/video in sequence by using a template with the size of N x N, wherein N is the width/height of the template, and replacing the gray value of the pixel point in the center of the template by the weighted average gray value of all pixels in the template.
By adopting the technical scheme, the visible light image/video and the far infrared light image/video are subjected to smooth denoising, the weight of the pixel point closer to the center is larger, and the weight of the pixel point far away from the center is smaller, so that the edge characteristics of the image can be maintained to a certain extent.
Optionally, the image enhancement processing specifically includes: and converting the denoised image/video into a YUV color space, redistributing pixel values of the image/video by using a gray histogram equalization algorithm, and converting the image/video into an RGB image/video for enhancing the contrast of the image/video.
By adopting the technical scheme, the image enhancement processing is carried out on the denoised image/video, the overall or local characteristics of the image are purposefully emphasized, the original unclear image is made clear, the difference between different object characteristics in the image is enlarged, the image quality is improved, and the image interpretation and identification effects are enhanced.
Optionally, the image registration processing specifically includes: extracting the features of the enhanced image/video, performing feature description on the extracted image features for image feature matching, and screening and removing the image features which are wrongly matched through a search optimization strategy after the image features are matched to obtain an accurate image feature matching pair; selecting a transformation model and parameters thereof according to the feature description; according to the transformation model and the matched image characteristics, carrying out image registration; the transformation model includes an affine transformation model, a perspective transformation model, and a polynomial transformation model.
By adopting the technical scheme, deformation such as image offset, rotation and distortion is easy to occur in the fusion process of the visible light image and the far infrared light image, and the images are finely matched before fusion in order to further improve the fusion quality of the images, so that the performance loss caused by pixel offset errors in the fusion process of the images is reduced.
Optionally, the fusion processing is performed on the processed visible light image/video and far-infrared light image/video, and specifically: firstly, decomposing an image/video into low-frequency components and high-frequency components with different scales and different directions; secondly, respectively fusing the low-frequency component and the high-frequency component through a fusion rule; and finally obtaining a fused image/video through Contourlet conversion.
By adopting the technical scheme, the visible light image and the far infrared light image under the same scene are fused, so that the generated fused image contains the texture details, the edge contour, the thermal target and other significant information in the visible light image, and has abundant information content and stronger scene perception capability.
On the other hand, this application provides a double-waveband far infrared fuses folds formation of image and fire early warning system, adopts following technical scheme:
a dual-band far infrared fusion and superposition imaging and fire early warning system comprises:
the temperature measurement acquisition module comprises a temperature measurement acquisition module,
the visible light imaging submodule is used for acquiring visible light images/videos of the monitoring area; and the number of the first and second groups,
the far infrared thermal imaging sub-module is used for collecting far infrared images/videos and temperatures of a monitored area;
the image fusion and superposition module is used for fusing and superposing the far infrared light image/video and the visible light image/video to obtain a fusion and superposition image/video, and the fusion and superposition image/video can display a thermal imaging edge-hooking graph according to the temperature difference; and the number of the first and second groups,
the temperature measurement analysis module is used for setting an alarm temperature interval and analyzing whether the temperature acquired by the far infrared thermal imaging submodule enters the alarm temperature interval or not; and the number of the first and second groups,
the fire detection module is used for detecting fire signals in the monitoring area; and the number of the first and second groups,
the sound-light alarm module is used for giving out sound-light alarm when the temperature enters an alarm temperature range or when the fire detection module detects a fire light signal; and the number of the first and second groups,
the alarm display is used for displaying alarm information when the temperature enters an alarm temperature range or when the fire detection module detects a fire light signal; and the number of the first and second groups,
the monitoring center is used for monitoring and outputting and displaying the visible light image/video, the far infrared light image/video, the temperature and the fire light signal; and when the temperature enters an alarm temperature range or when the fire detection module detects a fire light signal, controlling the temperature measurement acquisition module to snapshot images/videos of the monitored area.
By adopting the technical scheme, all-weather monitoring is carried out on a monitored area by adopting the visible light imaging equipment, the far infrared thermal imaging equipment and the fire detection equipment, the collected temperature, fire signals and images/videos are transmitted to the control center, the over-temperature point position and the over-temperature degree are automatically detected and identified by the temperature measurement analysis module, the fire signals are detected by the fire detection module, and an alarm is sent out when abnormality occurs. The automatic detection and early warning prevention of the over-temperature spontaneous combustion, explosion and other disastrous accidents are realized, so that the accidents are reduced.
Optionally, the image overlaying module includes,
the image preprocessing submodule is used for carrying out image denoising, image enhancement and image registration processing on the visible light image/video and the far infrared light image/video; and the number of the first and second groups,
and the image fusion submodule is used for carrying out fusion processing on the preprocessed visible light image/video and far infrared light image/video.
By adopting the technical scheme, the image preprocessing submodule and the image fusion submodule are adopted, so that the purpose of keeping edges is achieved while the image is smoothed, high-frequency components in the fused image can be fully kept, and the quality of the fused image can be improved.
Optionally, the system further includes a vehicle identification module, configured to collect basic information of the vehicle; the basic information of the vehicle comprises information such as a license plate and a vehicle type.
By adopting the technical scheme, when the system is applied to a highway, all-weather real-time monitoring is carried out on a monitored road section, the traffic flow and the number of the vehicles can be clearly identified aiming at a bidirectional lane, the overtemperature point and the ignition point of the vehicle are monitored in real time, and the potential safety hazard of the vehicle in a transportation state is reduced.
Optionally, the system further includes a storage configuration module, configured to store basic information of the vehicle, visible light images/videos, far infrared images/videos, overlay images/videos, and alarm information, and set storage locations of the visible light images/videos, the far infrared images/videos, and the overlay images/videos.
By adopting the technical scheme, the system can timely and accurately record and store each detail condition generated in the monitoring area, and provide an important reference basis for the inspection and maintenance of related management departments.
To sum up, the application comprises the following beneficial technical effects:
(1) the method and the device can collect and display the visible light image/video and the far infrared light image/video of the monitored target, and obtain a clear fusion image/video after the visible light image/video and the far infrared light image/video are fused. By fusing the images/videos, the user can easily find the positions of the over-temperature point and the ignition point.
(2) When the temperature of the monitored target enters an alarm temperature interval or a fire signal appears on the monitored target, an alarm is automatically sent out, and meanwhile, the temperature measurement acquisition module carries out image/video capture on the monitored target, so that automatic detection on hidden dangers such as overtemperature or spontaneous combustion is realized.
(3) The system can store basic information, visible light images/videos, far infrared images/videos, fusion and overlapping images/videos and alarm information of the vehicle collected by the system, and provides management and analysis data for a manager.
Drawings
Fig. 1 is a schematic flowchart illustrating an image fusion module according to an embodiment of the present disclosure.
Fig. 2 is a block diagram of a system architecture according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is further described in detail below with reference to fig. 1-2 and the embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
A far infrared thermal imaging device belongs to a passive infrared night vision technology, and adopts the principle that a semiconductor device with an internal photoelectric effect is used as a detector, a far infrared image radiated by an object is converted into a charge image, and the charge image is converted into a visible image of human eyes by a display device after information processing. The thermal imaging principle is to image by using the thermal contrast generated by the difference of temperature or emissivity between the target and the surrounding environment, and the imaging function is not realized by the refraction and reflection of light. Therefore, the equipment can effectively penetrate through rain and fog, the observation range and distance of the equipment are not influenced by rain and fog weather, a monitored target can be identified through smoke even in the period of fire, and the environment framing adaptability of the video monitoring system is greatly improved. The far infrared thermal imaging sub-module supports stepless amplification, has an image edge hooking function and a target highlighting function, and can be switched randomly according to the use scene of a user in a visible light fusion and far infrared light fusion mode (visible light is used as a main image and far infrared light is used as a main image).
Example 1:
the embodiment of the application discloses two-waveband far infrared fusion and superposition imaging and fire early warning method, including:
step 1, collecting visible light images/videos, far infrared light images/videos, temperatures and fire signals of a monitored area;
the method comprises the steps that visible light detection equipment, far infrared light detection equipment and fire detection equipment are installed in an area to be monitored, visible light images/videos of the monitored area are collected through the visible light equipment, far infrared light images/videos and temperatures of the monitored area are collected through the far infrared light equipment, and fire signals are detected through the fire detection equipment.
Step 2, as shown in fig. 1, fusing and overlapping the far-infrared light image/video and the visible light image/video to obtain a fused and overlapped image/video;
in this embodiment, a gaussian filtering method is used to denoise the visible light image and the far-infrared light image, a template with a size of 3 × 3 is used to scan each pixel point of the image in sequence, 3 is the width/height of the template, and the weighted average gray value of all pixels in the template is used to replace the gray value of the central pixel point of the template. The calculation formula is as follows:
g(x,y)={f(x-1,y-1)+f(x-1,y+1)+f(x+1,y-1)+f(x+1,y+1)+[f(x-1,y)+f(x,y-1)+f(x+1,y)+f(x,y+1)]*2+f(x,y)*4}/16； （1）
where f (x, y) is the gray value of the (x, y) point in the image, and g (x, y) is the value of the point after gaussian filtering.
the image enhancement processing specifically comprises: and converting the denoised image into a YUV color space image, redistributing the pixel value of the image/video by utilizing a gray histogram equalization algorithm, and converting the image into an RGB image for enhancing the contrast of the image.
The gray histogram equalization algorithm is specifically as follows:
let r denote the original image gray scale, s denote the image gray scale after histogram correction:
wherein,
for the transformation function, the following conditions are satisfied:
firstly, a monotone increasing function is formed within r is more than or equal to 0 and less than or equal to 1, and the order of gray levels from black to white is ensured to be unchanged;
r is more than or equal to 0 and less than or equal to 1, and r is more than or equal to 0 and less than or equal to 1
And ≦ 1, ensuring that the mapped pixel grayscale is within the allowed range.
The inverse transformation relationship is as follows:
for distribution functions of variable s
Expressed, then the distribution function is:
wherein,
is the probability density of the variable r,
is the probability density of the variable s;
histogram of the original image is passed
Adjusted to a uniformly distributed histogram, assuming
Then, there are:
integrating two sides to obtain:
the above equation shows that the histogram equalization can be achieved when the transform function is a cumulative histogram function of r.
For discrete digital images, frequency is used instead of probability, and then the transformation function
Can be expressed as:
the above equation shows that the gray value of each pixel after equalization
Can be directly calculated from the histogram of the original image.
III, carrying out image registration processing on the visible light image/video and the far infrared light image/video;
the image registration processing specifically comprises: extracting the features of the enhanced image/video, performing feature description on the extracted image features for image feature matching, and screening and removing the image features which are wrongly matched through a search optimization strategy after the image features are matched to obtain an accurate image feature matching pair; selecting an affine transformation model and parameters thereof according to the feature description; and carrying out image registration according to the affine transformation model and the matched image characteristics.
The affine transformation model is represented as:
wherein,
，
respectively representing the corresponding coordinates in the two images,k、
、
、
i.e. the registration parameters of the model,ka scale factor representing the scaling of the image,
which indicates the angle of rotation of the disc,
、
representing coordinate points
。
IV, carrying out image fusion processing on the processed visible light image/video and far infrared light image/video;
the image fusion processing specifically comprises the following steps: first, the visible light image a and the far-infrared light image B are decomposed into low-frequency components and high-frequency components by the Contourlet transform. Secondly, defining significance according to gray level difference between pixels, adaptively constructing a weight value by means of image significance and a Gaussian membership function, fusing low-frequency components of the image A and the image B by a weighted average method, and fusing high-frequency components of the image A and the image B by a method of taking a large absolute value; and finally, obtaining an image C formed by fusing the image B by taking the image A as a base and an image D formed by fusing the image A by taking the image B as a base through Contourlet inverse transformation. Wherein the transparency of the image B is adjustable. In the image C, the position of the image B may be moved, or the image B may be enlarged and reduced.
Step 3, setting an alarm temperature interval, and sending an alarm when the temperature enters the alarm temperature interval or a fire signal is detected;
the far infrared detection device is capable of converting the received infrared thermal radiation signal into an electrical signal for convenient measurement or observation. Setting an alarm temperature range of 80-150 ℃, collecting the temperature in a monitoring area by far infrared detection equipment, converting the collected temperature into an electrical signal, sending the converted electrical signal to a temperature analysis module to analyze the electrical signal, and when the temperature is judged to enter the alarm temperature range, sending an overtemperature alarm by connected alarm equipment, and making a precautionary measure by a monitoring manager in time.
The solar blind ultraviolet light searchlight starts to work, when a fire light signal is captured, the photoelectric effect is utilized, the optical radiation signal is converted into an electrical signal which is easy to receive and process, the alarm equipment connected with the solar blind ultraviolet light searchlight sends out a fire alarm after receiving the electrical signal, and monitoring management personnel can take precautionary measures on the basis.
Step 4, when an alarm is given out, the monitoring area is subjected to snapshot of visible light images/videos and far infrared light images/videos;
when the monitoring center monitors that an overtemperature alarm is sent, the visible light low-illumination searchlight and the non-contact far-infrared thermal imaging searchlight are controlled to instantly grab a monitored target, and a corresponding temperature value is marked in the grabbed image; if the temperature is continuously in the alarm temperature interval, the image is grabbed at certain time intervals; and a section of video is intercepted by taking a time point before the temperature enters the alarm temperature interval as a starting point.
When the monitoring center monitors that a fire alarm is sent out, the visible light low-illumination searchlight and the non-contact far-infrared thermal imaging searchlight are controlled to instantly grab a monitored target; if the fire signal exists continuously, the picture is grabbed at certain time intervals; but also intercepts a section of video starting from a time point before the fire signal appears.
And all the alarm information, the captured images and the videos are stored for later reference and analysis.
Example 2:
the embodiment of the application discloses two wave band far infrared fuse folds formation of image and fire early warning system, as shown in figure 2, include: the system comprises a temperature measurement acquisition module, an image fusion and superposition module, a temperature measurement analysis module, a fire detection module, an audible and visual alarm module, an alarm display and a monitoring center.
The temperature measurement acquisition module comprises a visible light imaging sub-module and a far infrared thermal imaging sub-module;
the visible light imaging submodule is a visible light low-illumination searchlight, and can automatically adjust the intensity of absorbed visible light flux according to the intensity of external illumination, so that the consistency of fusion and superposition with a far-infrared thermal imaging image is better facilitated. The specific parameters are as follows: the sensor type is 2500 x 1600, and the effective pixel is 400 ten thousand pixels; the lowest illumination is color 0.01Lux and black and white 0.005 Lux; the fixed focus lens F =1.4, and the focal length is 7mm-45 mm; video-driven automatic aperture; video compression standard H264/MJPEG; support dual code stream or multi code stream, support ONVIF protocol.
The far infrared thermal imaging submodule is a far infrared thermal imaging searchlight, and has the main advantages that the temperature measurement visual field MFOV is 1, and the temperature measurement is accurate to 1 pixel point. The specific parameters are as follows: the sensor type is vanadium oxide, and the effective pixel is 220 multiplied by 160; the focal length of the fixed-focus lens is 19mm/25mm/35 mm; video-driven automatic aperture; video compression standard H264/MJPEG; support dual code stream or multi code stream, support ONVIF protocol.
The fire detection equipment is a solar blind ultraviolet light searchlight, has a wide band ultraviolet detection range and an advanced gallium oxide detection substrate material, has sensitive capturing and early warning capabilities on sparks and flames, and can effectively detect fire light signals such as electric sparks, electric arcs, static electricity and the like.
The monitoring center comprises a video output module which is used for outputting and displaying visible light images/videos, far infrared light images/videos, temperature and fire signals. The video output module supports more than 6 color background switching and output display, supports output display of far infrared images/videos, visible light images/videos and fusion images/videos, and simultaneously supports the overlapped window display of the two imaging videos.
The system also comprises a vehicle identification module used for collecting the basic information of the vehicle; the basic information of the vehicle comprises information such as a license plate and a vehicle type.
The working principle of the system is as follows: the method comprises the steps of firstly setting a visible light low-illumination searchlight, a far-infrared thermal imaging searchlight, a solar blind ultraviolet searchlight, an acousto-optic alarm module, an alarm display and a vehicle identification module, configuring information, starting an automatic detection function and detecting whether a vehicle passes through or not. When a vehicle passes through the system, the vehicle identification module shoots the license plate number, the solar blind ultraviolet searchlight starts to detect a fire light signal, the visible light low-illumination searchlight starts to shoot a visible light image, and the far infrared thermal imaging searchlight starts to measure the temperature and shoots a far infrared image. The image fusion module fuses the obtained visible light image and the far infrared light image to obtain a fused image, and transmits the measured temperature, the snap-shot image and the fused image to a video output module of the monitoring center for displaying. The temperature measurement analysis module judges the measured temperature, the judgment result shows that the temperature enters an alarm temperature interval, the sound and light alarm module gives an alarm, the alarm display displays warning information, and meanwhile, the monitoring center controls the temperature measurement acquisition module to carry out image/video snapshot and stores the monitoring data into the alarm database; and if the judgment result shows that the temperature does not enter the alarm temperature interval, ending the overtemperature monitoring. The solar blind ultraviolet searchlight detects a fire light signal, the acousto-optic alarm module gives an alarm, the alarm display displays warning information, and meanwhile, the monitoring center controls the temperature measurement acquisition module to carry out image/video snapshot and stores the monitoring data into the alarm database; and when the fire signal is not detected, ending the fire monitoring. According to the supervision data of different road sections in the alarm database, a management unit can increase or decrease the cooling pool or the danger avoiding lane in different road sections according to different seasons, and management and analysis data are provided for keeping smooth, building smooth and safe and high-speed.
The system adopts a network transmission system based on an IP standard protocol, and can directly utilize an optical fiber network in the existing video monitoring network to realize connection with the original video monitoring system and the acousto-optic alarm module. The whole system has strong capability of crossing gateways and network segments. Corresponding master control management centers and branch control management centers are established in the project road monitoring center and the three-road section monitoring branch centers, namely, the whole system adopts a 2-level distributed architecture, namely, a plurality of master control rooms and a plurality of branch control rooms, and the functional requirements are as follows:
(1) the master control room can manage all temperature measurement early warning points of the whole project;
(2) the sub-control rooms can only manage temperature measurement early warning points within the authority range of the sub-control rooms, and a plurality of sub-control rooms are not interfered with one another;
(3) the priority level of a distributed architecture default master control room is highest, when the master control and the branch control operate the same point at the same time, the system defaults to execute the task of the master control first, and after the execution is finished, if the task of the branch control is not finished, the task of the branch control is executed;
(4) the system management software of the master control room and the sub-control room can be installed on a highly-configured PC to normally operate, and the background server can adopt an original video server.
The foregoing is a preferred embodiment of the present application and is not intended to limit the scope of the application in any way, and any features disclosed in this specification (including the abstract and drawings) may be replaced by alternative features serving equivalent or similar purposes, unless expressly stated otherwise. That is, unless expressly stated otherwise, each feature is only an example of a generic series of equivalent or similar features.
Claims (10)
1. A dual-band far infrared fusion imaging and fire early warning method is characterized by comprising the following steps:
collecting visible light images/videos, far infrared light images/videos, temperatures and fire signals of a monitoring area;
fusing and overlapping the far infrared light image/video and the visible light image/video to obtain a fused and overlapped image/video;
when the temperature enters an alarm temperature range or a fire signal is detected, an alarm is given;
when an alarm is given, the monitoring area is subjected to snapshot of visible light images/videos and far infrared light images/videos.
2. The dual-band far infrared fusion imaging and fire early warning method of claim 1, wherein the far infrared light image/video and the visible light image/video are fused and superposed to obtain a fusion image/video, and the method specifically comprises the following steps: and performing image denoising, image enhancement and image registration on the visible light image/video and the far infrared light image/video, and performing fusion processing on the processed visible light image/video and the far infrared light image/video.
3. The dual-band far infrared fusion imaging and fire early warning method of claim 2, wherein the image denoising process specifically comprises: denoising the visible light image/video and the far infrared light image/video by a Gaussian filtering mode, scanning each pixel point of the image/video in sequence by using a template with the size of N x N, wherein N is the width/height of the template, and replacing the gray value of the pixel point in the center of the template by the weighted average gray value of all pixels in the template.
4. The dual-band far infrared fusion imaging and fire early warning method of claim 2, wherein the image enhancement processing specifically comprises: and converting the denoised image/video into a YUV color space, redistributing pixel values of the image/video by using a gray histogram equalization algorithm, and converting the image/video into an RGB image/video for enhancing the contrast of the image/video.
5. The dual-band far infrared fusion imaging and fire early warning method of claim 2, wherein the image registration processing specifically comprises: extracting the features of the enhanced image/video, performing feature description on the extracted image features for image feature matching, and screening and removing the image features which are wrongly matched through a search optimization strategy after the image features are matched to obtain an accurate image feature matching pair; selecting a transformation model and parameters thereof according to the feature description; and carrying out image registration according to the transformation model and the matched image characteristics.
6. The dual-band far infrared fusion imaging and fire early warning method of claim 2, wherein the fusion processing is performed on the processed visible light image/video and far infrared light image/video, and specifically comprises: firstly, decomposing an image/video into low-frequency components and high-frequency components with different scales and different directions; secondly, respectively fusing the low-frequency component and the high-frequency component through a fusion rule; and finally obtaining a fused image/video through Contourlet conversion.
7. The utility model provides a dual band far infrared fuses folds formation of image and fire early warning system which characterized in that includes:
the temperature measurement acquisition module comprises a temperature measurement acquisition module,
the visible light imaging submodule is used for acquiring visible light images/videos of the monitoring area; and the number of the first and second groups,
the far infrared thermal imaging sub-module is used for collecting far infrared images/videos and temperatures of a monitored area;
the image fusion and superposition module is used for fusing and superposing the far infrared light image/video and the visible light image/video to obtain a fusion and superposition image/video; and the number of the first and second groups,
the temperature measurement analysis module is used for setting an alarm temperature interval and analyzing whether the temperature acquired by the far infrared thermal imaging submodule enters the alarm temperature interval or not; and the number of the first and second groups,
the fire detection module is used for detecting fire signals in the monitoring area; and the number of the first and second groups,
the sound-light alarm module is used for giving out sound-light alarm when the temperature enters an alarm temperature range or when the fire detection module detects a fire light signal; and the number of the first and second groups,
the alarm display is used for displaying alarm information when the temperature enters an alarm temperature range or when the fire detection module detects a fire light signal; and the number of the first and second groups,
the monitoring center is used for monitoring and outputting and displaying the visible light image/video, the far infrared light image/video, the temperature and the fire light signal; and when the temperature enters an alarm temperature range or when the fire detection module detects a fire light signal, controlling the temperature measurement acquisition module to snapshot images/videos of the monitored area.
8. The dual band far infrared fusion imaging and fire early warning system of claim 7, wherein the image fusion module comprises,
the image preprocessing submodule is used for performing image denoising processing, image enhancement processing and image registration processing on the visible light image/video and the far infrared light image/video; and the number of the first and second groups,
and the image fusion submodule is used for carrying out fusion processing on the preprocessed visible light image/video and far infrared light image/video.
9. The dual band far infrared fusion imaging and fire early warning system of claim 7, further comprising a vehicle identification module for collecting basic information of the vehicle.
10. The dual band far infrared stacked imaging and fire pre-warning system of claim 9, further comprising a storage configuration module for storing basic information of the vehicle, visible light images/video, far infrared images/video, stacked images/video and alarm information, and setting storage locations of the visible light images/video, far infrared images/video and stacked images/video.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011077708.9A CN112365669A (en) | 2020-10-10 | 2020-10-10 | Dual-band far infrared fusion-superposition imaging and fire early warning method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011077708.9A CN112365669A (en) | 2020-10-10 | 2020-10-10 | Dual-band far infrared fusion-superposition imaging and fire early warning method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112365669A true CN112365669A (en) | 2021-02-12 |
Family
ID=74507530
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011077708.9A Pending CN112365669A (en) | 2020-10-10 | 2020-10-10 | Dual-band far infrared fusion-superposition imaging and fire early warning method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112365669A (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114187735A (en) * | 2021-12-03 | 2022-03-15 | 武汉理工光科股份有限公司 | Fire alarm method and device based on multiple modes, electronic equipment and storage medium |
| CN114239654A (en) * | 2021-12-17 | 2022-03-25 | 电子科技大学 | Temperature identification and classification method based on dual-waveband pyroelectric device |
| CN114399882A (en) * | 2022-01-20 | 2022-04-26 | 红骐科技(杭州)有限公司 | Fire source detection, identification and early warning method for fire-fighting robot |
| CN114566056A (en) * | 2022-02-28 | 2022-05-31 | 招商局重庆公路工程检测中心有限公司 | Highway tunnel driving safety risk identification, prevention and control method and system |
| CN114693579A (en) * | 2022-05-23 | 2022-07-01 | 北京物资学院 | Monitoring image processing system in logistics warehouse |
| CN114913667A (en) * | 2022-05-06 | 2022-08-16 | 合肥科大立安安全技术有限责任公司 | Anti-interference sensing device for smoke in early stage of fire and anti-interference method thereof |
| CN115512528A (en) * | 2022-08-16 | 2022-12-23 | 金茂云科技服务(北京)有限公司 | Fire monitoring system and method and computer equipment |
| CN116092018A (en) * | 2023-04-10 | 2023-05-09 | 同方德诚(山东)科技股份公司 | Fire-fighting hidden danger monitoring method and system based on intelligent building |
| CN116913033A (en) * | 2023-05-29 | 2023-10-20 | 东莞市众可智能科技有限公司 | Fire big data remote detection and early warning system |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101568029A (en) * | 2008-11-28 | 2009-10-28 | 深圳市高斯贝尔家居智能电子有限公司 | Device and method for monitoring mobile phone video based on optimized H.264 coding standard |
| CN105551110A (en) * | 2016-02-05 | 2016-05-04 | 北京奇虎科技有限公司 | Traveling vehicle data recording method, device and system |
| CN106335432A (en) * | 2016-08-29 | 2017-01-18 | 赵谕 | Smart rearview mirror with driving assistant system, control system and processing method |
| CN108921205A (en) * | 2018-06-14 | 2018-11-30 | 浙江大学 | A kind of skin disease clinical image classification method based on multi-feature fusion |
| KR102084884B1 (en) * | 2019-07-10 | 2020-03-04 | 에이치에스솔루션 주식회사 | The fire prevention detection system based on artificial intelligence technology |
| CN111371895A (en) * | 2020-03-11 | 2020-07-03 | 四川路桥建设集团交通工程有限公司 | Electromechanical equipment management system for expressway tunnel and method thereof |
| CN111627183A (en) * | 2020-06-10 | 2020-09-04 | 烟台艾睿光电科技有限公司 | Fire-fighting early warning linkage system |
-
2020
- 2020-10-10 CN CN202011077708.9A patent/CN112365669A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101568029A (en) * | 2008-11-28 | 2009-10-28 | 深圳市高斯贝尔家居智能电子有限公司 | Device and method for monitoring mobile phone video based on optimized H.264 coding standard |
| CN105551110A (en) * | 2016-02-05 | 2016-05-04 | 北京奇虎科技有限公司 | Traveling vehicle data recording method, device and system |
| CN106335432A (en) * | 2016-08-29 | 2017-01-18 | 赵谕 | Smart rearview mirror with driving assistant system, control system and processing method |
| CN108921205A (en) * | 2018-06-14 | 2018-11-30 | 浙江大学 | A kind of skin disease clinical image classification method based on multi-feature fusion |
| KR102084884B1 (en) * | 2019-07-10 | 2020-03-04 | 에이치에스솔루션 주식회사 | The fire prevention detection system based on artificial intelligence technology |
| CN111371895A (en) * | 2020-03-11 | 2020-07-03 | 四川路桥建设集团交通工程有限公司 | Electromechanical equipment management system for expressway tunnel and method thereof |
| CN111627183A (en) * | 2020-06-10 | 2020-09-04 | 烟台艾睿光电科技有限公司 | Fire-fighting early warning linkage system |
Non-Patent Citations (3)
| Title |
|---|
| 杨帆: "《数字图像处理与分析 第4版》", 31 January 2019, 北京航空航天大学出版社 * |
| 潘文明等: "《手把手教你学FPGA设计：基于大道至简的至简设计法》", 30 April 2017, 北京航空航天大学出版社 * |
| 苗启广等: "《多传感器图像融合技术及应用》", 30 April 2014, 西安电子科技大学出版社 * |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114187735B (en) * | 2021-12-03 | 2024-04-23 | 武汉理工光科股份有限公司 | Multi-mode-based fire alarm method and device, electronic equipment and storage medium |
| CN114187735A (en) * | 2021-12-03 | 2022-03-15 | 武汉理工光科股份有限公司 | Fire alarm method and device based on multiple modes, electronic equipment and storage medium |
| CN114239654A (en) * | 2021-12-17 | 2022-03-25 | 电子科技大学 | Temperature identification and classification method based on dual-waveband pyroelectric device |
| CN114239654B (en) * | 2021-12-17 | 2023-05-12 | 电子科技大学 | Temperature identification and classification method based on dual-band pyroelectric device |
| CN114399882A (en) * | 2022-01-20 | 2022-04-26 | 红骐科技(杭州)有限公司 | Fire source detection, identification and early warning method for fire-fighting robot |
| CN114566056A (en) * | 2022-02-28 | 2022-05-31 | 招商局重庆公路工程检测中心有限公司 | Highway tunnel driving safety risk identification, prevention and control method and system |
| CN114566056B (en) * | 2022-02-28 | 2023-01-24 | 招商局重庆公路工程检测中心有限公司 | Highway tunnel driving safety risk identification, prevention and control method and system |
| CN114913667A (en) * | 2022-05-06 | 2022-08-16 | 合肥科大立安安全技术有限责任公司 | Anti-interference sensing device for smoke in early stage of fire and anti-interference method thereof |
| CN114913667B (en) * | 2022-05-06 | 2023-08-11 | 合肥科大立安安全技术有限责任公司 | Fire early smoke anti-interference sensing device and anti-interference method thereof |
| CN114693579A (en) * | 2022-05-23 | 2022-07-01 | 北京物资学院 | Monitoring image processing system in logistics warehouse |
| CN115512528A (en) * | 2022-08-16 | 2022-12-23 | 金茂云科技服务(北京)有限公司 | Fire monitoring system and method and computer equipment |
| CN115512528B (en) * | 2022-08-16 | 2024-05-24 | 金茂云科技服务(北京)有限公司 | Fire monitoring system, method and computer equipment |
| CN116092018A (en) * | 2023-04-10 | 2023-05-09 | 同方德诚(山东)科技股份公司 | Fire-fighting hidden danger monitoring method and system based on intelligent building |
| CN116092018B (en) * | 2023-04-10 | 2023-08-25 | 同方德诚(山东)科技股份公司 | Fire-fighting hidden danger monitoring method and system based on intelligent building |
| CN116913033B (en) * | 2023-05-29 | 2024-04-05 | 深圳市兴安消防工程有限公司 | Fire big data remote detection and early warning system |
| CN116913033A (en) * | 2023-05-29 | 2023-10-20 | 东莞市众可智能科技有限公司 | Fire big data remote detection and early warning system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112365669A (en) | Dual-band far infrared fusion-superposition imaging and fire early warning method and system | |
| WO2021253961A1 (en) | Intelligent visual perception system | |
| CN102693603B (en) | Dual spectrum based intelligent monitoring system for forest fire prevention | |
| KR101635000B1 (en) | Fire detector and system using plural cameras | |
| CN101673448B (en) | Method and system for detecting forest fire | |
| CN201629019U (en) | Forest fire detecting system | |
| CN201936415U (en) | Automatic forest fire identification and alarm system | |
| US9047515B2 (en) | Method and system for wildfire detection using a visible range camera | |
| KR102060045B1 (en) | Fire detector and system capable of measuring heat distribution | |
| JP6403687B2 (en) | Monitoring system | |
| Chumuang et al. | CCTV based surveillance system for railway station security | |
| CN105976365A (en) | Nocturnal fire disaster video detection method | |
| CN105828039A (en) | Harbor safety monitoring and water surface oil spilling monitoring system | |
| CN107229108B (en) | Passive infrared and active infrared panoramic monitoring linkage system and method | |
| CN113379628A (en) | Method for detecting thermal runaway of lithium battery based on infrared thermal imager | |
| CN109982047B (en) | Flight monitoring panorama fusion display method | |
| JPH0844874A (en) | Image change detector | |
| CN107147877A (en) | FX night fog day condition all-weather colorful video imaging system and its construction method | |
| KR20140109671A (en) | Flame dete ction method based on gray imaging signal of a cameras | |
| JP2006285399A (en) | Image monitoring method and device for monitoring motion of vehicle at intersection | |
| Tamaki et al. | An automatic compensation system for unclear area in 360-degree images using pan-tilt camera | |
| JP2007235760A (en) | Ultraviolet image, imaging apparatus for imaging ultraviolet image, and imaging method | |
| Wang et al. | Research on application of new technology in foreign object debris detection field | |
| KR102624333B1 (en) | Deep Learning-based Fire Monitoring System | |
| CN115361499B (en) | Dual-machine cooperative border defense target recognition and tracking system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210212 |