CN117237779B - Image recognition method and system for visible light image and infrared image combined analysis - Google Patents
Image recognition method and system for visible light image and infrared image combined analysis Download PDFInfo
- Publication number
- CN117237779B CN117237779B CN202311515793.6A CN202311515793A CN117237779B CN 117237779 B CN117237779 B CN 117237779B CN 202311515793 A CN202311515793 A CN 202311515793A CN 117237779 B CN117237779 B CN 117237779B
- Authority
- CN
- China
- Prior art keywords
- image
- visible light
- target
- area
- processed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 39
- 238000004458 analytical method Methods 0.000 title claims description 14
- 230000007704 transition Effects 0.000 claims abstract description 48
- 230000004927 fusion Effects 0.000 claims abstract description 26
- 238000013527 convolutional neural network Methods 0.000 claims description 13
- 238000012549 training Methods 0.000 claims description 13
- 238000009499 grossing Methods 0.000 claims description 9
- 238000004422 calculation algorithm Methods 0.000 claims description 6
- 230000008569 process Effects 0.000 claims description 5
- 238000004590 computer program Methods 0.000 claims description 3
- 230000004913 activation Effects 0.000 claims description 2
- 238000011176 pooling Methods 0.000 claims description 2
- 238000002372 labelling Methods 0.000 claims 6
- 230000000694 effects Effects 0.000 abstract description 7
- 238000004364 calculation method Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000012544 monitoring process Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 238000012216 screening Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000003331 infrared imaging Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Landscapes
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses an image recognition method and system for combining and analyzing visible light images and infrared images, wherein the method comprises the following steps: acquiring a to-be-processed area of a visible light image, sliding the to-be-processed area on the edge of the to-be-processed area based on a preset sliding window, and selecting a target transition area connected with the to-be-processed area; intercepting a target area corresponding to a to-be-processed area and a target transition area of the visible light image in the infrared image to obtain an infrared sub-image; embedding the infrared sub-image into the visible light image according to the matching result of the characteristic points of the target transition region in the visible light image and the characteristic points in the infrared sub-image, and carrying out characteristic fusion on the region to be processed of the visible light image after the embedding of the infrared sub-image to obtain a target visible light image; and inputting the target visible light image into a preset image recognition model, and outputting an image type corresponding to the target visible light image. Unnecessary calculation amount is avoided, and the effect can be selectively improved.
Description
Technical Field
The invention belongs to the technical field of image recognition, and particularly relates to an image recognition method and system for combining and analyzing a visible light image and an infrared image.
Background
In a real-time monitoring and identification scene, due to outdoor illumination change or indoor low-light environment, the problems of color distortion, target shielding and the like are often faced when the visible light is used for shooting, so that the identification effect is poor. While using infrared imaging, although the target can be identified under complex illumination, the color and detail features are lost. The general solution is to combine the two, and fully exert the respective advantages.
However, in the existing method for fusing the visible light image and the infrared image, the whole image is generally fused in a process with larger computation amounts such as multi-scale decomposition and image registration, so that the efficiency requirement of real-time monitoring is difficult to meet.
Disclosure of Invention
The invention provides an image recognition method and system for combining and analyzing a visible light image and an infrared image, which are used for solving the technical problem that the original background structure of the visible light image is distorted only by simple pixel-level or feature-level global fusion.
In a first aspect, the present invention provides an image recognition method for combining and analyzing a visible light image and an infrared image, including:
acquiring a to-be-processed area of the visible light image, sliding the to-be-processed area on the edge of the to-be-processed area based on a preset sliding window, and selecting a target transition area connected with the to-be-processed area;
intercepting a target area corresponding to a to-be-processed area and a target transition area of the visible light image in the infrared image to obtain an infrared sub-image;
embedding the infrared sub-image into the visible light image according to the matching result of the characteristic points of the target transition region in the visible light image and the characteristic points in the infrared sub-image, and carrying out characteristic fusion on the region to be processed of the visible light image after the infrared sub-image is embedded to obtain a target visible light image;
inputting the target visible light image into a preset image recognition model, and enabling the image recognition model to output an image type corresponding to the target visible light image, wherein the image recognition model is obtained based on convolutional neural network training.
In a second aspect, the present invention provides an image recognition system for combined analysis of a visible light image and an infrared image, comprising:
the acquisition module is configured to acquire a to-be-processed area of the visible light image, slide on the edge of the to-be-processed area based on a preset sliding window, and select a target transition area connected with the to-be-processed area;
the intercepting module is configured to intercept a target area corresponding to a to-be-processed area and a target transition area of the visible light image in the infrared image to obtain an infrared sub-image;
the fusion module is configured to embed the infrared sub-image into the visible light image according to a matching result of the characteristic points of the target transition region in the visible light image and the characteristic points in the infrared sub-image, and perform characteristic fusion on a region to be processed of the visible light image after the infrared sub-image is embedded to obtain a target visible light image;
the output module is configured to input the target visible light image into a preset image recognition model, so that the image recognition model outputs an image type corresponding to the target visible light image, wherein the image recognition model is obtained based on convolutional neural network training.
In a third aspect, there is provided an electronic device, comprising: the system comprises at least one processor and a memory communicatively connected with the at least one processor, wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the steps of the image recognition method of the visible light image and infrared image combination analysis of any one of the embodiments of the invention.
In a fourth aspect, the present invention also provides a computer readable storage medium having stored thereon a computer program, the program instructions, when executed by a processor, cause the processor to perform the steps of the image recognition method of the combination analysis of a visible light image and an infrared image according to any of the embodiments of the present invention.
The image recognition method and system for combining and analyzing the visible light image and the infrared image have the following beneficial effects:
1. only the local shadow area or the interested target area of the visible light image is fused, the range is smaller, unnecessary calculation amount is avoided, and the effect can be selectively improved;
2. the texture features of the visible light image can be reserved in a large area, the infrared image can provide supplementary improvement, the integral visual effect of the visible light image can be kept, the advantages of the infrared image and the visible light image are combined better, the integral effect of the visible light image is exerted, and the effect of the infrared image on shadow and shielding is also exerted.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flowchart of an image recognition method for combining and analyzing visible light images and infrared images according to an embodiment of the present invention;
FIG. 2 is a block diagram of an image recognition system for combined analysis of visible light images and infrared images according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Referring to fig. 1, a flowchart of an image recognition method for combining and analyzing a visible light image and an infrared image is shown.
As shown in fig. 1, the image recognition method for combining and analyzing the visible light image and the infrared image specifically includes the following steps:
step S101, a region to be processed of the visible light image is obtained, sliding is carried out on the edge of the region to be processed based on a preset sliding window, and a target transition region connected with the region to be processed is selected.
In the step, after the to-be-processed area of the visible light image is acquired, the to-be-processed area is marked based on a marking frame, and the vertex of the marking frame is positioned at the edge of the to-be-processed area; sliding on the marking frame according to a preset sliding window, and selecting a first transition area connected with the area to be treated, wherein the first transition area is not overlapped with the area to be treated; judging whether the difference value between the pixel value of each characteristic point in the first transition area and the maximum pixel value of the characteristic point in the area to be processed is larger than a preset threshold value or not; if the difference value between the pixel value of a certain characteristic point in the first transition region and the maximum pixel value of the characteristic point in the region to be processed is larger than a preset threshold value, the preset region where the certain characteristic point is located is directly removed, and a target transition region is obtained, wherein the preset region is a circle with a preset radius taking the certain characteristic point as a circle center.
And step S102, intercepting a target area corresponding to the to-be-processed area and the target transition area of the visible light image in the infrared image to obtain an infrared sub-image.
In the step, the target size of the visible light image is obtained, and the size of the infrared image is adjusted to be consistent with the target size; and matching the characteristic points in the infrared image with the characteristic points of the target transition region in the visible light image based on the SIFT algorithm, and intercepting the infrared image according to a matching result to obtain an infrared sub-image.
It should be noted that, because the transition area is smaller, the scale invariance feature point detection can be realized by adopting the SIFT algorithm, the neighborhood range of the feature point in the SIFT algorithm is properly reduced, the feature point density is increased to make up for the shortage of the number, and meanwhile, the descriptor scale is correspondingly reduced to ensure consistency. And during matching, ratio screening and geometric verification are added to improve accuracy, and a relative self-adaptive ratio threshold is used for replacing a fixed threshold, so that the matching is more stable. And sorting the screening results through the matching scores, and selecting the subset with the highest confidence coefficient. And finally, performing model fitting by adopting a RANSAC algorithm for enlarging the inner points, so that more accurate image transformation estimation can be obtained, and accurate registration from the small image to the scene image is completed. The parameter tuning and matching strategy is improved, and the extraction and matching effects of SIFT feature points in the small image can be effectively improved.
Step S103, embedding the infrared sub-image into the visible light image according to the matching result of the characteristic points of the target transition region in the visible light image and the characteristic points in the infrared sub-image, and carrying out characteristic fusion on the region to be processed of the visible light image after the infrared sub-image is embedded to obtain the target visible light image.
In the step, firstly, gaussian smoothing is carried out on an input image I to obtain an image L, and then downsampling is carried out on the image L subjected to Gaussian smoothing by 2 times to obtain an image L1 which is used as a first layer of a Gaussian pyramid; repeating Gaussian smoothing and downsampling processes to construct a Gaussian pyramid containing higher-layer images, wherein the standard deviation of a first layer of the Gaussian pyramid is as followsThe standard deviation increases in order by k times, i.e. the standard deviation of the nth layer is +.>The method comprises the steps of carrying out a first treatment on the surface of the The pyramid layer number of the Gaussian pyramid is carried out according to the size proportion of the target area of the infrared sub-image and the area to be processed of the visible light imageRegistering, namely aligning pyramid scales corresponding to the target area and the area to be processed; setting target weights of different pyramid layers according to the image information quantity and importance under different scales, and carrying out weighted fusion on pixels or features corresponding to a target area and an area to be processed in each pyramid layer according to the target weights; and reconstructing a final target visible light image according to the fusion result of each layer of the pyramid and the corresponding relation between different layers of the pyramid.
Step S104, inputting the target visible light image into a preset image recognition model, and enabling the image recognition model to output an image type corresponding to the target visible light image, wherein the image recognition model is obtained based on convolutional neural network training.
In the step, iterative training is carried out on a convolutional neural network according to a visible light image sample and an infrared image sample to obtain an image recognition model, a target visible light image is input into a preset image recognition model, and the image recognition model outputs an image type corresponding to the target visible light image.
The network structure of the convolutional neural network comprises a convolutional layer, an activation layer, a pooling layer, a full connection layer and the like. The training set contains an original visible image and a corresponding infrared image sample. In the training process, the visible light image and the infrared image extract depth features through a network respectively, then the connection layer performs feature fusion, and the classification result is output through the full connection layer. And adopting a cross entropy loss function, iteratively adjusting network parameters through a back propagation algorithm, minimizing the loss function, and completing model training. The convolutional neural network fully fuses two mode information through end-to-end learning of the joint feature representation of the visible light and the infrared image, can perform more accurate image classification and identification, and improves the performance of a monitoring system based on image fusion.
In summary, the method only carries out fusion of the visible light and the infrared image in the target region of interest, so that the defect of a global method can be avoided, the background structure and color information of the visible light image are kept from being destroyed, the detection and identification capability of the target can be effectively improved, and meanwhile, the local fusion mode can reduce the calculated amount and realize real-time image processing. Overall, the local selective fusion strategy is more excellent than global fusion, and an optimal balance between accuracy and processing efficiency of the recognition result can be obtained.
Referring to fig. 2, a block diagram of an image recognition system for combined analysis of visible light images and infrared images is shown.
As shown in fig. 2, the image recognition system 200 includes an acquisition module 210, an interception module 220, a fusion module 230, and an output module 240.
The acquiring module 210 is configured to acquire a region to be processed of the visible light image, slide at an edge of the region to be processed based on a preset sliding window, and select a target transition region connected with the region to be processed; the intercepting module 220 is configured to intercept a target area corresponding to a to-be-processed area and a target transition area of the visible light image in the infrared image, so as to obtain an infrared sub-image; the fusion module 230 is configured to embed the infrared sub-image into the visible light image according to a matching result of the feature point of the target transition region in the visible light image and the feature point in the infrared sub-image, and perform feature fusion on a region to be processed of the visible light image after the infrared sub-image is embedded, so as to obtain a target visible light image; the output module 240 is configured to input the target visible light image into a preset image recognition model, so that the image recognition model outputs an image type corresponding to the target visible light image, wherein the image recognition model is obtained based on convolutional neural network training.
It should be understood that the modules depicted in fig. 2 correspond to the various steps in the method described with reference to fig. 1. Thus, the operations and features described above for the method and the corresponding technical effects are equally applicable to the modules in fig. 2, and are not described here again.
In other embodiments, the present invention further provides a computer readable storage medium, on which a computer program is stored, where the program instructions, when executed by a processor, cause the processor to perform the image recognition method of the combination analysis of the visible light image and the infrared image in any of the above method embodiments;
as one embodiment, the computer-readable storage medium of the present invention stores computer-executable instructions configured to:
acquiring a to-be-processed area of the visible light image, sliding the to-be-processed area on the edge of the to-be-processed area based on a preset sliding window, and selecting a target transition area connected with the to-be-processed area;
intercepting a target area corresponding to a to-be-processed area and a target transition area of the visible light image in the infrared image to obtain an infrared sub-image;
embedding the infrared sub-image into the visible light image according to the matching result of the characteristic points of the target transition region in the visible light image and the characteristic points in the infrared sub-image, and carrying out characteristic fusion on the region to be processed of the visible light image after the infrared sub-image is embedded to obtain a target visible light image;
inputting the target visible light image into a preset image recognition model, and enabling the image recognition model to output an image type corresponding to the target visible light image, wherein the image recognition model is obtained based on convolutional neural network training.
The computer readable storage medium may include a storage program area and a storage data area, wherein the storage program area may store an operating system, at least one application program required for a function; the storage data area may store data created from the use of an image recognition system that combines visible light images and infrared images for analysis, and the like. In addition, the computer-readable storage medium may include high-speed random access memory, and may also include memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the computer readable storage medium optionally includes memory remotely located relative to the processor, which may be connected to the image recognition system for combined analysis of the visible light image and the infrared image via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
Fig. 3 is a schematic structural diagram of an electronic device according to an embodiment of the present invention, as shown in fig. 3, where the device includes: a processor 310 and a memory 320. The electronic device may further include: an input device 330 and an output device 340. The processor 310, memory 320, input device 330, and output device 340 may be connected by a bus or other means, for example in fig. 3. Memory 320 is the computer-readable storage medium described above. The processor 310 executes various functional applications of the server and data processing, i.e., an image recognition method for implementing the above-described method embodiment visible light image and infrared image combination analysis, by running nonvolatile software programs, instructions, and modules stored in the memory 320. The input device 330 may receive input numeric or character information and generate key signal inputs related to user settings and function controls of the image recognition system for analysis in combination with the visible light image and the infrared image. The output device 340 may include a display device such as a display screen.
The electronic equipment can execute the method provided by the embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method. Technical details not described in detail in this embodiment may be found in the methods provided in the embodiments of the present invention.
As an embodiment, the electronic device is applied to an image recognition system for combining and analyzing a visible light image and an infrared image, and is used for a client, and includes: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executable by the at least one processor to enable the at least one processor to:
acquiring a to-be-processed area of the visible light image, sliding the to-be-processed area on the edge of the to-be-processed area based on a preset sliding window, and selecting a target transition area connected with the to-be-processed area;
intercepting a target area corresponding to a to-be-processed area and a target transition area of the visible light image in the infrared image to obtain an infrared sub-image;
embedding the infrared sub-image into the visible light image according to the matching result of the characteristic points of the target transition region in the visible light image and the characteristic points in the infrared sub-image, and carrying out characteristic fusion on the region to be processed of the visible light image after the infrared sub-image is embedded to obtain a target visible light image;
inputting the target visible light image into a preset image recognition model, and enabling the image recognition model to output an image type corresponding to the target visible light image, wherein the image recognition model is obtained based on convolutional neural network training.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on such understanding, the foregoing technical solutions may be embodied essentially or in part in the form of a software product, which may be stored in a computer-readable storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to perform the various embodiments or methods of some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (6)
1. An image recognition method for combining and analyzing a visible light image and an infrared image, comprising the steps of:
the method comprises the steps of obtaining a to-be-processed area of the visible light image, sliding the edge of the to-be-processed area based on a preset sliding window, and selecting a target transition area connected with the to-be-processed area, wherein the sliding the edge of the to-be-processed area based on the preset sliding window, and selecting the target transition area connected with the to-be-processed area comprises the following steps:
labeling the region to be processed based on a labeling frame, wherein the vertex of the labeling frame is positioned at the edge of the region to be processed;
sliding on the marking frame according to a preset sliding window, and selecting a first transition area connected with the area to be treated, wherein the first transition area is not overlapped with the area to be treated;
judging whether the difference value between the pixel value of each characteristic point in the first transition region and the maximum pixel value of the characteristic point in the region to be processed is larger than a preset threshold value or not;
if the difference value between the pixel value of a certain characteristic point in the first transition region and the maximum pixel value of the characteristic point in the region to be processed is larger than a preset threshold value, directly removing the preset region where the certain characteristic point is located to obtain a target transition region, wherein the preset region is a circle with a preset radius taking the certain characteristic point as a circle center;
intercepting a target area corresponding to a to-be-processed area and a target transition area of the visible light image in the infrared image to obtain an infrared sub-image;
embedding the infrared sub-image into the visible light image according to the matching result of the characteristic points of the target transition region in the visible light image and the characteristic points in the infrared sub-image, and carrying out characteristic fusion on the region to be processed of the visible light image after the infrared sub-image is embedded to obtain a target visible light image, wherein the obtaining the target visible light image comprises the following steps:
firstly, carrying out Gaussian smoothing on an input image I to obtain an image L, and then carrying out downsampling on the image L subjected to Gaussian smoothing by 2 times to obtain an image L1 serving as a first layer of a Gaussian pyramid;
repeating Gaussian smoothing and downsampling processes to construct a Gaussian pyramid containing higher-layer images, wherein the standard deviation of a first layer of the Gaussian pyramid is as followsThe standard deviation increases in order by k times, i.e. the standard deviation of the nth layer is +.>；
Registering the pyramid layers of the Gaussian pyramid according to the size proportion of the target area of the infrared sub-image and the area to be processed of the visible light image, so that pyramid scales corresponding to the target area and the area to be processed are aligned;
setting target weights of different pyramid layers according to the image information quantity and importance under different scales, and carrying out weighted fusion on pixels or features corresponding to a target area and an area to be processed in each pyramid layer according to the target weights;
reconstructing a final target visible light image according to the fusion result of each layer of the pyramid and the corresponding relation between different layers of the pyramid;
inputting the target visible light image into a preset image recognition model, and enabling the image recognition model to output an image type corresponding to the target visible light image, wherein the image recognition model is obtained based on convolutional neural network training.
2. The method for image recognition by combining analysis of a visible light image and an infrared image according to claim 1, wherein the capturing a target area of the infrared image corresponding to a to-be-processed area and a target transition area of the visible light image, to obtain an infrared sub-image, comprises:
acquiring a target size of the visible light image, and adjusting the size of the infrared image to be consistent with the target size;
and matching the characteristic points in the infrared image with the characteristic points of the target transition region in the visible light image based on the SIFT algorithm, and intercepting the infrared image according to a matching result to obtain an infrared sub-image.
3. The image recognition method of claim 1, wherein before inputting the target visible light image into a preset image recognition model, the image recognition model outputs an image type corresponding to the target visible light image, the method further comprises:
and performing iterative training on the convolutional neural network according to the visible light image sample and the infrared image sample to obtain an image recognition model, wherein the convolutional neural network comprises a convolutional layer, an activation layer, a pooling layer and a full-connection layer.
4. An image recognition system for combined analysis of a visible light image and an infrared image, comprising:
the acquisition module is configured to acquire a to-be-processed area of the visible light image, slide on the edge of the to-be-processed area based on a preset sliding window, and select a target transition area connected with the to-be-processed area, wherein the slide on the edge of the to-be-processed area based on the preset sliding window, and the selecting of the target transition area connected with the to-be-processed area comprises the following steps:
labeling the region to be processed based on a labeling frame, wherein the vertex of the labeling frame is positioned at the edge of the region to be processed;
sliding on the marking frame according to a preset sliding window, and selecting a first transition area connected with the area to be treated, wherein the first transition area is not overlapped with the area to be treated;
judging whether the difference value between the pixel value of each characteristic point in the first transition region and the maximum pixel value of the characteristic point in the region to be processed is larger than a preset threshold value or not;
if the difference value between the pixel value of a certain characteristic point in the first transition region and the maximum pixel value of the characteristic point in the region to be processed is larger than a preset threshold value, directly removing the preset region where the certain characteristic point is located to obtain a target transition region, wherein the preset region is a circle with a preset radius taking the certain characteristic point as a circle center;
the intercepting module is configured to intercept a target area corresponding to a to-be-processed area and a target transition area of the visible light image in the infrared image to obtain an infrared sub-image;
the fusion module is configured to embed the infrared sub-image into the visible light image according to a matching result of the characteristic point of the target transition region in the visible light image and the characteristic point in the infrared sub-image, and perform characteristic fusion on a region to be processed of the visible light image after the infrared sub-image is embedded, so as to obtain a target visible light image, wherein the obtaining the target visible light image comprises:
firstly, carrying out Gaussian smoothing on an input image I to obtain an image L, and then carrying out downsampling on the image L subjected to Gaussian smoothing by 2 times to obtain an image L1 serving as a first layer of a Gaussian pyramid;
repeating Gaussian smoothing and downsampling processes to construct a Gaussian pyramid containing higher-layer images, wherein the standard deviation of a first layer of the Gaussian pyramid is as followsThe standard deviation increases in order by k times, i.e. the standard deviation of the nth layer is +.>；
Registering the pyramid layers of the Gaussian pyramid according to the size proportion of the target area of the infrared sub-image and the area to be processed of the visible light image, so that pyramid scales corresponding to the target area and the area to be processed are aligned;
setting target weights of different pyramid layers according to the image information quantity and importance under different scales, and carrying out weighted fusion on pixels or features corresponding to a target area and an area to be processed in each pyramid layer according to the target weights;
reconstructing a final target visible light image according to the fusion result of each layer of the pyramid and the corresponding relation between different layers of the pyramid;
the output module is configured to input the target visible light image into a preset image recognition model, so that the image recognition model outputs an image type corresponding to the target visible light image, wherein the image recognition model is obtained based on convolutional neural network training.
5. An electronic device, comprising: at least one processor, and a memory communicatively coupled to the at least one processor, wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 3.
6. A computer readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the method of any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311515793.6A CN117237779B (en) | 2023-11-15 | 2023-11-15 | Image recognition method and system for visible light image and infrared image combined analysis |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311515793.6A CN117237779B (en) | 2023-11-15 | 2023-11-15 | Image recognition method and system for visible light image and infrared image combined analysis |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117237779A CN117237779A (en) | 2023-12-15 |
| CN117237779B true CN117237779B (en) | 2024-02-09 |
Family
ID=89093415
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311515793.6A Active CN117237779B (en) | 2023-11-15 | 2023-11-15 | Image recognition method and system for visible light image and infrared image combined analysis |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117237779B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107918748A (en) * | 2017-10-27 | 2018-04-17 | 南京理工大学 | A kind of multispectral two-dimension code recognition device and method |
| CN110213501A (en) * | 2019-06-25 | 2019-09-06 | 浙江大华技术股份有限公司 | A kind of grasp shoot method, device, electronic equipment and storage medium |
| CN113989336A (en) * | 2021-11-05 | 2022-01-28 | 上海热像科技股份有限公司 | Visible light image and infrared image registration method and device |
| CN114332702A (en) * | 2021-12-27 | 2022-04-12 | 浙江大华技术股份有限公司 | Target area detection method and device, storage medium and electronic equipment |
| CN114399450A (en) * | 2021-12-10 | 2022-04-26 | 浙江大华技术股份有限公司 | Image fusion method, target feature identification method and device and electronic equipment |
| CN116051444A (en) * | 2023-02-20 | 2023-05-02 | 长春理工大学 | Effective infrared and visible light image self-adaptive fusion method |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106780392B (en) * | 2016-12-27 | 2020-10-02 | 浙江大华技术股份有限公司 | Image fusion method and device |
| CN111080724B (en) * | 2019-12-17 | 2023-04-28 | 大连理工大学 | Fusion method of infrared light and visible light |
-
2023
- 2023-11-15 CN CN202311515793.6A patent/CN117237779B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107918748A (en) * | 2017-10-27 | 2018-04-17 | 南京理工大学 | A kind of multispectral two-dimension code recognition device and method |
| CN110213501A (en) * | 2019-06-25 | 2019-09-06 | 浙江大华技术股份有限公司 | A kind of grasp shoot method, device, electronic equipment and storage medium |
| CN113989336A (en) * | 2021-11-05 | 2022-01-28 | 上海热像科技股份有限公司 | Visible light image and infrared image registration method and device |
| CN114399450A (en) * | 2021-12-10 | 2022-04-26 | 浙江大华技术股份有限公司 | Image fusion method, target feature identification method and device and electronic equipment |
| CN114332702A (en) * | 2021-12-27 | 2022-04-12 | 浙江大华技术股份有限公司 | Target area detection method and device, storage medium and electronic equipment |
| CN116051444A (en) * | 2023-02-20 | 2023-05-02 | 长春理工大学 | Effective infrared and visible light image self-adaptive fusion method |
Non-Patent Citations (3)
| Title |
|---|
| FusionPID: A PID control system for the fusion of infrared and visible light images;Linlu Dong，Jun Wang;Measurement;第217卷;全文 * |
| 一种顾及区域特征差异的热红外与可见光图像多尺度融合方法;向天烛;高熔溶;闫利;徐振亮;;武汉大学学报(信息科学版)(第07期);全文 * |
| 基于NSST和区域生长的红外与可见光图像融合算法;罗杰;孔韦韦;刘睿;;中国科技论文(第14期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117237779A (en) | 2023-12-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Lv et al. | Attention guided low-light image enhancement with a large scale low-light simulation dataset | |
| US11055827B2 (en) | Image processing apparatus and method | |
| Ren et al. | Low-light image enhancement via a deep hybrid network | |
| Wan et al. | CoRRN: Cooperative reflection removal network | |
| Liu et al. | Joint contrast enhancement and exposure fusion for real-world image dehazing | |
| Tursun et al. | An objective deghosting quality metric for HDR images | |
| WO2022062346A1 (en) | Image enhancement method and apparatus, and electronic device | |
| CN108960404B (en) | Image-based crowd counting method and device | |
| Moriwaki et al. | Hybrid loss for learning single-image-based HDR reconstruction | |
| CN110276831B (en) | Method and device for constructing three-dimensional model, equipment and computer-readable storage medium | |
| CN116681636B (en) | Light infrared and visible light image fusion method based on convolutional neural network | |
| Steffens et al. | Cnn based image restoration: Adjusting ill-exposed srgb images in post-processing | |
| Fan et al. | Multiscale cross-connected dehazing network with scene depth fusion | |
| CN113505702A (en) | Pavement disease identification method and system based on double neural network optimization | |
| Wang et al. | Low-light image enhancement based on virtual exposure | |
| Choi et al. | A method for fast multi-exposure image fusion | |
| Ji et al. | A two-stage multi-focus image fusion framework robust to image mis-registration | |
| CN111179245B (en) | Image quality detection method, device, electronic equipment and storage medium | |
| CN117237779B (en) | Image recognition method and system for visible light image and infrared image combined analysis | |
| CN116167945A (en) | Image restoration method and device, electronic equipment and storage medium | |
| Wu et al. | Multifocus image fusion using random forest and hidden Markov model | |
| CN111080543B (en) | Image processing method and device, electronic equipment and computer readable storage medium | |
| Van Vo et al. | High dynamic range video synthesis using superpixel-based illuminance-invariant motion estimation | |
| CN114119376A (en) | Image processing method and device, electronic equipment and storage medium | |
| CN111008604A (en) | Prediction image acquisition method and device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |