CN116993729B - Night vision device imaging system and method based on second harmonic - Google Patents
Night vision device imaging system and method based on second harmonic Download PDFInfo
- Publication number
- CN116993729B CN116993729B CN202311248291.1A CN202311248291A CN116993729B CN 116993729 B CN116993729 B CN 116993729B CN 202311248291 A CN202311248291 A CN 202311248291A CN 116993729 B CN116993729 B CN 116993729B
- Authority
- CN
- China
- Prior art keywords
- imaging
- night vision
- image
- vision device
- feature point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003384 imaging method Methods 0.000 title claims abstract description 220
- 230000004297 night vision Effects 0.000 title claims abstract description 204
- 238000000034 method Methods 0.000 title abstract description 19
- 238000012545 processing Methods 0.000 claims abstract description 18
- 238000000605 extraction Methods 0.000 claims description 36
- 238000004364 calculation method Methods 0.000 claims description 21
- 230000003287 optical effect Effects 0.000 claims description 18
- 230000003014 reinforcing effect Effects 0.000 claims description 18
- 230000011218 segmentation Effects 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 14
- 238000003331 infrared imaging Methods 0.000 claims description 14
- 230000015654 memory Effects 0.000 claims description 14
- 238000003709 image segmentation Methods 0.000 claims description 11
- 238000005728 strengthening Methods 0.000 claims description 9
- 238000005286 illumination Methods 0.000 claims description 8
- 238000012544 monitoring process Methods 0.000 claims description 6
- 230000004927 fusion Effects 0.000 claims description 5
- KJONHKAYOJNZEC-UHFFFAOYSA-N nitrazepam Chemical compound C12=CC([N+](=O)[O-])=CC=C2NC(=O)CN=C1C1=CC=CC=C1 KJONHKAYOJNZEC-UHFFFAOYSA-N 0.000 claims description 3
- 230000002708 enhancing effect Effects 0.000 claims 1
- 238000010586 diagram Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 230000001276 controlling effect Effects 0.000 description 2
- 238000013500 data storage Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000004438 eyesight Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/12—Edge-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/761—Proximity, similarity or dissimilarity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Quality & Reliability (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Studio Devices (AREA)
Abstract
The invention discloses a night vision device imaging system and a method based on second harmonic, which belong to the field of image data processing.
Description
Technical Field
The invention belongs to the technical field of image data processing, and particularly relates to a night vision device imaging system and method based on second harmonic waves.
Background
The infrared vision device can help people observe, search, aim and drive vehicles at night, although people find infrared rays very early, the infrared remote sensing technology is limited by infrared components, the development of the infrared remote sensing technology is very slow, and the infrared night vision device is a military night vision device utilizing the photoelectric conversion technology and is divided into an active type and a passive type: the infrared searchlight irradiates the target and receives the reflected infrared radiation to form an image, but the existing night vision device uses infrared imaging to form an image, so that the imaging is fuzzy, and the enemy cannot be clearly distinguished, thus the misjudgment on a battlefield is extremely easy to cause, and the problems in the prior art are all caused;
for example, in chinese patent publication No. CN111458890B, a true color dual-light night vision device system is disclosed, which includes a front-end common-path optical device, a back-end optical device, where the back-end optical device includes an infrared imaging unit, a color image imaging unit, and an image fusion unit; the light received by the front-end light path optical device enters an infrared imaging unit through an infrared light splitting sheet; after passing through the rear group of visible light objective lens, the light rays passing through the infrared light splitting sheet are enhanced and formed into a color image after being split by adopting the red, green and blue light splitting sheets. The realization method of the true color dual-light night vision device comprises the following steps: collecting long-wave infrared and visible light image data; and processing the image data, and fusing to obtain a target image. The true color double-light night vision system and the realization method inherit the advantages of fusion of long-wave infrared and visible light gray level images, and take account of the imaging advantages of visible light wave bands on textures and colors of sceneries and the imaging advantages of long-wave infrared on target contrast, so that the night vision can perform imaging with high contrast, texture definition and high color rendition at night;
meanwhile, for example, in the Chinese patent with the publication number of CN102752626B, a method for adjusting the voltage of an image tube MCP to improve the imaging quality of a night vision device is provided, so that the problem of overexposure when an ultraviolet near infrared digital camera shoots a fingerprint on a white background is solved. The first step: two wires corresponding to the voltages of the anode and the cathode of the MCP in four wires connected with the high-voltage power supply box are externally connected with the wires for voltage measurement; and a second step of: the illumination condition is fixed, and the MCP initial voltage value of the image tube is measured in a darkroom or a darkbox and recorded; and a third step of: adjusting a potentiometer knob corresponding to the high-voltage power supply box, and measuring the MCP voltage value adjusted by the knob in real time; adjusting the voltage of the image tube to a reference voltage value; fourth step: after the night vision device is installed, whether the camera works normally or not is judged according to the whole night vision device photographing test after the voltage is regulated, if the night vision device photographing is too slow in response, the night vision device is regarded as abnormal working, the MCP wires are connected with the external connection again, and the potentiometer knob is pulled back clockwise.
The problems proposed in the background art exist in the above patents: when the existing night vision device is used for imaging, the imaging is fuzzy due to the fact that infrared imaging is relied on, and then the enemy cannot be clearly distinguished, misjudgment on a battlefield is extremely easy to occur, and in order to solve the problems, the application designs a night vision device imaging system and method based on second harmonic waves.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides a night vision device imaging system and a method based on second harmonic, which are characterized in that a monitoring image is subjected to harmonic imaging through a night vision device, the imaging image of the night vision device is acquired, then the imaging range of the night vision device is acquired, the imaging image of the night vision device is acquired by using a camera, the image of the night vision device is segmented, the segmented image is subjected to characteristic point acquisition, the imaging image of the natural light is segmented, the feature points of the imaging segmented image of the night vision device are extracted, the feature points of the imaging segmented image of the night vision device are led into an enhanced feature point extraction strategy to extract the enhanced feature points, the data of the feature points of the imaging segmented image of the natural light corresponding to the enhanced feature points are led into the enhanced strategy, and the enhanced imaging image of the night vision device is substituted into the imaging strategy to acquire the secondary imaging picture, so that the outline of the object needing attention is clear, and the object to be focused is convenient to clearly distinguish in a battlefield, and the situation is effectively avoided.
In order to achieve the above purpose, the present invention provides the following technical solutions:
a night vision device imaging method based on second harmonic comprises the following specific steps:
s1, carrying out harmonic imaging on a monitoring image by a night vision device, and collecting an imaging image of the night vision device;
s2, acquiring an imaging range of the night vision device, and acquiring an imaging range image of the night vision device under natural illumination by using a camera to acquire a natural light imaging image;
s3, image segmentation is carried out on the imaging image of the night vision device to obtain imaging segmented image of the night vision device, characteristic point acquisition is carried out on the imaging segmented image of the night vision device, the imaging image under natural light is segmented to obtain imaging segmented image of the natural light, and characteristic points are extracted from the imaging segmented image of the night vision device;
s4, feature points in the imaging segmentation image of the night vision device are led into a strengthening feature point extraction strategy to extract strengthening feature points, and data of the feature points of the natural light imaging segmentation image at the corresponding positions of the strengthening feature points are extracted;
s5, importing the data of the characteristic points of the night vision device into an enhancement strategy to enhance the characteristic points, and substituting the enhanced imaging image of the night vision device into the imaging strategy to acquire a secondary imaging picture.
Specifically, the step S1 includes the following specific steps:
the S1 comprises the following specific steps:
s11, acquiring infrared image data by a front-end acquisition optical module of the night vision device to obtain an infrared imaging image;
s12, enabling the received light to enter an infrared harmonic imaging unit through an infrared light splitting sheet by a rear-end optical module of the night vision device to obtain a color image;
s13, the image fusion unit fuses the color image and the infrared imaging image to obtain an imaging image of the night vision device, and transmits the imaging image of the night vision device.
Specifically, the specific steps of S3 are as follows:
s31, acquiring an imaging image of the night vision device, carrying out contour division on the imaging image of the night vision device according to color chromaticity, carrying out image segmentation on the imaging contour, and extracting the contour image;
s32, importing the extracted contour image into a contour edge feature point unit to extract feature point data of the contour edge to form a night vision feature point set, wherein the night vision feature point data is expressed asWherein->Expressed as corresponding->Profile(s)>Indicate->Feature points;
s33, dividing the outline of an imaging image under natural light according to color chromaticity, dividing the imaging outline, and extracting the natural light outline image;
s34, importing the extracted natural light contour image into a contour edge feature point unit to extract feature point data of the contour edge to form a natural light feature point set, wherein the natural light feature point data is expressed asWherein->Represented as corresponding firstProfile(s)>Indicate->And feature points.
Specifically, the specific steps of the enhanced feature point extraction strategy of S4 are as follows:
s41, collecting feature points and contours in the imaging segmentation image, extracting the contours of an object to be focused, placing the object to be focused in the same coordinate system as the imaging segmentation image, extracting the feature points of the contours to be focused, and setting the feature points as a focused feature point set;
s42, importing feature point data of the concerned feature point set data and feature point data of one contour edge in the night vision feature point set into a similarity calculation formula to calculate similarity, wherein the similarity calculation formula is a cosine similarity calculation formula or a Pelson correlation coefficient similarity calculation formula;
s43, extracting the similarity between the feature point data of the contour edge in each night vision feature point set and the attention feature point set data, extracting the feature point data of the contour edge in the night vision feature point set with the similarity larger than the set similarity threshold, setting the feature points of the contour images as enhanced feature points, and extracting the feature point data of the natural light imaging segmentation image at the corresponding positions of the enhanced feature points.
Specifically, the specific steps of S5 are as follows:
s51, extracting data of reinforcing feature points of the night vision device and data of natural light feature points at corresponding positions, overlapping a bottom-layer picture with the data of the reinforcing feature points of the night vision device as an upper-layer picture, and then guiding the overlapped picture into a reinforcing strategy to reinforce the feature points;
s52, using the enhanced night vision imaging image as a secondary imaging picture.
Specifically, the specific steps of the enhancement strategy in S51 are as follows:
s511, extracting a superimposed picture, capturing infrared light irradiation intensity of the positions of the reinforcing feature points, improving the intensity of the positions of the reinforcing feature points by a specified intensity set value, and reducing the intensity of the positions of the reinforcing feature points by the specified intensity set value;
s512, acquiring an imaging image of the night vision device after the operation of S511, and obtaining a final imaging image.
The night vision device imaging system based on the second harmonic is realized based on the night vision device imaging method based on the second harmonic, and comprises a control module, a data acquisition module, an image processing module, a characteristic point extraction module, an image enhancement module and a secondary imaging module, wherein the control module is used for controlling the operation of the data acquisition module, the image processing module, the characteristic point extraction module, the image enhancement module and the secondary imaging module, the data acquisition module is used for acquiring real-time night vision device and camera shooting images, the image processing module is used for carrying out image segmentation on the night vision device imaging images and natural light imaging images, the characteristic point extraction module is used for extracting characteristic points of the contour of the segmented images, the image enhancement module is used for leading the data of the characteristic points of the night vision device into an enhancement strategy to enhance the characteristic points, and the secondary imaging module is used for substituting the enhanced night vision device imaging images into the imaging strategy to carry out the acquisition of secondary imaging pictures.
Specifically, the data acquisition module comprises a night vision device acquisition unit and a camera acquisition unit, wherein the night vision device acquisition unit is used for acquiring infrared image data by using a front-end acquisition optical module of the night vision device, a rear-end optical module of the night vision device enters received light into an infrared harmonic imaging unit through an infrared light splitting sheet, the light after passing through the infrared light splitting sheet passes through a rear group of visible light objective lenses, is enhanced and forms a color image after being split by adopting the infrared light splitting sheet, the infrared imaging image and the color image are fused, the imaging image of the night vision device is obtained, the imaging image of the night vision device is transmitted, and the camera acquisition unit is used for acquiring the image in the imaging range of the night vision device under natural illumination by using a camera; the image processing module comprises a contour dividing unit and a contour extraction unit, wherein the contour dividing unit is used for collecting an imaging image of the night vision device, dividing the imaging image of the night vision device into contours according to color chromaticity, dividing the imaging contours into images, dividing the image collected by the camera according to the dividing contours of the imaging image of the night vision device, and the contour extraction unit is used for extracting the image contours of the night vision device and the camera after division; the feature point extraction module comprises a contour feature point extraction module and a similarity calculation module, wherein the contour feature point extraction module is used for extracting feature points of image contours of the night vision device and the camera, and the similarity calculation module is used for importing feature point data of one contour edge in the focused feature point set and the focused feature point set into a similarity calculation formula to calculate similarity.
An electronic device, comprising: a processor and a memory, wherein the memory stores a computer program for the processor to call;
the processor executes the night vision device imaging method based on the second harmonic by calling the computer program stored in the memory.
A computer readable storage medium storing instructions that when executed on a computer cause the computer to perform a second harmonic based night vision imaging method as described above.
Compared with the prior art, the invention has the beneficial effects that:
according to the invention, the monitoring image is subjected to harmonic imaging through the night vision device, the imaging image of the night vision device is acquired, then the imaging range of the night vision device is acquired, the imaging image of the night vision device is acquired by using the camera, the imaging image of the night vision device is subjected to image segmentation, the segmented image of the night vision device is subjected to characteristic point acquisition, the imaging image of the natural light is segmented, the characteristic points of the imaging segmented image of the night vision device are led into an enhanced characteristic point extraction strategy to extract enhanced characteristic points, the data of the characteristic points of the natural light imaging segmented image at the corresponding positions of the enhanced characteristic points are led into the enhanced strategy to enhance the characteristic points, and the enhanced imaging image of the night vision device is substituted into the imaging strategy to acquire secondary imaging pictures.
Drawings
FIG. 1 is a schematic flow chart of a night vision device imaging method based on second harmonic;
FIG. 2 is a schematic diagram showing a specific flow of steps of a second harmonic-based night vision device imaging method S1 of the present invention;
FIG. 3 is a schematic diagram showing a specific flow of the step S3 of the night vision device imaging method based on the second harmonic;
FIG. 4 is a schematic diagram of a specific architecture of a second harmonic based night vision imaging system of the present invention;
FIG. 5 is a schematic diagram of a second harmonic based night vision imaging system data acquisition module architecture of the present invention;
FIG. 6 is a schematic diagram of an image processing module architecture of the second harmonic based night vision imaging system of the present invention;
fig. 7 is a schematic diagram of a feature point extraction module of the night vision device imaging system based on the second harmonic.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments.
Example 1
Referring to fig. 1-3, an embodiment of the present invention is provided: a night vision device imaging method based on second harmonic comprises the following specific steps:
s1, carrying out harmonic imaging on a monitoring image by a night vision device, and collecting an imaging image of the night vision device;
here, S1 includes the following specific steps:
s11, acquiring infrared image data by a front-end acquisition optical module of the night vision device to obtain an infrared imaging image;
s12, enabling the received light to enter an infrared harmonic imaging unit through an infrared light splitting sheet by a rear-end optical module of the night vision device to obtain a color image;
s13, an image fusion unit fuses the color image and the infrared imaging image to obtain an imaging image of the night vision device, and transmits the imaging image of the night vision device;
thus, the image data acquired by the night vision device are acquired to obtain infrared imaging image data;
s2, acquiring an imaging range of the night vision device, and acquiring an imaging range image of the night vision device under natural illumination by using a camera to acquire a natural light imaging image;
extracting an imaging range image of the night vision device under natural illumination by using the camera so as to obtain a bottom layer image;
s3, image segmentation is carried out on the imaging image of the night vision device to obtain imaging segmented image of the night vision device, characteristic point acquisition is carried out on the imaging segmented image of the night vision device, the imaging image under natural light is segmented to obtain imaging segmented image of the natural light, and characteristic points are extracted from the imaging segmented image of the night vision device;
the specific steps of S3 are as follows:
s31, acquiring an imaging image of the night vision device, carrying out contour division on the imaging image of the night vision device according to color chromaticity, carrying out image segmentation on the imaging contour, and extracting the contour image;
s32, importing the extracted contour image into a contour edge feature point unit to extract feature point data of the contour edge to form a night vision feature point set, wherein the night vision feature point data is expressed asWherein->Expressed as corresponding->Profile(s)>Indicate->Feature points;
s33, dividing the outline of an imaging image under natural light according to color chromaticity, dividing the imaging outline, and extracting the natural light outline image;
s34, importing the extracted natural light contour image into a contour edge feature point unit to extract feature point data of the contour edge to form a natural light feature point set, wherein the natural light feature point data is expressed asWherein->Represented as corresponding firstProfile(s)>Indicate->Feature points;
s4, feature points in the imaging segmentation image of the night vision device are led into a strengthening feature point extraction strategy to extract strengthening feature points, and data of the feature points of the natural light imaging segmentation image at the corresponding positions of the strengthening feature points are extracted;
the specific steps of the enhanced feature point extraction strategy of S4 are as follows:
s41, collecting feature points and contours in the imaging segmentation image, extracting the contours of an object to be focused, placing the object to be focused in the same coordinate system as the imaging segmentation image, extracting the feature points of the contours to be focused, and setting the feature points as a focused feature point set;
s42, importing the feature point data of one contour edge in the attention feature point set data and the night vision feature point set into a similarity calculation formula to calculate similarity, wherein the similarity calculation formula is a cosine similarity calculation formula or a Pelson correlation coefficient similarity calculation formula;
s43, extracting the similarity between the feature point data of the contour edge in each night vision feature point set and the attention feature point set data, extracting the feature point data of the contour edge in the night vision feature point set with the similarity larger than a set similarity threshold, setting the feature points of the contour images as reinforced feature points, extracting the data of the feature points of the natural light imaging segmentation image at the corresponding positions of the reinforced feature points, wherein the similarity threshold is obtained in a mode of scoring and averaging by 500 experts of the person skilled in the art, and the similarity threshold is preferably 0.95;
s5, importing the data of the characteristic points of the night vision device into an enhancement strategy to enhance the characteristic points, substituting the enhanced imaging image of the night vision device into the imaging strategy to acquire a secondary imaging picture;
the specific steps of S5 are as follows:
s51, extracting data of reinforcing feature points of the night vision device and data of natural light feature points at corresponding positions, overlapping a bottom-layer picture with the data of the reinforcing feature points of the night vision device as an upper-layer picture, and then guiding the overlapped picture into a reinforcing strategy to reinforce the feature points;
s52, using the enhanced night vision imaging image as a secondary imaging picture.
The specific steps of the enhancement strategy in S51 are as follows:
s511, extracting a superimposed screen, capturing infrared light irradiation intensity of the positions of the reinforcing feature points, increasing the intensity of the positions of the reinforcing feature points by a specified intensity set value, and decreasing the intensity of the positions of the reinforcing feature points by a specified intensity set value, wherein the intensity set value is set according to a specific use scene of a user;
s512, acquiring an imaging image of the night vision device after the operation of S511, and obtaining a final imaging image.
The method comprises the steps of carrying out harmonic imaging on a monitoring image through a night vision device, collecting an imaging image of the night vision device, then acquiring an imaging range of the night vision device, carrying out image segmentation on the imaging image of the night vision device by using a camera, carrying out characteristic point collection on the segmented image, segmenting the imaging image of the natural light, carrying out characteristic point extraction on the imaging segmented image of the night vision device, leading the characteristic points in the imaging segmented image of the night vision device into an enhanced characteristic point extraction strategy to carry out enhanced characteristic point extraction, extracting the data of the characteristic points of the natural light imaging segmented image at the corresponding positions of the enhanced characteristic points, leading the data of the characteristic points of the night vision device into the enhanced strategy to carry out characteristic point enhancement, substituting the enhanced imaging image of the night vision device into the imaging strategy to carry out secondary imaging picture acquisition, so that the outline of an object needing to be concerned is clear, thereby being convenient for carrying out clear resolution of an enemy and further effectively avoiding misjudgment on a battlefield.
Example 2
As shown in fig. 4-7, a second harmonic-based night vision device imaging system is implemented based on the second harmonic-based night vision device imaging method, which comprises a control module, a data acquisition module, an image processing module, a feature point extraction module, an image enhancement module and a secondary imaging module, wherein the control module is used for controlling the operation of the data acquisition module, the image processing module, the feature point extraction module, the image enhancement module and the secondary imaging module, the data acquisition module is used for acquiring real-time night vision device and camera shooting images, the image processing module is used for carrying out image segmentation on the night vision device imaging images and natural light imaging images, the feature point extraction module is used for extracting feature points of the contours of the segmented images, the image enhancement module is used for introducing the data of the feature points of the night vision device into an enhancement strategy to enhance the feature points, and the secondary imaging module is used for substituting the enhanced night vision device imaging images into the imaging strategy to acquire secondary imaging pictures;
in this embodiment, the data acquisition module includes a night vision device acquisition unit and a camera acquisition unit, the night vision device acquisition unit is configured to acquire infrared image data by using a front end acquisition optical module of the night vision device, the rear end optical module of the night vision device enters received light into an infrared harmonic imaging unit through an infrared beam splitter, after passing through a rear group of visible light objective lenses, the light after passing through the infrared beam splitter is enhanced by adopting red, green and blue beam splitters to form a color image, the infrared imaging image and the color image are fused to obtain an imaging image of the night vision device, and the imaging image of the night vision device is transmitted, and the camera acquisition unit is configured to acquire an image in an imaging range of the night vision device under natural illumination by using the camera; the image processing module comprises a contour dividing unit and a contour extraction unit, the contour dividing unit is used for collecting imaging images of the night vision device, dividing the imaging images of the night vision device into contours according to color chromaticity, dividing the imaging contours into images, dividing the images collected by the camera according to the dividing contours of the imaging images of the night vision device, and the contour extraction unit is used for extracting the contours of the images of the night vision device and the camera after division; the feature point extraction module comprises a contour feature point extraction module and a similarity calculation module, wherein the contour feature point extraction module is used for extracting feature points of image contours of the night vision device and the camera, and the similarity calculation module is used for importing feature point data of one contour edge in the focused feature point set and the feature point data of one contour edge in the night vision feature point set into a similarity calculation formula to calculate similarity.
Example 3
The present embodiment provides an electronic device including: a processor and a memory, wherein the memory stores a computer program for the processor to call;
the processor executes a second harmonic based night vision device imaging method as described above by invoking a computer program stored in the memory.
The electronic device may be configured or configured differently to generate a larger difference, and may include one or more processors (Central Processing Units, CPU) and one or more memories, where at least one computer program is stored in the memories, and the computer program is loaded and executed by the processors to implement a second harmonic-based night vision device imaging method provided by the above method embodiment. The electronic device can also include other components for implementing the functions of the device, for example, the electronic device can also have wired or wireless network interfaces, input-output interfaces, and the like, for inputting and outputting data. The present embodiment is not described herein.
Example 4
The present embodiment proposes a computer-readable storage medium having stored thereon an erasable computer program;
the computer program, when run on a computer device, causes the computer device to perform a second harmonic based night vision imaging method as described above.
For example, the computer readable storage medium can be Read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), compact disk Read-Only Memory (Compact Disc Read-Only Memory, CD-ROM), magnetic tape, floppy disk, optical data storage device, etc.
It should be understood that, in various embodiments of the present application, the sequence numbers of the foregoing processes do not mean the order of execution, and the order of execution of the processes should be determined by the functions and internal logic thereof, and should not constitute any limitation on the implementation process of the embodiments of the present application.
It should be understood that determining B from a does not mean determining B from a alone, but can also determine B from a and/or other information.
The above embodiments may be implemented in whole or in part by software, hardware, firmware, or any other combination. When implemented in software, the above-described embodiments may be implemented in whole or in part in the form of a computer program product. The computer program product comprises one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, the processes or functions in accordance with embodiments of the present invention are produced in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another, for example, by way of wired or/and wireless networks from one website site, computer, server, or data center to another. Computer readable storage media can be any available media that can be accessed by a computer or data storage devices, such as servers, data centers, etc. that contain one or more collections of available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. The semiconductor medium may be a solid state disk.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, and are not repeated herein.
In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely one, and there may be additional divisions in actual implementation, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.
In the description of the present specification, the descriptions of the terms "one embodiment," "example," "specific example," and the like, mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
The preferred embodiments of the invention disclosed above are intended only to assist in the explanation of the invention. The preferred embodiments are not intended to be exhaustive or to limit the invention to the precise form disclosed. Obviously, many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the invention and the practical application, to thereby enable others skilled in the art to best understand and utilize the invention. The invention is limited only by the claims and the full scope and equivalents thereof.
Claims (12)
1. The night vision device imaging method based on the second harmonic is characterized by comprising the following specific steps of:
s1, carrying out harmonic imaging on a monitoring image by a night vision device, and collecting an imaging image of the night vision device;
s2, acquiring an imaging range of the night vision device, and acquiring an imaging range image of the night vision device under natural illumination by using a camera to acquire a natural light imaging image;
s3, image segmentation is carried out on the imaging image of the night vision device to obtain imaging segmented image of the night vision device, characteristic point acquisition is carried out on the imaging segmented image of the night vision device, the imaging image under natural light is segmented to obtain imaging segmented image of the natural light, and characteristic points are extracted from the imaging segmented image of the night vision device;
s4, feature points in the imaging segmentation image of the night vision device are led into a strengthening feature point extraction strategy to extract strengthening feature points, and data of the feature points of the natural light imaging segmentation image at the corresponding positions of the strengthening feature points are extracted;
s5, importing the data of the characteristic points of the night vision device into an enhancement strategy to enhance the characteristic points, and substituting the enhanced imaging image of the night vision device into the imaging strategy to acquire a secondary imaging picture.
2. The second harmonic based night vision imaging method of claim 1, wherein S1 comprises the specific steps of:
s11, acquiring infrared image data by a front-end acquisition optical module of the night vision device to obtain an infrared imaging image;
s12, enabling the received light to enter an infrared harmonic imaging unit through an infrared light splitting sheet by a rear-end optical module of the night vision device to obtain a color image;
s13, the image fusion unit fuses the color image and the infrared imaging image to obtain an imaging image of the night vision device, and transmits the imaging image of the night vision device.
3. The second harmonic based night vision device imaging method as claimed in claim 2, wherein the specific steps of S3 are as follows:
s31, acquiring an imaging image of the night vision device, carrying out contour division on the imaging image of the night vision device according to color chromaticity, carrying out image segmentation on the imaging contour, and extracting the contour image;
s32, importing the extracted contour image into a contour edge feature point unit to extract feature point data of the contour edge to form a night vision feature point set, wherein the night vision feature point data is expressed asWherein->Expressed as corresponding->Profile(s)>Indicate->Feature points;
s33, dividing the outline of an imaging image under natural light according to color chromaticity, dividing the imaging outline, and extracting the natural light outline image;
s34, importing the extracted natural light contour image into a contour edge feature point unit to extract feature point data of the contour edge to form a natural light feature point set, wherein the natural light feature point data is expressed asWherein->Expressed as corresponding->Profile(s)>Indicate->And feature points.
4. The second harmonic based night vision device imaging method as claimed in claim 3, wherein the specific steps of the enhanced feature point extraction strategy of S4 are as follows:
s41, collecting feature points and contours in the imaging segmentation image, extracting the contours of an object to be focused, placing the object to be focused in the same coordinate system as the imaging segmentation image, extracting the feature points of the contours to be focused, and setting the feature points as a focused feature point set;
s42, importing feature point data of the concerned feature point set data and feature point data of one contour edge in the night vision feature point set into a similarity calculation formula to calculate similarity, wherein the similarity calculation formula is a cosine similarity calculation formula or a Pelson correlation coefficient similarity calculation formula;
s43, extracting the similarity between the feature point data of the contour edge in each night vision feature point set and the attention feature point set data, extracting the feature point data of the contour edge in the night vision feature point set with the similarity larger than the set similarity threshold, setting the feature points of the contour images as enhanced feature points, and extracting the feature point data of the natural light imaging segmentation image at the corresponding positions of the enhanced feature points.
5. The second harmonic based night vision device imaging method of claim 4, wherein the specific step of S5 is as follows:
s51, extracting data of reinforcing feature points of the night vision device and data of natural light feature points at corresponding positions, overlapping a bottom-layer picture with the data of the reinforcing feature points of the night vision device as an upper-layer picture, and then guiding the overlapped picture into a reinforcing strategy to reinforce the feature points;
s52, using the enhanced night vision imaging image as a secondary imaging picture.
6. The second harmonic based night vision device imaging method as claimed in claim 5, wherein the enhancing strategy in S51 specifically comprises the steps of:
s511, extracting a superimposed picture, capturing infrared light irradiation intensity of the positions of the reinforcing feature points, improving the intensity of the positions of the reinforcing feature points by a specified intensity set value, and reducing the intensity of the positions of the reinforcing feature points by the specified intensity set value;
s512, acquiring an imaging image of the night vision device after the operation of S511, and obtaining a final imaging image.
7. A second harmonic-based night vision device imaging system, which is realized based on the second harmonic-based night vision device imaging method according to any one of claims 1-6, and is characterized by comprising a control module, a data acquisition module, an image processing module, a feature point extraction module, an image enhancement module and a secondary imaging module, wherein the control module is used for controlling the operation of the data acquisition module, the image processing module, the feature point extraction module, the image enhancement module and the secondary imaging module, the data acquisition module is used for acquiring real-time night vision device and camera shooting images, the image processing module is used for carrying out image segmentation on the night vision device imaging images and natural light imaging images, the feature point extraction module is used for carrying out feature point extraction on the contour of the segmented images, the image enhancement module is used for introducing the data of the feature points of the night vision device into an enhancement strategy for carrying out feature point enhancement, and the secondary imaging module is used for substituting the enhanced night vision device imaging images into the imaging strategy for obtaining secondary imaging pictures.
8. The night vision device imaging system based on the second harmonic wave according to claim 7, wherein the data acquisition module comprises a night vision device acquisition unit and a camera acquisition unit, the night vision device acquisition unit is used for acquiring infrared image data by using a front end acquisition optical module of the night vision device, the rear end optical module of the night vision device is used for enabling received light to enter the infrared harmonic imaging unit through an infrared beam splitter, after passing through a rear group of visible light objective lenses, the light after passing through the infrared beam splitter is enhanced and formed into a color image by adopting the infrared beam splitter, the green beam splitter and the blue beam splitter, the infrared imaging image and the color image are fused, the night vision device imaging image is obtained, the night vision device imaging image is transmitted, and the camera acquisition unit is used for acquiring the image in the imaging range of the night vision device under natural illumination by using the camera.
9. The second harmonic based night vision device imaging system of claim 8, wherein the image processing module comprises a contour dividing unit and a contour extraction unit, the contour dividing unit is used for collecting night vision device imaging images, dividing the night vision device imaging images into contours according to color chromaticity, dividing the imaging contours into images, dividing the camera collected images according to the contour of the night vision device imaging images, and the contour extraction unit is used for extracting the divided night vision device and camera image contours.
10. The second harmonic based night vision device imaging system as recited in claim 9, wherein the feature point extraction module comprises a contour feature point extraction module for extracting feature points of the night vision device and camera image contours, and a similarity calculation module for importing feature point data of one contour edge of the feature point set of interest and the feature point data of one contour edge of the night vision feature point set into a similarity calculation formula for similarity calculation.
11. An electronic device, comprising: a processor and a memory, wherein the memory stores a computer program for the processor to call;
the processor performs a second harmonic based night vision imaging method as claimed in any one of claims 1-6 by invoking a computer program stored in the memory.
12. A computer-readable storage medium, characterized by: instructions stored thereon which, when executed on a computer, cause the computer to perform a second harmonic based night vision imaging method as claimed in any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311248291.1A CN116993729B (en) | 2023-09-26 | 2023-09-26 | Night vision device imaging system and method based on second harmonic |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311248291.1A CN116993729B (en) | 2023-09-26 | 2023-09-26 | Night vision device imaging system and method based on second harmonic |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116993729A CN116993729A (en) | 2023-11-03 |
| CN116993729B true CN116993729B (en) | 2024-03-29 |
Family
ID=88521731
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311248291.1A Active CN116993729B (en) | 2023-09-26 | 2023-09-26 | Night vision device imaging system and method based on second harmonic |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116993729B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102148999A (en) * | 2011-01-10 | 2011-08-10 | 南京理工大学 | Wide-waveband large-span tuned filter single-channel colorful low-level-light night vision method and device |
| CN105590305A (en) * | 2016-03-04 | 2016-05-18 | 温健阳 | Color night vision system based on infrared and low light level image fusion technology |
| CN106023129A (en) * | 2016-05-26 | 2016-10-12 | 西安工业大学 | Infrared and visible light image fused automobile anti-blooming video image processing method |
| CN106454216A (en) * | 2016-11-02 | 2017-02-22 | 南京理工大学 | Night driving system based on uncooled infrared and low-light-level fusion |
| CN111458890A (en) * | 2020-04-12 | 2020-07-28 | 国科天成（北京）科技有限公司 | True-color double-light night vision device system and implementation method |
| CN113038029A (en) * | 2021-03-29 | 2021-06-25 | 深圳鑫景源科技股份有限公司 | Compensation method for enhancing night vision effect |
| CN115100556A (en) * | 2022-08-25 | 2022-09-23 | 北京亮亮视野科技有限公司 | Augmented reality method and device based on image segmentation and fusion and electronic equipment |
| CN116664460A (en) * | 2023-04-18 | 2023-08-29 | 中航国画（上海）激光显示科技有限公司 | Infrared night vision fusion method |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7307793B2 (en) * | 2004-07-02 | 2007-12-11 | Insight Technology, Inc. | Fusion night vision system |
| US9503623B2 (en) * | 2014-06-03 | 2016-11-22 | Applied Minds, Llc | Color night vision cameras, systems, and methods thereof |
-
2023
- 2023-09-26 CN CN202311248291.1A patent/CN116993729B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102148999A (en) * | 2011-01-10 | 2011-08-10 | 南京理工大学 | Wide-waveband large-span tuned filter single-channel colorful low-level-light night vision method and device |
| CN105590305A (en) * | 2016-03-04 | 2016-05-18 | 温健阳 | Color night vision system based on infrared and low light level image fusion technology |
| CN106023129A (en) * | 2016-05-26 | 2016-10-12 | 西安工业大学 | Infrared and visible light image fused automobile anti-blooming video image processing method |
| CN106454216A (en) * | 2016-11-02 | 2017-02-22 | 南京理工大学 | Night driving system based on uncooled infrared and low-light-level fusion |
| CN111458890A (en) * | 2020-04-12 | 2020-07-28 | 国科天成（北京）科技有限公司 | True-color double-light night vision device system and implementation method |
| CN113038029A (en) * | 2021-03-29 | 2021-06-25 | 深圳鑫景源科技股份有限公司 | Compensation method for enhancing night vision effect |
| CN115100556A (en) * | 2022-08-25 | 2022-09-23 | 北京亮亮视野科技有限公司 | Augmented reality method and device based on image segmentation and fusion and electronic equipment |
| CN116664460A (en) * | 2023-04-18 | 2023-08-29 | 中航国画（上海）激光显示科技有限公司 | Infrared night vision fusion method |
Non-Patent Citations (5)
| Title |
|---|
| Night Vision Anti-Halation Method Based on Infrared and Visible Video Fusion;Quanmin Guo等;《sensors》;20221002;1-17 * |
| Thermal Infrared Image Colorization for Nighttime Driving Scenes with Top-Down Guided Attention;Fuya Luo 等;《A MANUSCRIPT SUBMITTED TO IEEE TRANSACTIONS ON INTELLIGENT TRANSPOTATION SYSTEMS》;20170831;1-14 * |
| 基于场景识别的红外与微光图像彩色融合算法与应用研究;瞿哲;《中国优秀硕士学位论文全文数据库（信息科技辑）》;20191215;I138-530 * |
| 微光与红外图像实时融合关键技术研究;田思;《中国博士学位论文全文数据库（信息科技辑）》;20120515;I138-109 * |
| 真彩微光夜视图像融合算法;蒋云峰 等;《激光与光电子学进展》;20181113;第56卷(第8期);081008-1-7 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116993729A (en) | 2023-11-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10997696B2 (en) | Image processing method, apparatus and device | |
| US11443550B2 (en) | Face recognition monitoring system based on spectrum and multi-band fusion and recognition method using same | |
| WO2021022983A1 (en) | Image processing method and apparatus, electronic device and computer-readable storage medium | |
| JP2017005389A (en) | Image recognition device, image recognition method, and program | |
| CN110855889B (en) | Image processing method, image processing apparatus, image processing device, and storage medium | |
| CN110619318B (en) | Image processing method, microscope, system and medium based on artificial intelligence | |
| CN109712192A (en) | Camera module scaling method, device, electronic equipment and computer readable storage medium | |
| CN107481186A (en) | Image processing method, device, computer-readable recording medium and computer equipment | |
| van Zwanenberg et al. | Edge detection techniques for quantifying spatial imaging system performance and image quality | |
| CN115063331B (en) | Multi-scale block LBP operator-based ghost-free multi-exposure image fusion method | |
| CN109559353A (en) | Camera module scaling method, device, electronic equipment and computer readable storage medium | |
| CN110533709B (en) | Depth image acquisition method, device and system and image acquisition equipment | |
| CN109040746A (en) | Camera calibration method and apparatus, electronic equipment, computer readable storage medium | |
| CN113506275B (en) | Urban image processing method based on panorama | |
| CN110392211A (en) | Image processing method and device, electronic equipment, computer readable storage medium | |
| CN116993729B (en) | Night vision device imaging system and method based on second harmonic | |
| CN109951637A (en) | Safety monitoring probe analysis and processing method based on big data | |
| CN113762161A (en) | Intelligent obstacle monitoring method and system | |
| CN109034004A (en) | A kind of identity verification method and device based on recognition of face | |
| CN108289170A (en) | The camera arrangement and method of metering region can be detected | |
| CN112749610A (en) | Depth image, reference structured light image generation method and device and electronic equipment | |
| CN111970424A (en) | Light field camera shielding removing system and method based on micro-lens array synthetic aperture | |
| CN115100556B (en) | Augmented reality method and device based on image segmentation and fusion and electronic equipment | |
| CN114998980B (en) | Iris detection method and device, electronic equipment and storage medium | |
| CN114511781A (en) | Method, device and medium for identifying disguise personnel by carrying multispectral camera by unmanned aerial vehicle |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |