CN111856061A - Subminiature dual-light imaging system with multi-information fusion and light stream speed measurement functions - Google Patents
Subminiature dual-light imaging system with multi-information fusion and light stream speed measurement functions Download PDFInfo
- Publication number
- CN111856061A CN111856061A CN202010585768.5A CN202010585768A CN111856061A CN 111856061 A CN111856061 A CN 111856061A CN 202010585768 A CN202010585768 A CN 202010585768A CN 111856061 A CN111856061 A CN 111856061A
- Authority
- CN
- China
- Prior art keywords
- module
- image
- information fusion
- visible light
- infrared
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000004927 fusion Effects 0.000 title claims abstract description 63
- 238000003384 imaging method Methods 0.000 title claims abstract description 51
- 238000005259 measurement Methods 0.000 title claims abstract description 38
- 230000006870 function Effects 0.000 title claims abstract description 14
- 230000003287 optical effect Effects 0.000 claims abstract description 34
- 238000003331 infrared imaging Methods 0.000 claims abstract description 27
- 230000000007 visual effect Effects 0.000 claims abstract description 7
- 238000012634 optical imaging Methods 0.000 claims abstract description 4
- 238000000034 method Methods 0.000 claims description 11
- 230000006835 compression Effects 0.000 claims description 8
- 238000007906 compression Methods 0.000 claims description 8
- 238000000605 extraction Methods 0.000 claims description 8
- 230000008569 process Effects 0.000 claims description 4
- 238000000827 velocimetry Methods 0.000 claims 4
- 238000001514 detection method Methods 0.000 abstract description 13
- 238000005516 engineering process Methods 0.000 abstract description 9
- 238000012544 monitoring process Methods 0.000 abstract description 4
- 230000009471 action Effects 0.000 abstract description 2
- 230000008447 perception Effects 0.000 abstract description 2
- 238000012545 processing Methods 0.000 description 7
- 238000004458 analytical method Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 238000005286 illumination Methods 0.000 description 5
- 238000009529 body temperature measurement Methods 0.000 description 4
- 238000011161 development Methods 0.000 description 4
- 230000007613 environmental effect Effects 0.000 description 3
- 239000000284 extract Substances 0.000 description 3
- 239000000779 smoke Substances 0.000 description 3
- 238000000691 measurement method Methods 0.000 description 2
- 230000002265 prevention Effects 0.000 description 2
- 230000002411 adverse Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 239000008264 cloud Substances 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 239000000428 dust Substances 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 239000003897 fog Substances 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000007500 overflow downdraw method Methods 0.000 description 1
- 230000035515 penetration Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01P—MEASURING LINEAR OR ANGULAR SPEED, ACCELERATION, DECELERATION, OR SHOCK; INDICATING PRESENCE, ABSENCE, OR DIRECTION, OF MOVEMENT
- G01P3/00—Measuring linear or angular speed; Measuring differences of linear or angular speeds
- G01P3/64—Devices characterised by the determination of the time taken to traverse a fixed distance
- G01P3/68—Devices characterised by the determination of the time taken to traverse a fixed distance using optical means, i.e. using infrared, visible, or ultraviolet light
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01P—MEASURING LINEAR OR ANGULAR SPEED, ACCELERATION, DECELERATION, OR SHOCK; INDICATING PRESENCE, ABSENCE, OR DIRECTION, OF MOVEMENT
- G01P3/00—Measuring linear or angular speed; Measuring differences of linear or angular speeds
- G01P3/36—Devices characterised by the use of optical means, e.g. using infrared, visible, or ultraviolet light
- G01P3/38—Devices characterised by the use of optical means, e.g. using infrared, visible, or ultraviolet light using photographic means
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- General Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Electromagnetism (AREA)
- Power Engineering (AREA)
- Studio Devices (AREA)
Abstract
The invention belongs to the technical field of double-light imaging systems carried by unmanned aerial vehicles, and particularly relates to a subminiature double-light imaging system with multi-information fusion and optical flow speed measurement functions. The invention integrates the infrared imaging technology, the visible light imaging technology and the information fusion technology, innovatively adds the functions of optical flow speed measurement and face recognition in a double-optical system, realizes a subminiature double-optical imaging system, further expands the operation capability of the small reconnaissance type unmanned aerial vehicle in a complex environment, fully expands the visual field of a ground operation unit in a hidden environment, and enables action personnel to keep situation perception, threat detection and monitoring capability anytime and anywhere.
Description
Technical Field
The invention belongs to the technical field of double-light imaging systems carried by unmanned aerial vehicles, and particularly relates to a subminiature double-light imaging system with multi-information fusion and optical flow speed measurement functions and an optical flow speed measurement method.
Background
With the rapid development of electronic technology, the small unmanned aerial vehicle is widely applied to various fields such as security monitoring, search and rescue, police law enforcement, forest fire prevention, electric power line patrol, environmental protection and scientific research by means of high maneuvering performance, good environmental adaptability and economy. Technological development is so far, except the development of unmanned aerial vehicle platform itself, the continuous promotion of the continuous abundance and the performance of payload has also greatly expanded unmanned aerial vehicle's application scope. The unmanned aerial vehicle mainly acquires ground image information through imaging detection equipment to complete tasks such as monitoring and reconnaissance. In the present situation, unmanned airborne equipment is restricted by the technical characteristics of unmanned aerial vehicles, and has different requirements from other carrier equipment:
1. the unmanned aerial vehicle has small volume, and the size and the shape of a circuit board of airborne equipment of the unmanned aerial vehicle are limited;
2. because the load capacity of the unmanned aerial vehicle, especially a small unmanned aerial vehicle, is limited, the airborne mission load needs to meet the light weight requirement so as to prolong the endurance time of the unmanned aerial vehicle;
3. the power supply provided by the unmanned aerial vehicle is limited, and cannot provide a high-power supply as the ground equipment, so that the airborne task load is required to be low in power consumption.
Based on the characteristics, the unmanned aerial vehicle has strict requirements on the weight, the volume and the power consumption of the load, and the subminiature load is particularly important.
Due to the fact that different imaging detection devices have different factors such as imaging mechanisms, working wave bands and working environments, when different types of sensors image the same target or scene, the acquired target or scene information is different. The visible light sensor can obtain rich and fine visible light images, but the imaging quality of the visible light sensor is seriously influenced when the illumination condition is poor; the infrared detector has better target detection capability under poor illumination conditions such as night and the like, but the detail expression capability of the infrared image formed by the infrared detector is poor. Particularly for low-resolution infrared imaging equipment, the number of detector pixels is limited, the edge of an output infrared image is fuzzy, and the development of the infrared imaging equipment towards high-performance light weight is limited. In addition, if unmanned aerial vehicle appeared losing the star condition in the flight process, the gesture was dispersed easily, can directly lead to the flight mission to fail even the aircraft crash.
Disclosure of Invention
In order to solve the problems, the invention provides a subminiature dual-light imaging system with multi-information fusion and optical flow velocity measurement functions and an optical flow velocity measurement method.
The invention is realized in this way, provides a subminiature double-light imaging system with multi-information fusion and light stream speed measurement functions, which is applied to an aircraft and comprises an infrared imaging module, a visible light imaging module, an ultrasonic ranging module, an inertia measurement module and a core image processor, wherein the core image processor comprises a multi-information fusion module, a light stream speed measurement module, an OSD module and a video compression module, the infrared imaging module is connected with the input end of the multi-information fusion module, the visible light imaging module is respectively connected with the multi-information fusion module and the light stream speed measurement module, the ultrasonic ranging module and the inertia measurement module are respectively connected with the input end of the light stream speed measurement module, in the core image processor, the light speed flow measurement module is respectively connected with the input end of the multi-information fusion module and the OSD module, the output end of the OSD module is connected with the multi-information fusion module, and the output end of the, and the video compression module sends the processed image information out through a network.
Preferably, the system further comprises a face recognition module, the face recognition module is connected between the multi-information fusion module and the OSD module, the face recognition module comprises a face image feature extraction module, a face image matching and recognition module and a face image archive storage module, the face image feature extraction module is connected with the multi-information fusion module and is used for extracting captured face features, then the face image matching and recognition module matches the captured face features with face images stored in the face image archive storage module for face recognition, and then recognition results are transmitted to the multi-information fusion module through the OSD module for fusion.
Further preferably, the system also comprises a sound pick-up, wherein the sound pick-up is connected with the input end of the multi-information fusion module and is used for collecting and fusing audio for the video image.
Further preferably, the system further comprises a searchlight, wherein the searchlight is connected with the visible light imaging module and is used for providing a light source for the visible light imaging module under low illumination.
Further preferably, the multi-information fusion module fuses the infrared image input by the infrared imaging module and the light image input by the visible light imaging module in a manner that the infrared image input by the infrared imaging module is overlapped and embedded into the visible light image input by the visible light imaging module or the infrared image and the visible light image are overlapped on a pixel level.
Further preferably, the specific steps of superimposing the infrared image and the visible light image at the pixel level are as follows:
501) aligning optical axes of the visible light imaging module and the infrared imaging module by using an off-line calibration method;
502) then, pixel matching of the visible light image and the infrared image is realized through a bilinear difference method;
503) calculating the significance of the matched visible light image block and infrared image block;
504) and in the process of superposing the visible light pixels and the infrared pixels, high weights are distributed to the pixels with high significance, and the superposition on the pixel level is completed.
Further preferably, the infrared imaging module has a size of 10.50 × 12.701 × 7.14mm, a weight of 0.9g, and an athermal infrared optical lens with a horizontal field angle of 56 ° HFOV, a focal length of 1.8mm, and an F-number of 1.1; the visible light imaging module is an automatic focusing camera module, the size of the automatic focusing camera module is 8.50 multiplied by 8.5 multiplied by 5.03mm, the mass of the automatic focusing camera module is less than 0.5g, and a visible light automatic focusing lens with the visual angle of 60-69 degrees, the focal length of 3.34mm and the F number of 2.8 is configured; the sound pickup is an MEMS microphone; the ultrasonic ranging module comprises an ultrasonic transmitter, an ultrasonic receiver and a control circuit; the inertial measurement module is a MEMS gyroscope.
Compared with the prior art, the invention has the advantages that:
the invention integrates the infrared imaging technology, the visible light imaging technology and the information fusion technology, innovatively adds the functions of optical flow speed measurement and face recognition in a double-optical system, realizes a subminiature double-optical imaging system, further expands the operation capability of the small reconnaissance type unmanned aerial vehicle in a complex environment, fully expands the visual field of a ground operation unit in a hidden environment, and enables action personnel to keep situation perception, threat detection and monitoring capability anytime and anywhere.
Drawings
FIG. 1 is a block diagram of the system;
FIG. 2 is a diagram of a face recognition module;
fig. 3 is a composition diagram of an ultrasonic ranging module.
FIG. 4 is a schematic diagram illustrating an infrared image input by an infrared imaging module in a multi-information fusion module being superimposed and embedded in a visible light image input by a visible light imaging module;
fig. 5 is a schematic diagram of superposition of an infrared image and a visible light image at a pixel level in the multi-information fusion module.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Referring to fig. 1, the present invention provides a subminiature dual-optical imaging system with multi-information fusion and optical flow velocity measurement functions, which is applied to an aircraft, and comprises an infrared imaging module, a visible light imaging module, an ultrasonic ranging module, an inertia measurement module and a core image processor, wherein the core image processor comprises a multi-information fusion module, an optical flow velocity measurement module, an OSD module and a video compression module, the infrared imaging module is connected with the input end of the multi-information fusion module, the visible light imaging module is respectively connected with the multi-information fusion module and the optical flow velocity measurement module, the ultrasonic ranging module and the inertia measurement module are respectively connected with the input end of the optical flow velocity measurement module, the optical velocity flow measurement module is respectively connected with the input end of the multi-information fusion module and the OSD module, the output end of the OSD module is connected with the multi-information fusion module, and the output end of the multi-information fusion module, and the video compression module sends the processed image information out through a network.
Referring to fig. 2, the core image processor includes a face recognition module, the face recognition module is connected between the multi-information fusion module and the OSD module, the face recognition module includes a face image feature extraction module, a face image matching and recognition module, and a face image archive storage module, the face image feature extraction module is connected with the multi-information fusion module for extracting captured face features, and then the face image matching and recognition module matches the captured face features with the face images stored in the face image archive storage module for face recognition, and then transmits recognition results to the multi-information fusion module through the OSD module for fusion.
In order to increase audio and improve the visible light image acquisition effect, the system further comprises a sound pick-up and a searchlight, wherein the sound pick-up is connected with the input end of the multi-information fusion module and is used for fusing the video image acquisition audio. The searchlight is connected with the visible light imaging module and is used for providing a light source for the visible light imaging module under low illumination.
An infrared imaging module: an infrared video image of 9Hz resolution of 160 x 120 is provided, an HFOV with a horizontal field angle of 56 DEG is configured, and an athermal infrared optical lens with a focal length of 1.8mm and an F number of 1.1 is provided. Under the conditions that the visibility is more than 5km, the relative humidity is less than 70 percent and the target background temperature difference is more than 5k, the personnel detection distance of 60m and the personnel identification distance of 30m can be realized. The infrared imaging module is connected with the core image processor through the SPI interface, and can realize multiple intelligent infrared display by utilizing the powerful image processing function of the infrared imaging module, and the infrared imaging module comprises up to 9 pseudo-color templates. The diversified infrared intelligent display mode can meet the detection requirements of different tasks in different environments. The infrared imaging module can also provide temperature data which cannot be sensed by naked eyes, and the rapid analysis and decision-making on site are facilitated. The temperature measurement mode comprises point temperature measurement, regional temperature measurement, isotherm display and the like, and is mainly used for temperature-related searching and detecting tasks, such as application fields of city inspection, forest fire prevention, dangerous object detection, fire fighting, search and rescue and the like. The infrared imaging module has the size of only 10.50 multiplied by 12.701 multiplied by 7.14mm, the weight is 0.9g, the power consumption is 140mW, and powerful support is provided for the miniaturization and the light weight of the whole subminiature dual-optical system.
Visible light imaging module: the visible light imaging module is composed of an automatic focusing camera module. The support outputs a maximum of 500-ten-thousand-pixel image (2592x1944 resolution), 720p HD video at 60fps, and 1080p HD video at 30 fps. The visual light automatic focusing lens with the field angle of 60-69 degrees, the focal length of 3.34mm and the F number of 2.8 is configured, so that the personnel detection distance of more than 200m, the personnel identification distance of more than 100m and the face identification distance of more than 25m can be realized. The visible light imaging module can perform self-adaptive adjustment on brightness, contrast chroma and saturation by utilizing the powerful image processing function of the core processor, so that a night scene mode and a defogging algorithm are realized. The automatic focusing camera module is only 8.50 multiplied by 8.5 multiplied by 5.03mm in size, the mass is less than 0.5g, and the power is between 150mW and 200mW during working.
The ultrasonic ranging module: referring to fig. 3, the ultrasonic ranging module includes an ultrasonic transmitter, an ultrasonic receiver, and a control circuit, providing 2-400 cm of non-contact distance sensing.
An inertia measurement module: is a MEMS gyroscope.
A sound pick-up: is a MEMS microphone.
Searchlight: the searchlight uses the LED lamp pearl of low-power consumption to throw light on, uses under the illumination condition is unsatisfactory, needs visible light video output's condition. LED lamp pearl consumption 0.5W of low-power consumption.
In the core image processor:
the multi-information fusion module: the multi-information fusion module realizes effective fusion of visible light and infrared imaging, video and sound. The fusion mode of the visible light image and the infrared image is divided into two modes of picture-in-picture and pixel level fusion, wherein the picture-in-picture fusion carries out on-screen display by embedding the infrared into the visible light image, as shown in fig. 4. The pixel-level fusion aligns the optical axes of the infrared image and the visible light image through image calibration, and then superposes the pixel level according to the information quantity represented by the visible light image and the infrared image to obtain a video image with richer information, as shown in fig. 5, the specific fusion method is as follows:
501) aligning optical axes of the visible light imaging module and the infrared imaging module by using an off-line calibration method;
502) then, pixel matching of the visible light image and the infrared image is realized through a bilinear difference method;
503) calculating the significance of the matched visible light image block and infrared image block;
504) and in the process of superposing the visible light pixels and the infrared pixels, high weights are distributed to the pixels with high significance, and the superposition on the pixel level is completed.
The multi-information fusion module utilizes complementary information provided by the modules, and the fused image contains more comprehensive and abundant information, so that the multi-information fusion module is more in line with the visual characteristics of people or machines and more beneficial to further analysis and processing of the image and automatic target identification; under adverse environmental conditions (e.g., smoke, dust, cloud, fog, rain, etc.), detection performance may be improved by multi-sensor image fusion. For example, in smoke and fog environments, the quality of visible light images is poor (even objects cannot be seen clearly), and the obtained infrared images have strong penetration capacity to smoke and fog, and clear images can be obtained despite some attenuation of signals.
The optical flow speed measurement module:
and processing the surface images of the object continuously acquired by the visible light imaging module through an optical flow tracking algorithm, and analyzing and processing the generated image digital matrix. Because two adjacent images always have the same characteristics, the average motion of the surface characteristics of the object can be judged by comparing the position change information of the characteristic points, the analysis result is finally converted into two-dimensional coordinate offset and is stored in a specific register in the form of pixel number, and the detection of the moving object is realized. Because the unit of the visible light image is pixel, the speed calculated by the optical flow method is not provided with scale, the distance of the plane movement from the camera is measured by the ultrasonic module, and the pixel movement is converted into real movement. In order to enable the optical flow velocity measurement module to be used on an unmanned aerial vehicle, a plane represented by an image is found, the ultrasonic module and the two sensors of the inertial measurement element are fused on an algorithm to obtain stable velocity output, and the optical flow velocity measurement algorithm flow is as follows:
801) the visible light imaging module collects the ith frame image, selects a characteristic salient target in the image, and extracts a characteristic point set F of the target i；
802) The visible light imaging module collects the (i + 1) th frame image, and the feature point set F matched with the feature point set of the (i + 1) th frame image is searched in the (i + 1) th frame imagei+1；
803) According to the obtained feature point set FiAnd Fi+1Calculating an average optical flow vector by the matched feature point pairs:
wherein
Set of feature points FiAnd Fi+1The coordinate value of the matched characteristic point;
804) and obtaining the relative speed of the aircraft to the ground according to the average optical flow vector:
wherein T is the time interval between two adjacent frames i, i + 1;
805) according to the distance Z between the visible light imaging module and the ground target measured by the ultrasonic ranging module, converting the relative speed of the aircraft to the ground into an absolute speed:
wherein f is the focal length of the visible light lens;
806) since the aircraft shakes itself during the flight, the deviation caused by the shake is measured by the inertial measurement unit, and the deviation is eliminated from the absolute speed calculated in step 805), so as to obtain the target ground absolute speed.
A face recognition module:
the face recognition module comprises a face image archive, a face image feature extraction module and a face image matching and recognition module. The face image archive uses a camera to collect face image files of unit personnel, and generates feature templates of the face image files to be stored in a database. The face image feature extraction module adopts a regional feature analysis method, integrates a computer image processing technology and a biological statistics principle, extracts face feature points from an image returned by the pan-tilt camera by using the computer image processing technology, and establishes a face feature template by using the biological statistics principle. The face image matching and identifying module compares and identifies the extracted face characteristic template with the characteristic templates stored in the database, and finally judges and confirms the identity information of the personnel according to the compared similar values. The whole work flow of face recognition is as follows:
2) The subminiature dual-optical system captures and tracks the human face through the visible light imaging module;
3) the visible light imaging module inputs the collected image into the face recognition module;
4) the face recognition module extracts feature points of the input face picture/image and establishes a feature template;
5) and identifying and comparing the extracted face feature template with the feature templates in the database so as to confirm the identity of the personnel.
An OSD module: the speed measured by the optical flow speed measuring module is input into the multi-information fusion module, and the speed information is embedded into the visual flow for offline analysis.
A video compression module: and carrying out compression coding on the fused video and transmitting the video to the outside.
The subminiature dual-light imaging system with the multi-information fusion and the optical flow velocity measurement functions has the following main performance indexes:
1. infrared index:
the field angle: 56 ° × 48 °
Resolution ratio: 160 (h). times.120 (v)
Frame rate: 9HZ
NETD<50mk
Temperature measurement precision: 5 degree @25 deg.C
2. Visible light index:
the field angle: 60-69 deg
Highest resolution: 2592x1944
Frame rate: 1080p30Hz
720p 60Hz
3. Weight: <30g
4. Volume: 72 (length) × 36 (width) × 20 (height) mm
5. Power consumption: <2W
6. Scout performance:
the target detection distance is more than or equal to 200m (visible) and more than or equal to 60m (infrared)
The target detection distance is more than or equal to 200m (visible) and more than or equal to 30m (infrared)
The face recognition distance is more than or equal to 25m (visible).
Claims (7)
1. The subminiature dual-light imaging system with the functions of multi-information fusion and light stream speed measurement is characterized by being applied to an aircraft and comprising an infrared imaging module, a visible light imaging module, an ultrasonic ranging module, an inertia measurement module and a core image processor, wherein the core image processor comprises the multi-information fusion module, the light stream speed measurement module, an OSD module and a video compression module, the infrared imaging module is connected with the input end of the multi-information fusion module, the visible light imaging module is respectively connected with the multi-information fusion module and the light stream speed measurement module, the ultrasonic ranging module and the inertia measurement module are respectively connected with the input end of the light stream speed measurement module, the light speed flow measurement module is respectively connected with the input end of the multi-information fusion module and the input end of the OSD module in the core image processor, the output end of the OSD module is connected with the multi-information fusion module, and the output end of, and the video compression module sends the processed image information out through a network.
2. The system of claim 1, further comprising a face recognition module connected between the multi-information fusion module and the OSD module, wherein the face recognition module comprises a face image feature extraction module, a face image matching and recognition module, and a face image archive storage module, the face image feature extraction module is connected to the multi-information fusion module and is configured to extract the captured face features, and the face image matching and recognition module matches the captured face features with the face images stored in the face image archive storage module for face recognition, and then transmits the recognition results to the multi-information fusion module through the OSD module for fusion.
3. The subminiature dual-light imaging system with multi-information fusion and optical flow velocimetry of claim 1, further comprising a sound pick-up connected to the input of the multi-information fusion module for fusion of video image capture audio.
4. The subminiature dual-light imaging system with multi-information fusion and optical flow velocimetry of claim 1, further comprising a searchlight coupled to the visible light imaging module for providing a light source to the visible light imaging module at low light levels.
5. The ultra-small dual-light imaging system with multi-information fusion and optical flow velocity measurement functions as claimed in claim 1, wherein the multi-information fusion module fuses the infrared image inputted by the infrared imaging module and the optical image inputted by the visible light imaging module in a manner that the infrared image inputted by the infrared imaging module is superimposed and embedded into the visible light image inputted by the visible light imaging module or the infrared image and the visible light image are superimposed on a pixel level.
6. The subminiature dual-light imaging system with multi-information fusion and optical flow velocimetry of claim 5, wherein the step of superimposing the infrared image and the visible image at pixel level comprises:
501) Aligning optical axes of the visible light imaging module and the infrared imaging module by using an off-line calibration method;
502) then, pixel matching of the visible light image and the infrared image is realized through a bilinear difference method;
503) calculating the significance of the matched visible light image block and infrared image block;
504) and in the process of superposing the visible light pixels and the infrared pixels, high weights are distributed to the pixels with high significance, and the superposition on the pixel level is completed.
7. The subminiature dual-optical imaging system with multi-information fusion and optical flow velocimetry of claim 1, wherein the infrared imaging module has a size of 10.50 x 12.701 x 7.14mm, a weight of 0.9g, and is configured with an athermal infrared optical lens with a horizontal field angle of 56 ° HFOV, a focal length of 1.8mm and an F-number of 1.1; the visible light imaging module is an automatic focusing camera module, the size of the automatic focusing camera module is 8.50 multiplied by 8.5 multiplied by 5.03mm, the mass of the automatic focusing camera module is less than 0.5g, and a visible light automatic focusing lens with the visual angle of 60-69 degrees, the focal length of 3.34mm and the F number of 2.8 is configured; the sound pickup is an MEMS microphone; the ultrasonic ranging module comprises an ultrasonic transmitter, an ultrasonic receiver and a control circuit; the inertial measurement module is a MEMS gyroscope.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010585768.5A CN111856061A (en) | 2020-06-24 | 2020-06-24 | Subminiature dual-light imaging system with multi-information fusion and light stream speed measurement functions |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010585768.5A CN111856061A (en) | 2020-06-24 | 2020-06-24 | Subminiature dual-light imaging system with multi-information fusion and light stream speed measurement functions |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111856061A true CN111856061A (en) | 2020-10-30 |
Family
ID=72989481
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010585768.5A Pending CN111856061A (en) | 2020-06-24 | 2020-06-24 | Subminiature dual-light imaging system with multi-information fusion and light stream speed measurement functions |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111856061A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106500852A (en) * | 2016-09-28 | 2017-03-15 | 北方夜视技术股份有限公司 | Infrared and visible light image registration and the System and method for for merging |
| CN106570861A (en) * | 2016-10-25 | 2017-04-19 | 深圳市高巨创新科技开发有限公司 | Optical flow velocity measurement method and system for unmanned plane |
| US20190005673A1 (en) * | 2017-06-30 | 2019-01-03 | Intel Corporation | System for determining anatomical feature orientation |
| CN109635760A (en) * | 2018-12-18 | 2019-04-16 | 深圳市捷顺科技实业股份有限公司 | A kind of face identification method and relevant device |
| CN110428008A (en) * | 2019-08-02 | 2019-11-08 | 深圳市唯特视科技有限公司 | A kind of target detection and identification device and method based on more merge sensors |
| US10527423B1 (en) * | 2016-04-07 | 2020-01-07 | Luftronix, Inc. | Fusion of vision and depth sensors for navigation in complex environments |
-
2020
- 2020-06-24 CN CN202010585768.5A patent/CN111856061A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10527423B1 (en) * | 2016-04-07 | 2020-01-07 | Luftronix, Inc. | Fusion of vision and depth sensors for navigation in complex environments |
| CN106500852A (en) * | 2016-09-28 | 2017-03-15 | 北方夜视技术股份有限公司 | Infrared and visible light image registration and the System and method for for merging |
| CN106570861A (en) * | 2016-10-25 | 2017-04-19 | 深圳市高巨创新科技开发有限公司 | Optical flow velocity measurement method and system for unmanned plane |
| US20190005673A1 (en) * | 2017-06-30 | 2019-01-03 | Intel Corporation | System for determining anatomical feature orientation |
| CN109635760A (en) * | 2018-12-18 | 2019-04-16 | 深圳市捷顺科技实业股份有限公司 | A kind of face identification method and relevant device |
| CN110428008A (en) * | 2019-08-02 | 2019-11-08 | 深圳市唯特视科技有限公司 | A kind of target detection and identification device and method based on more merge sensors |
Non-Patent Citations (1)
| Title |
|---|
| 都伊林: "《智能安防新发展与应用》", 31 July 2018, 华中科技大学出版社 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110830756B (en) | Monitoring method and device | |
| US8416298B2 (en) | Method and system to perform optical moving object detection and tracking over a wide area | |
| CN108802758B (en) | Intelligent security monitoring device, method and system based on laser radar | |
| CN101969548B (en) | Active video acquiring method and device based on binocular camera shooting | |
| CN103390281B (en) | A kind of two spectrum night vision device onboard systems and two spectrum Combined design method | |
| WO2012023766A2 (en) | Security camera tracking and monitoring system and method using thermal image coordinates | |
| CN210526874U (en) | Airborne three-light photoelectric pod system | |
| KR101985711B1 (en) | Augmented reality CCTV system, and control method for the same | |
| CN208240087U (en) | A kind of infrared and visible light emerging system and image fusion device | |
| CN109816702A (en) | A kind of multiple target tracking device and method | |
| KR102392822B1 (en) | Device of object detecting and tracking using day type camera and night type camera and method of detecting and tracking object | |
| CN108540707A (en) | Recognition of face crime scene investigation device | |
| KR101834882B1 (en) | Camara device to detect the object having a integral body with a optical video camera and a thermal camera | |
| CN108881846B (en) | Information fusion method and device and computer readable storage medium | |
| CN105025240A (en) | Law enforcement recorder and control method thereof | |
| CN111856061A (en) | Subminiature dual-light imaging system with multi-information fusion and light stream speed measurement functions | |
| CN109708659B (en) | Distributed intelligent photoelectric low-altitude protection system | |
| CN106156764B (en) | Realize the optical tracking system and its control method of high speed tracking | |
| KR102305438B1 (en) | System and method for UAV surveillance | |
| Angkhem et al. | Night-Time Human Detection From UAV | |
| CN115035470A (en) | Low, small and slow target identification and positioning method and system based on mixed vision | |
| KR20230101974A (en) | integrated image providing device for micro-unmanned aerial vehicles | |
| CN111343431B (en) | Airport target detection system based on image rectification | |
| CN112990187A (en) | Target position information generation method based on handheld terminal image | |
| Picus et al. | Novel Smart Sensor Technology Platform for Border Crossing Surveillance within FOLDOUT |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20201030 |
|
| RJ01 | Rejection of invention patent application after publication |