CN107248139B - Compressive sensing imaging method based on significant vision and DMD array partition control - Google Patents
Compressive sensing imaging method based on significant vision and DMD array partition control Download PDFInfo
- Publication number
- CN107248139B CN107248139B CN201610670618.8A CN201610670618A CN107248139B CN 107248139 B CN107248139 B CN 107248139B CN 201610670618 A CN201610670618 A CN 201610670618A CN 107248139 B CN107248139 B CN 107248139B
- Authority
- CN
- China
- Prior art keywords
- dmd
- image
- target object
- array
- resolution
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003384 imaging method Methods 0.000 title claims abstract description 71
- 238000005192 partition Methods 0.000 title claims abstract description 50
- 230000004438 eyesight Effects 0.000 title description 4
- 230000000007 visual effect Effects 0.000 claims abstract description 31
- 238000000034 method Methods 0.000 claims abstract description 23
- 238000005070 sampling Methods 0.000 claims abstract description 16
- 238000001514 detection method Methods 0.000 claims abstract description 10
- 239000011159 matrix material Substances 0.000 claims description 31
- 238000012545 processing Methods 0.000 claims description 31
- 238000005259 measurement Methods 0.000 claims description 23
- 230000003044 adaptive effect Effects 0.000 claims description 15
- 238000006243 chemical reaction Methods 0.000 claims description 9
- 238000010606 normalization Methods 0.000 claims description 5
- 238000011084 recovery Methods 0.000 claims description 5
- 230000000877 morphologic effect Effects 0.000 claims description 4
- 230000001678 irradiating effect Effects 0.000 claims description 3
- 239000000126 substance Substances 0.000 claims description 3
- 238000012935 Averaging Methods 0.000 claims description 2
- 230000010339 dilation Effects 0.000 claims description 2
- 230000009191 jumping Effects 0.000 claims description 2
- 238000007500 overflow downdraw method Methods 0.000 claims description 2
- 238000005457 optimization Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 230000007547 defect Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 241000282412 Homo Species 0.000 description 1
- 241000288906 Primates Species 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000005314 correlation function Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000004424 eye movement Effects 0.000 description 1
- 210000000887 face Anatomy 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000007306 turnover Effects 0.000 description 1
- 230000004304 visual acuity Effects 0.000 description 1
- 230000003945 visual behavior Effects 0.000 description 1
- 230000004382 visual function Effects 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4007—Scaling of whole images or parts thereof, e.g. expanding or contracting based on interpolation, e.g. bilinear interpolation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10056—Microscopic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20004—Adaptive image processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20216—Image averaging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
- Studio Devices (AREA)
Abstract
The invention discloses a self-adaptive compressive sensing imaging method based on visual saliency and Digital Micromirror Device (DMD) array partition control, which reduces the resolution of a DMD by means of DMD array partition control and realizes low-resolution compressive sensing imaging of a scene. Then, the area containing the target object in the low-resolution image is found through a visual saliency detection algorithm, the size of the area of the DMD array is reduced continuously, and the sampling resolution of the area containing the target object is improved, so that the actual area of the target object on the DMD array in the scene can be gradually and accurately obtained, and high-resolution imaging of only the target object can be completed. The target object can be classified and identified by using the high-resolution target object image. The method is simple to operate, strong in anti-interference performance, good in robustness, short in imaging time and small in calculated amount.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a compressive sensing imaging method based on significant vision and DMD array partition control.
Background
Under the guidance of the traditional shannon nyquist sampling theorem, the signal processing often faces two problems, namely the limitation of analog-to-digital (A/D) converter technology and the processing pressure of massive sampling data. The CS theory shows that when the signal has sparsity or compressibility, all information of the signal can be acquired by a global non-adaptive linear projection mode with a frequency far lower than the requirement of the Shannon Nyquist sampling theorem. Based on the birth of the DMD, the compressed sensing imaging technology combining the CS theory and the DMD is rapidly developed in recent years, so that the number of sensors can be remarkably saved, and the resource waste caused by 'sampling before compression' can be avoided.
Considering a two-dimensional image I of size n1 × n2, image I can be considered as a matrix of n1 rows and n2 columns, which is vectorized to x ═ vec (I), and the notation vec (·) denotes the vectorization of the matrix into a one-dimensional signal, following the principle from top to bottom and from left to right. For a one-dimensional signal x ∈ RN×1Existence of a measurement matrix phi ∈ RM×N(M < N), the sampling rate R is M/N, and the linear measurement value y in the matrix belongs to RM×1Their relationship is as follows:
y＝Φx (1)
equation (1) can be viewed as a linear projection of the original signal x at Φ. Since y is much smaller than x, there are infinite solutions to (1) and it is generally difficult to reconstruct the original signal, but since the signal has compressibility and Φ satisfies the constrained equi-Rectangular (RIP) condition as shown in equation (2), i.e. for any K-sparse signal and the constant δk∈(0，1)：
It is theorized that the signal x can be accurately reconstructed from the measured value y by solving the optimal L0 norm problem. Other algorithms such as norm-minimum algorithm based on L1, orthogonal matching algorithm, combining algorithm, and total variation recovery algorithm (TV) have also been developed after that.
The existing compressed sensing imaging method has the defects of long imaging time, quite large data volume and insufficient memory of a computer under the condition that a DMD is imaged at full resolution. In practical applications, however, one often focuses on the information of a region of interest (ROI). If the ROI information and the background region information are not distinguished during the imaging process, a lot of resources are wasted, and the obtained ROI information is not fine enough. The resolving power of human eyes on the image is 0.35mm, namely two lines which are 0.35mm away from each other on the image can be distinguished by human eyes, and finer structures cannot be distinguished. The resolution of an image refers to the amount of information stored in the image, and can be measured in a number of ways including pixels per centimeter and pixels per inch. The resolution of the image is characterized by the number of pixels per centimeter, and the resolution of the image which can be recognized by human eyes is 10/0.35 to 28 pixels per centimeter. When the number of pixel points per centimeter of the image is less than 28, the image is low-resolution and appears as image blurring or image overlapping. When the number of pixel points on each centimeter is more than 28, the image is high-resolution, and objects in the image are clear and the details are clear.
Humans and other primates exhibit selective visual behavior such as eye movement, attention, and memory. Selectivity is the most fundamental one of many visual functions, and appears to discard a portion of the information, thereby effectively processing important information. From a human perspective, this is the process of selecting content in a scene for viewing, referred to as visual selectivity. From a scene perspective, some content in a scene is more noticeable to a viewer than other content, which may be referred to as visual saliency. The salient visual detection algorithm is used for detecting a more prominent target object in a scene, so that the ROI is detected. The algorithm principle is that, in brief, the input image is respectively processed with Gaussian blur and image average, and then two obtained images I are processed1(x, y) and I2(x, y) calculating the Euclidean distance S (x, y) | | | I1(x，y)-I2And (x, y) l and carrying out normalization processing to obtain an image, namely the saliency map S (x, y) of the input image. The existing compressed sensing imaging method cannot directly carry out high-resolution imaging on a target object in a scene, and has long consumption time and large calculation amount under the condition of high-resolution imaging.
Disclosure of Invention
The technical problem to be solved by the invention is to overcome the defects of the prior art and provide a compressive sensing imaging method based on the obvious vision and the DMD array partition control, the method can effectively detect the position of a target object, thereby realizing the purpose of only carrying out high-resolution imaging on the target object, and the method has the advantages of short imaging time, small calculated amount and good imaging effect.
The invention adopts the following technical scheme for solving the technical problems:
according to the adaptive compressed sensing imaging method based on visual saliency and Digital Micromirror Device (DMD) array partition control, low-resolution imaging of the DMD is achieved through the DMD array partition control mode, and the position of a target object on the DMD is found through a saliency visual detection algorithm; the imaging resolution of the area containing the target object is improved by reducing the partition size of the DMD array for multiple times, and finally, high-resolution compressed sensing imaging of the target object in the scene is achieved.
The further optimization scheme of the adaptive compressed sensing imaging method based on visual saliency and digital micromirror device DMD array partition control provided by the invention specifically comprises the following steps:
step one, selecting a DMD with a known resolution and building a compressed sensing imaging system;
setting a sampling rate, and generating a measurement matrix for sampling under an initial condition;
step three, setting the initial partition size of the DMD array and adjusting the stepping according to the actual resolution of the DMD and the size of the measurement matrix in the step two;
sampling the scene for multiple times by adopting the measurement matrix and obtaining an observation value vector matrix corresponding to the measurement matrix;
step five, the measurement matrix and the observation value vector matrix in the step four are used as input variables of a reconstruction algorithm, and a regional image containing the target object is obtained through recovery;
sixthly, when the resolution of the target object image in the area image recovered in the step five reaches the required resolution or only one micromirror is included in the DMD array subarea, executing a step thirteen, otherwise, jumping to a step seven to a step twelve;
processing the area image in the step five by adopting a significant visual detection algorithm to obtain a primary significant image, and performing normalization processing on the primary significant image to obtain a final significant image;
step eight, performing self-adaptive binarization processing and morphological operation on the final saliency map in the step seven to obtain a binary image, and then performing morphological dilation operation on the binary image to obtain a new binary image;
step nine, in the new binary image in the step eight, marking the region containing the target object as 1, marking the background region as 0, finding out the maximum region from the region marked as 1, wherein the area of the maximum region is S, and removing the regions marked as 1, and the areas of the maximum region are all smaller than S/10;
step ten, calculating to obtain the actual position and size of the target object on the DMD array according to the position and size of the area containing the target object in the step nine;
eleventh, according to the actual position and size of the target object on the DMD array in the tenth step, reducing the size of the partition of the DMD array, and according to the actual position and size of the target object on the DMD array and the size of the partition of the DMD array, generating a new measurement matrix;
step twelve, executing the step four to the step six;
and step thirteen, proportionally fusing the target objects in the area images obtained in the step five into the initial background of the scene.
As a further optimization scheme of the adaptive compressed sensing imaging method based on visual saliency and Digital Micromirror Device (DMD) array partition control, the method for detecting the visual saliency specifically comprises the following steps:
step one, respectively carrying out image averaging and image blurring processing on an input image I to obtain an image I1And I2；
Step two, calculating an image I1And I2The Euclidean distance between them is used to obtain image I3；
Step three, image I3And carrying out normalization processing to obtain a saliency map of the input image.
As a further optimization scheme of the adaptive compressive sensing imaging method based on visual saliency and Digital Micromirror Device (DMD) array partition control, the DMD array partition control method specifically comprises the following steps: a square area on the DMD array is selected, and all the micromirrors in the area are turned in the same state.
As a further optimization scheme of the adaptive compressed sensing imaging method based on visual saliency and digital micromirror device DMD array partition control, the saliency map adaptive binarization processing in the seventh step comprises the following specific steps: by calculating the number S of pixels in the background area of the saliency map1Counting the number of pixels from the gray level of 0 on the gray histogram of the saliency map, and finding S1In the minimum gray level interval [ L ]1L2]Will gray level L1As a binary threshold.
As a further optimization scheme of the adaptive compressive sensing imaging method based on visual saliency and Digital Micromirror Device (DMD) array partition control, the fusion method in the step thirteen specifically comprises the following steps: and filling the target image on the relationship of pixel values around the original image by adopting a bilinear interpolation algorithm.
As a further optimization scheme of the adaptive compressed sensing imaging method based on visual saliency and Digital Micromirror Device (DMD) array partition control, the compressed sensing imaging system comprises an imaging lens, a DMD, a focusing lens, a detector, an analog-to-digital conversion module and a processing module; wherein the content of the first and second substances,
the imaging lens is used for converging incident light of a scene onto the DMD;
the DMD is used for generating reflected light through a micro lens of a measuring matrix in the DMD and irradiating the reflected light to the focusing lens;
the focusing lens is used for converging the reflected light on the DMD to the detector;
the detector is used for converting the converged reflected light into an electric signal and outputting the electric signal to the analog-to-digital conversion module;
and the analog-to-digital conversion module is used for converting the electric signal into a digital signal and outputting the digital signal to the processing module for processing.
Compared with the prior art, the invention adopting the technical scheme has the following technical effects:
(1) the invention combines the visual saliency detection algorithm, can find the ROI containing the target object in the low-resolution image, and then realizes the high-resolution imaging of the target object in the ROI;
(2) the invention can realize the low-resolution sampling of the DMD to the scene by the DMD array partition control method, effectively reduces the size of the measurement matrix and reduces the complexity;
(3) the invention continuously and accurately images the actual imaging area of the target object on the DMD in the scene through the iterative algorithm, and has strong anti-interference capability and good robustness;
(4) the imaging method is simple to operate, short in imaging time and small in data calculation amount.
Drawings
FIG. 1 is a schematic diagram of a method for controlling the partitioning of an array of digital micromirror devices according to the present invention; the control method (a) is a control method of the DMD array partition size of 4 × 4, and the control method (b) is a control method of the DMD array partition size of 2 × 2.
Fig. 2 is a schematic diagram of a compressed sensing imaging system according to the invention.
Fig. 3 is a flow chart of saliency map generation as described by the present invention.
FIG. 4 is a flow chart of an adaptive compressive sensing imaging method based on visual saliency and digital micromirror device array partition control according to the present invention.
Detailed Description
The technical scheme of the invention is further explained in detail by combining the attached drawings:
fig. 1 is a schematic diagram of a method for controlling partitions of a DMD array according to the present invention, in which the initial DMD resolution is 8 × 8 and there are 64 micromirrors in total; fig. 1 (a) shows the control manner of the partition size of the 4 × 4 DMD array, i.e., the micromirrors in the 4 × 4 square DMD array use the same modulation manner, so that the 16 micromirrors flip in the same manner, and the whole partition is regarded as a whole. The initial DMD array is divided into a new DMD array by the DMD array partition control mode, and the new DMD array only comprises 2 x 2 block areas. In practical application, each area is used as a pixel, so that the resolution of the DMD is greatly reduced, the size of a measurement matrix is reduced, and the purpose of reducing the calculation complexity of a reconstruction end is achieved. While (b) in fig. 1 shows that the DMD array partitions are reduced in size based on (a) in fig. 1, and have a partition size of only 2 × 2, each partition containing 4 micromirrors; in the same control manner, (b) in fig. 1 realizes the division of the original DMD array of 8 × 8 into a new DMD array of 4 × 4. The figure shows that the resolution of the DMD can be reduced through the partition control of the DMD array, and the resolution can be improved on the basis of the original resolution by reducing the size of the partition of the DMD array. The invention is just in this way to sample the scene with low resolution and to gradually refine the actual area of the target object on the DMD array in the scene by continuously reducing the size of the DMD array partition.
As shown in fig. 2, which is a schematic diagram of a compressive sensing imaging system device, the DMD is composed of several million micromirrors with a size of the order of microns, each micromirror can be flipped independently, and the flipping condition of the micromirror is modulated by a binary element (0, 1); the initial micromirror flip angle is 0 degree, i.e. the horizontal direction, when the modulation state of the micromirror is '1', the micromirror flip angle is +12 degrees, i.e. the incident light is emitted to the focusing lens and is represented as a white area on the DMD array; when the modulation state of the micromirror is '0', the micromirror flip angle is-12 degrees, i.e., the incident light is not reflected, appearing as a black area on the DMD array. A scene shown in fig. 2 is mapped on a modulated DMD through an imaging lens, the DMD partially reflects incident light to a focusing lens, a detector converts the converged light intensity into an electric signal, an analog-to-digital converter a/D samples the electric signal and transmits a measured value to a computer; the computer can obtain a plurality of different measurement values by using the measurement matrix to continuously change the overturning state of the DMD, and the image of the scene is recovered by using a reconstruction algorithm through the known measurement matrix and the observation value. The resolution of the restored scene image is determined by the resolution of the DMD, that is, the number of array segments.
Fig. 3 is a flow chart of saliency map generation according to the present invention. The initial image is an image obtained by computer recovery after the target object in the scene is sampled by the DMD each time. And dividing the sum of all pixel values in the initial image by the number of pixels to obtain the average pixel value of the initial image. The size of the image i is the same as the original image, and the values of all pixels are the average pixel value of the original image. And (3) performing filtering processing on the initial image by using a 3 x 3 Gaussian template, so as to achieve the purpose of blurring the initial image and obtain an image II. And calculating the Euclidean distance between the image I and the image II, namely taking the square of the pixel value difference of the corresponding positions of the image I and the image II as the pixel value of the corresponding position of the new image, and normalizing the new image to obtain the saliency map of the initial image.
Fig. 4 is a flowchart of a self-adaptive compressive sensing imaging method based on visual saliency and digital micromirror array partition control according to the present invention, and the method comprises the steps of firstly, building a compressive sensing imaging system as shown in fig. 2, wherein the compressive sensing imaging system comprises an imaging lens, a DMD, a focusing lens, a detector PD, an analog-to-digital conversion module, and a processing module; the processing module may be a computer; wherein the content of the first and second substances,
the imaging lens is used for converging incident light of a scene onto the DMD;
the DMD is used for generating reflected light through a micro lens of a measuring matrix in the DMD and irradiating the reflected light to the focusing lens;
the focusing lens is used for converging the reflected light on the DMD to the detector;
the detector is used for converting the converged reflected light into an electric signal and outputting the electric signal to the analog-to-digital conversion module;
and the analog-to-digital conversion module is used for converting the electric signal into a digital signal and outputting the digital signal to the processing module for processing.
Before the experiment is carried out, the whole system is initialized, including the setting of the sampling rate of each compressed sensing imaging and the size and the adjustment parameters of the initial DMD array partition; we set the sampling rate r to 30% empirically, and the sampling rate at each time is consistent. According to the resolution of our DMD array of 512 × 512, the initial DMD subarea array size is set to 32 × 32 in the case of initial low-resolution imaging of 16 × 16, the method of DMD subarea array control is as shown in fig. 1, the subarea adjustment parameter is set to 1/4 of the last size, i.e., if the last subarea array size is 32 × 32 micromirrors, the next DMD array subarea size is 16 × 16 micromirrors. After the measurement matrix is designed, compressed sensing imaging needs to be performed on the scene shown in fig. 2, the measurement matrix changes the turnover state of the DMD shown in fig. 2 for multiple times to obtain multiple observation values, and the computer obtains a recovery effect graph shown in fig. two by using a reconstruction algorithm through the measurement matrix and the corresponding observation values. The recovered effect diagram with small resolution does not show any detail, and the place where the object exists shows that the brightness, color, contrast and texture features of the area are more obvious than other positions. It is the difference in this information that the visual saliency detection algorithm has the ability to highlight target objects from the background and other objects. The visual saliency algorithm processes the compressed sensing imaging restored image to obtain a saliency map of the compressed sensing imaging restored image, and the brightness value of a target object area in the saliency map is larger than that of other positions. And processing the saliency map by a binarization method to obtain a binary image, and selecting a disc with the diameter of 3 as an expansion type for performing expansion processing on the binary image accurately. The region marked with '1' in the new binary image is regarded as a region including the target object, and the size (number of pixels), the center of gravity, the boundary perimeter, and the pixel coordinates of each marked with '1' and the like of the region marked with '1' can be obtained by the correlation function. The number of pixels marked as '1' and their coordinates can be used to design the size of the next measurement matrix and to determine the position of the area to be measured, respectively. The iteration stop condition is that only one micromirror is included in the DMD array segment or that the target object has reached the desired resolution. And if the iteration stopping condition is not reached and the parameters are adjusted, reducing the size of the subarea on the basis of the last compressed sensing imaging and improving the imaging resolution of the target object, and repeating the compressed sensing imaging and visual saliency algorithm processing. At this time, the size of the DMD array partition is 4 times of the size of the previous DMD array partition, and the imaging resolution is enlarged by 4 times. And proportionally fusing the obtained high-resolution target object image and the background obtained by low-resolution imaging through a bilinear interpolation algorithm. The fused image is the desired output image.
The invention provides a self-adaptive compressed sensing imaging method based on visual saliency and digital micromirror device array partition control, which utilizes a DMD array partition control and salient visual detection mode to effectively detect an area containing a target object in a scene and realize high-resolution imaging of only the target object in the scene. The whole imaging process is simple to operate and good in robustness. Compared with other methods, the method has the advantages of short imaging time, low reconstruction complexity and small calculated amount in order to obtain the target object image with the same high resolution.
The foregoing is a more detailed description of the present invention that is presented in conjunction with specific embodiments, and the practice of the invention is not to be considered limited to those descriptions. For those skilled in the art to which the invention pertains, several simple deductions or substitutions can be made without departing from the spirit of the invention, and all should be considered as belonging to the protection scope of the invention.
Claims (5)
1. A self-adaptive compressed sensing imaging method based on visual saliency and Digital Micromirror Device (DMD) array partition control is characterized in that low-resolution imaging of the DMD is achieved through the DMD array partition control mode, and a significant visual detection algorithm is used for finding the position of a target object on the DMD; the imaging resolution of the area containing the target object is improved by reducing the partition size of the DMD array for multiple times, and finally, high-resolution compressed sensing imaging of the target object in the scene is realized;
the method specifically comprises the following steps:
step one, selecting a DMD with a known resolution and building a compressed sensing imaging system;
setting a sampling rate, and generating a measurement matrix for sampling under an initial condition;
step three, setting the initial partition size of the DMD array and adjusting the stepping according to the actual resolution of the DMD and the size of the measurement matrix in the step two;
sampling the scene for multiple times by adopting the measurement matrix and obtaining an observation value vector matrix corresponding to the measurement matrix;
step five, the measurement matrix and the observation value vector matrix in the step four are used as input variables of a reconstruction algorithm, and a regional image containing the target object is obtained through recovery;
sixthly, when the resolution of the target object image in the area image recovered in the step five reaches the required resolution or only one micromirror is included in the DMD array subarea, executing a step thirteen, otherwise, jumping to a step seven to a step twelve;
processing the area image in the step five by adopting a significant visual detection algorithm to obtain a primary significant image, and performing normalization processing on the primary significant image to obtain a final significant image;
step eight, performing self-adaptive binarization processing and morphological operation on the final saliency map in the step seven to obtain a binary image, and then performing morphological dilation operation on the binary image to obtain a new binary image;
step nine, in the new binary image in the step eight, marking the region containing the target object as 1, marking the background region as 0, finding out the maximum region from the region marked as 1, wherein the area of the maximum region is S, and removing the regions marked as 1, and the areas of the maximum region are all smaller than S/10;
step ten, calculating to obtain the actual position and size of the target object on the DMD array according to the position and size of the area containing the target object in the step nine;
eleventh, according to the actual position and size of the target object on the DMD array in the tenth step, reducing the size of the partition of the DMD array, and according to the actual position and size of the target object on the DMD array and the size of the partition of the DMD array, generating a new measurement matrix;
step twelve, executing the step four to the step six;
step thirteen, proportionally fusing the target objects in the area images obtained in the step five into the initial background of the scene;
the compressive sensing imaging system comprises an imaging lens, a DMD (digital mirror device), a focusing lens, a detector, an analog-to-digital conversion module and a processing module; wherein the content of the first and second substances,
the imaging lens is used for converging incident light of a scene onto the DMD;
the DMD is used for generating reflected light through a micro lens of a measuring matrix in the DMD and irradiating the reflected light to the focusing lens;
the focusing lens is used for converging the reflected light on the DMD to the detector;
the detector is used for converting the converged reflected light into an electric signal and outputting the electric signal to the analog-to-digital conversion module;
and the analog-to-digital conversion module is used for converting the electric signal into a digital signal and outputting the digital signal to the processing module for processing.
2. The adaptive compressed sensing imaging method based on visual saliency and Digital Micromirror Device (DMD) array partition control according to claim 1, characterized in that the salient visual detection algorithm comprises the following specific steps:
step one, respectively carrying out image averaging and image blurring processing on an input image I to obtain an image I1And I2；
Step two, calculating an image I1And I2The Euclidean distance between them is used to obtain image I3；
Step three, image I3And carrying out normalization processing to obtain a saliency map of the input image.
3. The adaptive compressive sensing imaging method based on visual saliency and Digital Micromirror Device (DMD) array partition control according to claim 1, wherein the DMD array partition control method specifically comprises: a square area on the DMD array is selected, and all the micromirrors in the area are turned in the same state.
4. The adaptive compressed sensing imaging method based on visual saliency and Digital Micromirror Device (DMD) array partition control according to claim 1, wherein the saliency map adaptive binarization processing in step seven is specifically as follows: by calculating the number S of pixels in the background area of the saliency map1Counting the number of pixels from the gray level of 0 on the gray histogram of the saliency map, and finding S1In the minimum gray level interval [ L ]1L2]Will gray level L1As a binary threshold.
5. The adaptive compressed sensing imaging method based on visual saliency and Digital Micromirror Device (DMD) array partition control according to claim 1, wherein the fusion method in the thirteenth step is specifically as follows: and filling the target image on the relationship of pixel values around the original image by adopting a bilinear interpolation algorithm.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610670618.8A CN107248139B (en) | 2016-08-15 | 2016-08-15 | Compressive sensing imaging method based on significant vision and DMD array partition control |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610670618.8A CN107248139B (en) | 2016-08-15 | 2016-08-15 | Compressive sensing imaging method based on significant vision and DMD array partition control |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107248139A CN107248139A (en) | 2017-10-13 |
| CN107248139B true CN107248139B (en) | 2020-02-21 |
Family
ID=60016121
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610670618.8A Active CN107248139B (en) | 2016-08-15 | 2016-08-15 | Compressive sensing imaging method based on significant vision and DMD array partition control |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107248139B (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108537804B (en) * | 2018-04-04 | 2022-02-22 | 中国科学院长春光学精密机械与物理研究所 | Interested target extraction method of parallel compressed sensing imaging system |
| CN109218712A (en) * | 2018-09-28 | 2019-01-15 | 长春长光中天光电科技有限公司 | A kind of digital micromirror array hardware fault and data transmission exception detection method |
| US10855965B1 (en) | 2019-06-28 | 2020-12-01 | Hong Kong Applied Science and Technology Research Institute Company, Limited | Dynamic multi-view rendering for autostereoscopic displays by generating reduced number of views for less-critical segments based on saliency/depth/eye gaze map |
| CN113569872B (en) * | 2021-08-10 | 2024-03-19 | 大连海事大学 | Multi-resolution shoe-wearing footprint sequence identification method based on pressure significance |
| CN113671466B (en) * | 2021-08-10 | 2024-05-31 | 南京大学 | SPAD array suitable for compressed sensing |
| CN113450259A (en) * | 2021-08-31 | 2021-09-28 | 深圳百胜扬工业电子商务平台发展有限公司 | Micro-image observation processing method and device |
| CN113992840B (en) * | 2021-09-15 | 2023-06-23 | 中国航天科工集团第二研究院 | Large-view-field high-resolution imaging method and device based on compressed sensing |
| CN114034385B (en) * | 2021-10-20 | 2022-08-02 | 山东师范大学 | Spectrum measurement method and system based on compressed sensing and dynamic local high resolution |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101964113A (en) * | 2010-10-02 | 2011-02-02 | 上海交通大学 | Method for detecting moving target in illuminance abrupt variation scene |
| CN102542267A (en) * | 2011-12-26 | 2012-07-04 | 哈尔滨工业大学 | Salient region detecting method combining spatial distribution and global contrast |
| CN103020965A (en) * | 2012-11-29 | 2013-04-03 | 奇瑞汽车股份有限公司 | Foreground segmentation method based on significance detection |
| CN103969693A (en) * | 2014-04-30 | 2014-08-06 | 中国科学院长春光学精密机械与物理研究所 | Photoelectric detection imaging system and method |
| CN105046681A (en) * | 2015-05-14 | 2015-11-11 | 江南大学 | Image salient region detecting method based on SoC |
-
2016
- 2016-08-15 CN CN201610670618.8A patent/CN107248139B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101964113A (en) * | 2010-10-02 | 2011-02-02 | 上海交通大学 | Method for detecting moving target in illuminance abrupt variation scene |
| CN102542267A (en) * | 2011-12-26 | 2012-07-04 | 哈尔滨工业大学 | Salient region detecting method combining spatial distribution and global contrast |
| CN103020965A (en) * | 2012-11-29 | 2013-04-03 | 奇瑞汽车股份有限公司 | Foreground segmentation method based on significance detection |
| CN103969693A (en) * | 2014-04-30 | 2014-08-06 | 中国科学院长春光学精密机械与物理研究所 | Photoelectric detection imaging system and method |
| CN105046681A (en) * | 2015-05-14 | 2015-11-11 | 江南大学 | Image salient region detecting method based on SoC |
Non-Patent Citations (1)
| Title |
|---|
| 基于压缩感知理论的高光谱图像重建和超分辨成像技术研究;刘海英;《博士学位论文》;20140401;说明书第81-82页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107248139A (en) | 2017-10-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107248139B (en) | Compressive sensing imaging method based on significant vision and DMD array partition control | |
| Zhang | Image processing | |
| KR102674646B1 (en) | Apparatus and method for obtaining distance information from a view | |
| US11601631B2 (en) | Super-resolution in structured light imaging | |
| US8950872B2 (en) | Image projecting device, image processing device, image projecting method, and computer-readable recording medium | |
| JP4082714B2 (en) | Image high resolution device, image high resolution method, image high resolution program, and image high resolution system | |
| US20220222776A1 (en) | Multi-Stage Multi-Reference Bootstrapping for Video Super-Resolution | |
| US10366480B2 (en) | Super-resolution systems and methods | |
| JP2017182796A (en) | Method and device for processing lightfield data | |
| US11908160B2 (en) | Method and apparatus for context-embedding and region-based object detection | |
| JP2016502704A (en) | Image processing method and apparatus for removing depth artifacts | |
| KR20210089166A (en) | Bright Spot Removal Using Neural Networks | |
| KR101906796B1 (en) | Device and method for image analyzing based on deep learning | |
| KR20170022736A (en) | Apparatus and method for resolution enhancement based on dictionary learning | |
| US20180183998A1 (en) | Power reduction and performance improvement through selective sensor image downscaling | |
| US20200211159A1 (en) | Method for image super resolution imitating optical zoom implemented on a resource-constrained mobile device, and a mobile device implementing the same | |
| Wang et al. | Fusing bird view lidar point cloud and front view camera image for deep object detection | |
| US20190340785A1 (en) | Image processing for object detection | |
| CN113269722A (en) | Training method for generating countermeasure network and high-resolution image reconstruction method | |
| US9995905B2 (en) | Method for creating a camera capture effect from user space in a camera capture system | |
| Wang et al. | Learning an epipolar shift compensation for light field image super-resolution | |
| Weng et al. | Boosting event stream super-resolution with a recurrent neural network | |
| CN114494258A (en) | Lens aberration prediction and image reconstruction method and device | |
| US9392146B2 (en) | Apparatus and method for extracting object | |
| US9160900B2 (en) | Systems and methods for compressive light sensing using multiple spatial light modulators |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |