CN115994865A

CN115994865A - Restoration method, acquisition restoration device and monitoring system for flutter blurred image

Info

Publication number: CN115994865A
Application number: CN202211315365.4A
Authority: CN
Inventors: 李响
Original assignee: Keshi Dalian Embedded Technology Development Co ltd
Current assignee: Keshi Dalian Embedded Technology Development Co ltd
Priority date: 2022-10-26
Filing date: 2022-10-26
Publication date: 2023-04-21

Abstract

The invention discloses a method for recovering a flutter blurred image, a device for collecting and recovering the flutter blurred image and a monitoring system, which comprise the following steps based on structural similarity and entropy: the method comprises the steps of collecting a flutter fuzzy image, constructing a fuzzy kernel through presetting a binary coding sequence and fuzzy lengths, calculating structural similarity indexes of the flutter fuzzy image and all decoded clear images, determining an image information entropy value in a certain range near the image with the maximum structural similarity, calculating an information entropy value of a reconstructed image, determining the minimum information entropy value as the fuzzy length of the most orderly reconstructed decoded image, and adopting an inverse filtering method to realize the clear restoration of the image. The invention starts from the aim of completely protecting the image information of the target, and simultaneously considers the method for removing the flutter fuzzy target from two angles of imaging and image post-processing. Modulating the exposure process by utilizing a pre-coding mode, protecting the original high-frequency information in the acquired image, and decoding the acquired image in the subsequent image restoration process to obtain a clear image.

Description

Restoration method, acquisition restoration device and monitoring system for flutter blurred image

Technical Field

The invention relates to the technical field of image restoration, in particular to a restoration method, an acquisition restoration device and a monitoring system for a flutter blurred image.

Background

With the development of digital storage media, consumer electronic devices such as digital cameras and smart phones have become a main source for acquiring emerging image information, and image clarity is one of the primary factors for guaranteeing reliable and effective information acquisition. Today, where digital cameras are popular, image information is collected anytime and anywhere, and each collected image contains characteristics of a target. The image is optimal when the acquired image can correctly reflect the characteristics of the target area. When the acquired image is unclear or the acquired image can not reflect the specific characteristics of the target area at the time, the effective information is lost, and the image is degraded. If the difference of the video monitoring images between day and night illumination is large, aviation and remote sensing images are easily affected by the atmosphere, and underwater imaging is affected by multipath interference such as refraction and the like. Therefore, when the image is degraded, the effective information is missing or wrong, so that the image quality is seriously affected, and how to protect the original information of the image is a problem to be solved in front-end image acquisition and back-end computer image processing.

The coded exposure imaging technique is a computational imaging means proposed by Raskar et al in 2006, and the core idea is to control the opening and closing of the camera shutter by presetting a specific binary coding sequence during the exposure of the camera. Compared with the traditional camera exposure mode, the method is equivalent to setting a broadband filter on a time domain, so that as much as possible of medium-high frequency information is reserved in the image acquisition process from the perspective of frequency domain rate analysis, the zero point part on the frequency domain is eliminated, the reversibility of restoration is realized, and the disease state problem of the restoration of the blurred image is greatly improved.

Image degradation is a comprehensive characterization of image blur, distortion, noise, etc., and as a result, the loss of image information. The reason for image degradation is various, and the image is generally subjected to the processes of acquisition, imaging, transmission, storage and the like, and each part is likely to cause image degradation. Motion blur is a problem frequently encountered in the optical imaging process, and the reason for the generation is that the acquired image is blurred due to the relative displacement between the acquired object and the camera in the camera exposure process, so that the image resolution is reduced, and the imaging quality is greatly influenced. The motion blurred image restoration technology is to restore the blurred image into a clear image by modeling and mathematical solving an imaging physical process by utilizing the existing motion blurred image on the premise of not re-acquiring a target scenery, and has important application value in the fields of civil, military and the like.

The Chinese patent CN110097509B is used for acquiring a fuzzy image by utilizing a coding exposure mode, performing primary extraction of a target by adopting a background difference method, and then comprehensively coding exposure motion fuzzy superposition characteristics and motion priori information to realize accurate extraction of a motion fuzzy target region, and performing PSF accurate estimation and restoration reconstruction by combining a student-t restoration algorithm, wherein a restoration result can be obtained after 2-3 iterations. In the present invention, it is necessary to input a scene background image and a local moving object blurred image. The Chinese patent publication CN202011448781.2 utilizes a deep learning method to estimate the fuzzy core, solves the problem that the fuzzy core is difficult to estimate in the traditional method, and searches out the image block with the highest content of the high-frequency information of the image by using a genetic algorithm. But does not solve the matching problem between coding and motion blur length. Because of the important role of the accuracy of blur length estimation in image restoration reconstruction, the present invention utilizes a method of jointly estimating blur length based on image Structural Similarity (SSIM) and image information Entropy (Entropy) to restore the encoded exposure image.

Disclosure of Invention

In order to effectively solve the technical problems, the invention adopts the following technical scheme based on structural similarity and entropy: a method for restoring a flutter blurred image,

step one: collecting flutter blurred images;

step two: by presetting the code length to be

Binary coding sequence->

And blur Length->

Construction of fuzzy core->

Said->

For binary coding sequences->

Is defined by the formula->

Calculating to obtain a reconstructed image, wherein->

For flutter blurred image +.>

To decode an image;

step three: calculating a flutter blurred image

And all decoded pictures +.>

Structural similarity index->

Length of blur at the time of searching for maximum +.>

The formula is: />

In the formula->

Search range for fuzzy length;

step four: determining an image information entropy value in a certain range near an image with the maximum structural similarity, wherein a search interval is

Wherein->

Is positive integer->

,/>

Is made of->

The total blur length limit range is determined to be +.>

Within the limits according to the formula ∈ ->

And

calculating spatial entropy of each decoded image>

And spectral entropy after image DCT (discrete cosine transform)>

Wherein->

Wherein->

Is a characteristic binary group->

Frequency of occurrence, < >>

Gray representing a pixelMetric value->

Representing neighborhood gray scale mean,/->

Is the scale of the image; if signal->

DCT transformation into (1)

，/>

Is a generalized frequency domain variable, normalized DCT transform coefficient is +.>

；

Step five: formula (VI)

Calculating the information entropy value of the reconstructed image>

Wherein->

For the picture space entropy value->

Weight of occupied->

Spectral entropy values for the coefficients of the DCT-transform (discrete cosine transform) of an image>

The weight is occupied;

step six: taking out

Minimum value, determined as most ordered reconstructed decoded picture +.>

Is +.>

The formula is->

；/>

Step seven: the image is restored clearly by adopting an inverse filtering method, and the formula is

Wherein->

For the most ordered reconstruction of the decoded pictures +.>

For flutter blurred image +.>

Is a fuzzy kernel.

Further, the step one is that the flutter blurred image is collected by setting exposure time

Exposure time +.>

Average divide into->

Time slots, & gt>

For the number of time slots (i.e. the code length), each time slot time is +.>

Each time slot is +.>

Collecting charges, outputting independently, and then superposing to form a flutter blurred image, wherein the total time required for collecting the image is

。

Further, the first step sets an exposure time

Exposure time +.>

Average divide into->

Time slots, each time slot time is +>

，/>

The transmission process is driven out only once for the number of time slots, i.e

。

Further, the second step is to preset the length to be

Binary coding sequence->

And blur Length->

Construction of fuzzy core->

The method comprises changing image convolution operation into product operation, adding zero vector +.>

An improved mathematical model of the relative displacement of the acquired image in the spatial domain is built, which corresponds to the blurred pixel length of the actual target projected into the image plane>

Greater than the pre-encoded length->

It is necessary to add/subtract the code sequence>

Then a plurality of 0 s are added to make the length equal to the length of the blurred pixel, so as to form a length of +.>

Is->

In the formula->

For the length of ambiguity after zero padding, +.>

For blurring image +.>

In the direction of movementA length; at this time, the blur kernel for reconstruction +.>

According to the blurring length after zero padding +.>

The Toeplitz matrix is formed by single direction shift.

On the other hand, the invention also provides a device for collecting and recovering the flutter blurred image, which uses a method for recovering the flutter blurred image to collect and recover the flutter blurred image, and comprises the following steps: the device comprises an image acquisition module, a core control module, a communication module, a storage module, a clock module, a transmission and display module; the image acquisition module consists of a CCD image sensor, a time sequence driving circuit, a signal conditioning and converting circuit and the like; the time sequence driving circuit is divided into a horizontal time sequence driving module and a vertical time sequence driving module, and the time sequence driving circuit provides proper driving level for the CCD image sensor; the signal conditioning and converting circuit comprises analog signal amplification filtering and analog-to-digital conversion; the pre-coding sends an instruction to the image acquisition module through the core control module to enable the image acquisition module to acquire a flutter blurred image of the motion; the image data and the prefabricated codes are stored in the storage module, and the whole circuit is ensured by the clock module and the communication module; the CCD image sensor includes a horizontal shift register and a vertical shift register.

Further, the core control module comprises a time sequence module, a decoding reconstruction module, an image reading module, a DDR control module, an FPGA gigabit network core, an exposure code and a reference clock.

Further, the driving level is a three-state level.

Further, the image acquisition module further comprises a laser generator and a laser receiver, and the CCD image sensor receives laser emitted by the laser generator through the laser receiver to perform automatic focusing so as to solve the problem of image blurring caused by inaccurate focusing.

Further, the image acquisition module further comprises an ultrasonic generator and an ultrasonic receiver, and the CCD image sensor receives ultrasonic waves sent by the ultrasonic generator through the ultrasonic receiver to perform automatic focusing so as to solve the problem of image blurring caused by focusing inaccuracy.

Further, the image acquisition module further comprises an infrared generator and an infrared receiver, and the CCD image sensor receives infrared light emitted by the infrared generator through the infrared receiver to perform automatic focusing so as to solve the problem of image blurring caused by inaccurate focusing.

Furthermore, the acquisition and restoration device for the flutter blurred image further comprises a three-dimensional model and a visualization module, wherein the image acquisition module is connected with the three-dimensional model and the visualization module, and the three-dimensional model and the visualization module are used for carrying out dynamic display in a three-dimensional visualization mode.

Further, the core control module further includes a target tracking module that designates a specific dither blur image as a target image when a plurality of dither blur images are acquired.

On the other hand, the invention also provides a monitoring system, which uses a plurality of the acquisition and restoration devices of the flutter blurred images, the monitoring system also comprises a remote data transmission module and a blockchain module, the blockchain module is divided into a monitoring node and a management node, the remote data transmission module is respectively installed on the plurality of the acquisition and restoration devices of the flutter blurred images to form a plurality of monitoring nodes, and the monitoring nodes reconstruct the decoding images generated by the nodes in the most order

Timestamping, generating simultaneously a pair of public and private keys, said most ordered reconstructed decoded image to be timestamped +.>

Uploading to the blockchain module, performing digital signature by using a private key, and transmitting a public key to the management node; the management node is a central control system provided with the remote data transmission module, and is connected withReceiving the public key of the monitoring node to verify the private key, and checking the most orderly reconstructed decoded image +.>

。

The beneficial effects of the invention are as follows:

1. the invention discloses a coding exposure imaging restoration method for generating blurring by flutter of a target relative to a camera in a single motion direction.

2. The invention starts from the aim of completely protecting the target image information, and provides a method for removing the blurring of a moving target from two angles of imaging and image post-processing. Modulating the exposure process by utilizing a pre-coding mode, protecting the original high-frequency information in the acquired image, and decoding the acquired image in the subsequent image restoration process to obtain a clear image.

3. The method omits manual selection intervention or the use time of external measurement equipment, and breaks the image restriction of natural image rules. The complexity of equipment used in the prior art is obviously reduced by estimating through a combined image restoration algorithm of image structure similarity and entropy.

4. The image acquisition and restoration device disclosed by the invention can solve the restoration problem of motion blurred images, realize clear restored images of cameras with higher frame rates with lower cost, and utilize the non-reference image evaluation index test, and compared with general exposure, the average value of the improvement of the related indexes is about 2 times of the related index value of the general common exposure.

5. The monitoring system of the invention uses the remote data transmission and control module to uplink the image information to the block chain module, and uses the advantages of the block chain technology such as decentralization, encryption, non-falsification and the like, so that the monitoring system can be widely applied to the industries such as traffic monitoring and the like.

Drawings

In order to better express the technical scheme of the invention, the following description of the invention is given by way of the accompanying drawings:

fig. 1 is a schematic diagram of an image structure similarity structure according to a first embodiment;

FIG. 2 is a fuzzy image fuzzy kernel construction diagram of a pre-coding exposure of the first embodiment;

FIG. 3 is a diagram showing the relationship between one-dimensional relative motion and convolution calculation according to the first embodiment;

fig. 4 is a diagram of a process of acquisition and reconstruction decoding of a flutter blurred image according to the first embodiment;

fig. 5 is a block diagram of a flutter blurred image acquisition and restoration device according to the third embodiment;

FIG. 6 is a flow chart of a codec procedure of a three-dither blurred image according to an embodiment;

FIG. 7 is a graph showing the experimental results of the present invention;

reference numerals illustrate: 1. the device comprises an image acquisition module 11, a CCD image sensor 12, a time sequence driving circuit 13, a signal conditioning and converting circuit 2, a core control module 21, a time sequence module 22, a decoding reconstruction module 23, an image reading module 24, a DDR control module 25, an FPGA gigabit network core module 26, an exposure coding module 27, a reference clock 3, a communication module 4, a storage module 5 and a clock module.

Detailed Description

Preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are merely for explaining the technical principles of the present invention, and are not intended to limit the scope of the present invention.

It should be noted that, in the description of the present invention, terms such as "upper," "lower," "left," "right," "inner," "outer," and the like indicate directions or positional relationships based on the directions or positional relationships shown in the drawings, which are merely for convenience of description, and do not indicate or imply that the apparatus or elements must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as limiting the present invention.

Furthermore, it should be noted that, in the description of the present invention, unless explicitly specified and limited otherwise, the terms "mounted," "connected," and "connected" are to be construed broadly, and may be either fixedly connected, detachably connected, or integrally connected, for example; can be mechanically or electrically connected; can be directly connected or indirectly connected through an intermediate medium, and can be communication between two elements. The specific meaning of the above terms in the present invention can be understood by those skilled in the art according to the specific circumstances.

The working principle of the invention is as follows: and a Toeplitz matrix is utilized to form a fuzzy core, and the problem of image restoration of a target and a camera in a single motion direction is solved according to the construction form of the fuzzy core.

The coded exposure will be for a complete exposure time

Is divided into->

The number of the time slots is consistent with the preset code length, and whether each time slot is exposed is consistent with the code word of the corresponding bit. Taking a one-dimensional signal as an example, if the acquisition length is +.>

Is>

The signal and length are +>

Is>

The convolution operation (namely binary code sequence) can be expressed as

/>

From the relative relation of signals, the above is relative to one-dimensional signals

Movement in one direction causes data misalignmentAnd (5) superposition. The method can be understood as using Toeplitz matrix to divide the clear one-dimensional signal +.>

And (2) fuzzy core->

The matrix convolution relationship between the two is converted into a product relationship of the matrix. If->

Any one of the data is +.>

The method is continuous data superposition in the traditional sense; while->

The data in (a) is->

Or->

When represented as discontinuous or intermittent data superposition, where

。

From the viewpoint of shutter exposure, the code exposure differs from the normal exposure in that the shutter is not kept in a one-time open state, but is controlled to open and close by a preset code at a certain frequency. If you get

Indicating the slot exposure; otherwise, if->

Indicating that the time slot is not exposed.

As established in fig. 3 is a static binary code sequenceColumn of

One-dimensional clear signal shifted from time-sharing>

Is a mathematical model of the acquisition of (a). Taking the one-dimensional relative motion of the imaging system and the target object as an example, the encoding exposure imaging and image restoration process will be described. If blur image +.>

Length in the direction of movement is +.>

Is>

Exposing by an imaging system, whether the imaging system exposes and has a length of +>

Is a preset binary coding sequence +.>

And keep the same. Will +_in FIG. 3>

Is set to +.>

. If the two signals move relatively, the acquisition length is equal to

Signal of->

The signal is a convolution model between the two. When->

At +.>

Middle is accumulated with time->

Information in (a); while->

When (I)>

Without any newly added information. Thus, the acquisition signal +.>

Whether or not to accumulate and +.>

Whether it is 1. In the restoration process, coding by time-sharing dislocation ∈>

Restoring the relatively stationary one-dimensional signal +.>

. The acquisition encoding exposure and image reconstruction decoding process is shown in fig. 4.

The same exposure code and different blur lengths can form different blur kernels, and the quality of the restored reconstructed decoded image is also different. In order to find a clear restored image from a plurality of reconstructed decoded images, the structural similarity of the image quality evaluation function is taken as a comparison basis, so that high-quality reconstruction of the image is formed, and a system block diagram of the high-quality reconstruction is shown in figure 1.

In the context of figure 1 of the drawings,

flutter blurred image indicative of degradation caused by flutter, < >>

Representing the original sharp image. Similar to human visual sense, the structural similarity will be from luminance +.>

Contrast->

Structural->

And independently judging the structural similarity of the two images at three angles. These three quantities can be expressed as

、/>

、/>

；

There is a compound of the formula (I),

，/>

，/>

，

，/>

，/>

、/>

for blurring image +.>

And original sharp image->

An average value of the pixels; />

For blurring image +.>

And original sharp image->

Standard deviation of each; />

For blurring image +.>

And original sharp image->

Is a cross-correlation function of (2); to avoid the situation that the molecular denominator is zero, three very small positive numbers are defined, e.g.>

、/>

、/>

. In this embodiment, <' > a->

For a range of pixel values of a grey image, e.g. 8 bits for a grey image, i.e +.>

。/>

、/>

Is a default value.

The image is blurred by the flutter

And original sharp image->

The similarity function between can be expressed as:

in this case, parameters

For adjusting->

、/>

、/>

The ratio of the three parameters is as

There is->

The formula shows the original sharp image +.>

And flutter blurred image +.>

Structural similarity of (3). />

The value range of (2) is +.>

。/>

The larger the value of (c), the more similar the two image qualities are.

However, in practical experiments, clear images cannot be obtained, and flutter blurred images

Is the only data that can be obtained, and the corresponding reconstructed image can be obtained by different blur lengths. Because the acquired code exposure blurred image is derived from the target object, similarity evaluation can be performed by reconstructing the image and the code exposure blurred image. Therefore, the dither blurred image +.>

And encoding exposure decoding image +.>

The structural similarity index of (2) is defined as +.>

. However, the dither blurred image +.>

Is a degraded image, with which the decoded image is evaluated +.>

Deviations in the structural similarity index are caused, and therefore, the order of the images needs to be calculated to finally determine the restored image.

The structural similarity method can only find the reconstructed image which is most similar to the coded exposure blurred image, but the image is not necessarily the most ordered image which accords with the natural statistics rule. The original clear image should be an orderly natural image, so that the restored reconstructed image is orderly used as an index for judging the image quality. To avoid deviations from structural similarity alone, information entropy is introduced. Entropy is a measure representing the amount of information. And determining the image range most similar to the blurred image by combining the structural similarity, and then searching the most ordered image information to determine the final restored image.

When a system is more ordered, the information entropy of the system is lower; conversely, the more chaotic the system, the higher the entropy of the system. The order degree of gray values among image pixels in the search interval is calculated by using the spatial entropy; meanwhile, in order to evaluate the flat characteristic of the reconstructed decoded signal, the signal is transformed into the frequency domain using discrete cosine transform and its spectral entropy is calculated. And determining the optimal fuzzy length by adjusting the weight estimation occupied by the spatial entropy and the spectral entropy, and reconstructing and restoring the clear image. The ordering degree formula of the spatial entropy representation image is as follows:

wherein->

Representing the maximum value in the gray scale range of the image; />

Representing gray scale +.>

Is a possibility of (1). The one-dimensional image entropy cannot reflect the spatial characteristics of the gray distribution of the image, and two-dimensional entropy of the image needs to be introduced, wherein the spatial distribution formula of the neighborhood gray mean value of the characteristic image is as follows:

in the formula->

Wherein->

Is a characteristic binary group->

Frequency of occurrence, < >>

Representing the gray value of a pixel +.>

Representing a neighborhood gray level average; />

Is the scale of the image.

To obtain an accurate solution, spectral entropy is introduced to detect the flatness of the signal spectrum. When the signal has strong correlation in the spatial domain, the transformation into the frequency domain appears as a concentrated convergence of a specific region, where the spatial domain signal is transformed into the frequency domain using a Discrete Cosine Transform (DCT). Since the image pixel value is a real number, the DCT is also a real number operation, and the operation speed is faster than the complex number operation in the Fourier transform. If the signal is

DCT transform to +.>

，/>

Is a generalized frequency domain variable, the normalized DCT coefficient is:/>

Its spectral entropy can be expressed as +.>

。

1-4, a method for restoring a flutter blurred image:

step one: the acquisition of the flutter blurred image is completed by using the CCD image sensor 11, and the exposure time is set

Exposure time +.>

Average divide into->

Time slots, each time slot time is +>

，/>

For the number of time slots, each time slot +.>

。

Step two: by presetting the code length to be

Binary coding sequence->

And blur Length->

Construction of fuzzy core->

Said->

For binary coding sequences->

Is defined by the formula->

Calculating to obtain a reconstructed image, wherein->

For flutter blurred image +.>

To decode a sharp image;

as shown in fig. 2, by presetting the length to be

Binary coding sequence->

And blur Length->

Construction of fuzzy core->

The method is that the image convolution operation is changed into a product operationFuzzy core +.>

The shift to the right and down in the main diagonal forms a Toeplitz-like matrix, as shown in fig. 2 (a), where the blur length of the object projected onto the image plane is equal to the code length. It is in fact difficult to ensure that the blur length of the object projected onto the image plane is equal to the encoding length. If the length of the blurred pixels of the actual target projected into the image plane is large, a clear restored image cannot be obtained by using the model.

In order not to change the original one-dimensional signal

Frequency information of (2) in the original one-dimensional signal +.>

The tail is added with zero vector of specific length +.>

An improved mathematical model of the relative displacement of the acquired image in the spatial domain is built, as shown in fig. 2 (b). This situation corresponds to a blurred pixel length of the actual target projected into the image plane being larger than the pre-coding length +.>

After encoding, a plurality of 0 s are added to be equal to the length of the blurred pixel, so that the length is +.>

Is->

In the formula->

For the length of ambiguity after zero padding, +.>

For blurring image +.>

Length in the direction of movement; at this time, the blur kernel for reconstruction +.>

According to the blurring length after zero padding +.>

The shift constitutes a Toeplitz matrix form.

One-dimensional signal

And->

The process of the relative shift superposition is equivalent to the convolution process of signals. When there is a flutter blurred image +.>

Length in the direction of movement is +.>

Is>

When generating a displacement, a one-dimensional signal with a length m is +.>

The convolution process will produce a length of +>

Is->

. If at this time->

For binary coding sequence, one-dimensional signal is selectively superimposed according to coding rule>

Is provided. Thus, the imaging process of the coded exposure is similar to the front-to-back accumulated smear at the time of ordinary exposure, and is the accumulation of the exposed image in a plurality of coded time slots.

Step three: calculating a flutter blurred image

And all decoded sharp pictures +.>

Is of the structural similarity index of (2)

Length of blur at the time of searching for maximum +.>

The formula is: />

In the formula->

Search range for fuzzy length;

Wherein->

Is positive integer->

,/>

Is made of->

The total blur length limit range is determined to be +.>

Within the limits according to the formula ∈ ->

And

calculating spatial entropy of each decoded image>

And spectral entropy after image DCT (discrete cosine transform)>

Wherein->

Wherein->

Is a characteristic binary group->

Frequency of occurrence, < >>

Representing the gray value of a pixel +.>

Representing neighborhood gray scale mean,/->

Is the scale of the image; if signal->

DCT transformation into (1)

，/>

；

Step five: formula (VI)

Calculating the information entropy value of the reconstructed image>

Wherein->

For the picture space entropy value->

Weight of occupied->

The weight is occupied;

step six: taking out

Minimum value, determined as most ordered reconstructed decoded picture +.>

Is +.>

The formula is->

；/>

Wherein->

For the most ordered reconstruction of the decoded pictures +.>

For flutter blurred image +.>

Is a fuzzy kernel.

The second embodiment differs from the first embodiment in that the exposure time is set in the first step

Exposure time +.>

Average divide into->

Time slots, each time slot time is +>

，/>

The transmission process is driven out only once for the number of time slots, i.e. +.>

。

The method uses only one charge to drive and transfer time

And meanwhile, the occurrence probability of electronic noise is reduced. The total exposure time of the method is kept consistent with the common exposure time. I.e. under equal conditions the pixel displacement of the object and camera projection in the image plane coincides with a normal camera. The method is characterized in that a motion blur superposition process is arranged in front, the motion process is converted into a charge accumulation process conforming to a preset coding rule, and in the process, because the sum of time slots is the total exposure time, the image acquisition interval time is the smallest in the scheme, namely the minimum blur length is obtained under the same condition, so that the clear image can be restored and reconstructed conveniently.

Embodiment three, a device for acquiring and recovering a flutter blurred image, is shown in fig. 5-6. As shown in fig. 5, the device consists of an image acquisition module 1, a core control module 2, a communication module 3, a storage module 4, a clock module 5, a three-dimensional model and visualization module, and a transmission and display module (not shown in the figure); the image acquisition module 1 consists of a CCD image sensor 11, a time sequence driving circuit 12, a signal conditioning and converting circuit 13 and the like; the time sequence driving circuit 2 is divided into a horizontal time sequence driving module and a vertical time sequence driving module, and provides proper driving level for the CCD image sensor 11, wherein the driving level is a three-state level; the signal conditioning and converting circuit 13 includes analog signal amplification filtering and analog-to-digital conversion. The CCD image sensor 11 includes a horizontal shift register and a vertical shift register (not shown). The core control module 2 comprises a time sequence module 21, a decoding reconstruction module 22, an image reading module 23, a DDR control module 24, an FPGA gigabit network core module 25, an exposure coding module 26 and a reference clock 27; the image acquisition module 1 dynamically displays in a three-dimensional visualization mode through a three-dimensional model and a visualization module. The pre-coding sends an instruction to the image acquisition module 1 through the core control module 2 to enable the image acquisition module to acquire a flutter blurred image of the motion; the image data and the prefabricated codes are stored in the storage module 4, and the whole circuit is ensured by the clock module 5 and the communication module 3.

To solve the image blur caused by the out-of-focus, the image acquisition module 1 may use a generator and a receiver, through which the CCD image sensor 11 receives the signal from the generator for auto-focusing. The generator and receiver may use commercially available types of products, not limited to laser generators and receivers, ultrasonic generators and receivers, infrared generators and receivers.

The core control module 2 can be finished by XC6SLX45T-3FG484C in Spartan6 series manufactured by Xilinx company; the reference clock 27 is selected from

A crystal oscillator; the driving time sequence required by the CCD image sensor 11 is generated by a time sequence module 21, a time sequence driving signal is generated by a core processing module (a field programmable logic device FPGA is used in the embodiment of the invention), and a special chip CXD3400N is selected because two levels and three levels are required to be ensured together for driving the CCD image sensor 11. The image reading module 23 reads the image data and saves the data in the DDR data memory 24 through the DDR management core of the FPGA. The decoding and reconstructing module 22 in the core control module 2 is reserved, and can reconstruct the clear restored image by intra-chip decoding by using a relevant restoring method after reading the flutter blurred image in the DDR data memory 24. The FPGA gigabit network core module 25 adopted for data communication with the upper computer is a high-speed bidirectional transmission channel, and reliably transmits image data and status commands. The physical layer transceiver chip adopts a physical layer gigabit Ethernet transceiver 88E1111; the CCD image sensor 11 adopts a high-sensitivity and low-noise ICX204AL black-and-white image sensor; the signal conditioning and converting circuit 13 uses a chip AD9949. Other modules are all commercially available universal devices.

As shown in fig. 6, the dither blurred image codec program flow chart. Preset binary coding sequence

When a control signal for shutter change is supplied to the substrate SUB,is also used as a blur kernel in the decoding module>

Is a structure of (a). Therefore, when the dither matrix image conforming to the preset coding +.>

After being acquired, the decoding module uses the inverse filtering method +.>

And obtaining a decoded clear image. Meanwhile, the decoding module is used as an optional module in the embedded coding camera, and the blurred image can be restored by being transmitted to an upper computer for processing and restoring or by downloading a complex reconstruction method.

The fourth embodiment is a monitoring system, which uses a plurality of vibration blurred image acquisition and restoration devices, the monitoring system further comprises a remote data transmission module and a blockchain module, the blockchain module is divided into a monitoring node and a management node, the plurality of vibration blurred image acquisition and restoration devices are respectively provided with the remote data transmission module to form a plurality of monitoring nodes, and the monitoring nodes reconstruct the most orderly decoded image generated by the monitoring nodes

Uploading to the blockchain module, performing digital signature by using a private key, and transmitting a public key to the management node; the management node is a central control system provided with the remote data transmission module, receives the public key of the monitoring node to verify the private key, and checks the most orderly reconstructed decoded image

。

Fig. 7 is a graph comparing experimental results of other methods using the prior art with experimental results of using the method and apparatus of the present invention.

The above examples illustrate only a few embodiments of the invention, which are described in detail and are not to be construed as limiting the scope of the invention. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the invention, which are all within the scope of the invention.

Claims

1. A method for restoring a flutter blurred image is characterized by comprising the following steps of

Step one: collecting flutter blurred images;

step two: by presetting the code length to be

Binary coding sequence->

And blur Length->

Construction of fuzzy core->

The said

For binary coding sequences->

Is defined by the formula->

Calculating to obtain a reconstructed image, wherein->

For flutter blurred image +.>

To decode an image;

step three: calculating a flutter blurred image

And all decoded pictures +.>

Structural similarity index->

Length of blur at the time of searching for maximum +.>

The formula is: />

In the formula->

Search range for fuzzy length;

Wherein->

Is positive integer->

,/>

Is made of->

The total blur length limit range is determined to be +.>

Within the limits according to the formula ∈ ->

And

calculating spatial entropy of each decoded image>

And spectral entropy after image DCT (discrete cosine transform)>

Wherein->

Wherein->

Is a characteristic binary group->

Frequency of occurrence, < >>

Representing the gray value of a pixel +.>

Representing a neighborhoodGray scale mean value->

Is the scale of the image; if signal->

DCT transformation into (1)

，/>

；

Step five: formula (VI)

Calculating the information entropy value of the reconstructed image>

Wherein->

For the picture space entropy value->

Weight of occupied->

The weight is occupied;

step six: taking out

Minimum value, determined as most ordered reconstructed decoded picture +.>

Is +.>

The formula is->

；

Wherein->

For the most ordered reconstruction of the decoded pictures +.>

For flutter blurred image +.>

Is a fuzzy kernel.

2. The method of claim 1, wherein the step one, the acquisition of the dither blurred image, sets the exposure time

Exposure time +.>

Average divide into->

Time slots each of which is

Each time slot is +.>

After the collected charges are singly output, the collected charges are overlapped to form a flutter blurred image, and the total time required for collecting the images is +.>

。

3. The method of recovering a dither blur image according to claim 1, wherein the first step sets an exposure time

Exposure time +.>

Average divide into->

Time slots, each time slot time is +>

The transmission of the driving out is done only once, i.e. +.>

。

4. The method of claim 1, wherein the second step is performed by presetting the length to be

Binary coding sequence->

And blur Length->

Construction of fuzzy core->

The method comprises changing image convolution operation into product operation, adding zero vector +_of specific length at the tail of the original signal>

Establishing an improved mathematical model of the relative displacement of the acquired image in the space domain, forming a mathematical model with a length of +.>

Is->

In the formula->

For the length of ambiguity after zero padding, +.>

For blurring image +.>

According to the blurring length after zero padding +.>

The shift constitutes a Toeplitz matrix form.

5. A dither blurred image acquisition and restoration apparatus characterized in that the dither blurred image acquisition and restoration is realized using the dither blurred image restoration method of any one of claims 1 to 4, comprising: the device comprises an image acquisition module, a core control module, a communication module, a storage module, a clock module, a transmission and display module; the image acquisition module consists of a CCD image sensor, a time sequence driving circuit and a signal conditioning and converting circuit; the time sequence driving circuit is divided into a horizontal time sequence driving module and a vertical time sequence driving module, and provides proper driving level for the CCD image sensor; the signal conditioning and converting circuit comprises analog signal amplification filtering and analog-to-digital conversion; the CCD image sensor includes a horizontal shift register and a vertical shift register.

6. The device for acquiring and recovering the flutter blurred image according to claim 5, wherein the core control module comprises a time sequence module, a decoding reconstruction module, an image reading module, a DDR control module, an FPGA gigabit network core, an exposure code and a reference clock.

7. The apparatus of claim 5, wherein the driving level is a three-state level.

8. The device for acquiring and recovering the flutter blurred image of claim 5, further comprising a three-dimensional model and a visualization module, wherein the image acquisition module is connected with the three-dimensional model and the visualization module.

9. The apparatus of claim 5, wherein the core control module further comprises a target tracking module.

10. A monitoring system, characterized in that the flutter blurred image acquisition and restoration device as defined in any one of claims 5-9 is used, the monitoring system further comprises a remote data transmission module and a blockchain module, the blockchain module is divided into a monitoring node and a management node, the remote data transmission module is respectively installed on a plurality of the flutter blurred image acquisition and restoration devices to form a plurality of monitoring nodes, and the monitoring nodes reconstruct the most orderly decoded images generated by the nodes

Uploading to the blockchain module, performing digital signature by using a private key, and transmitting a public key to the management node; the management node is a central control system provided with the remote data transmission module, receives the public key of the monitoring node to verify the private key, and views the most orderly reconstructed decoded image +.>

。/>