WO2022142009A1 - 一种模糊图像修正的方法、装置、计算机设备及存储介质 - Google Patents
一种模糊图像修正的方法、装置、计算机设备及存储介质 Download PDFInfo
- Publication number
- WO2022142009A1 WO2022142009A1 PCT/CN2021/090420 CN2021090420W WO2022142009A1 WO 2022142009 A1 WO2022142009 A1 WO 2022142009A1 CN 2021090420 W CN2021090420 W CN 2021090420W WO 2022142009 A1 WO2022142009 A1 WO 2022142009A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- image
- repaired
- frame
- target video
- matrix
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 75
- 238000003702 image correction Methods 0.000 title claims abstract description 22
- 239000011159 matrix material Substances 0.000 claims abstract description 215
- 230000004927 fusion Effects 0.000 claims abstract description 95
- 238000005457 optimization Methods 0.000 claims abstract description 73
- 230000006870 function Effects 0.000 claims description 67
- 238000001514 detection method Methods 0.000 claims description 54
- 230000007547 defect Effects 0.000 claims description 43
- 230000008676 import Effects 0.000 claims description 17
- 238000012545 processing Methods 0.000 claims description 12
- 230000008439 repair process Effects 0.000 claims description 7
- 230000033001 locomotion Effects 0.000 abstract description 12
- 238000013473 artificial intelligence Methods 0.000 abstract description 9
- 238000005516 engineering process Methods 0.000 abstract description 3
- 230000008569 process Effects 0.000 description 9
- 238000004364 calculation method Methods 0.000 description 6
- 238000004891 communication Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000002940 Newton-Raphson method Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000006399 behavior Effects 0.000 description 2
- 230000006835 compression Effects 0.000 description 2
- 238000007906 compression Methods 0.000 description 2
- 230000004069 differentiation Effects 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000013441 quality evaluation Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G06T5/73—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration by the use of more than one image, e.g. averaging, subtraction
-
- G06T5/77—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Definitions
- the present application belongs to the technical field of artificial intelligence, and specifically relates to a method, device, computer equipment and storage medium for blurred image correction.
- Kalman filter For image blur restoration caused by the rapid movement of the camera relative to the object to be photographed, some methods based on Kalman filter are usually used. Such methods have certain limitations, such as the generalization of image restoration methods based on Kalman filter. Insufficient ability, it cannot be repaired for high-speed moving images, so the scheme designed by simply using the Kalman filter method often does not have high adaptability, and other repair methods need to be added to achieve high-speed moving image repair, but this is undoubtedly It will increase the occupation of system resources, which is not conducive to deployment in mobile terminals. In addition, the deep learning convolutional neural network method that has emerged in recent years has good adaptability, but the computational overhead is high, which is not conducive to embedding in mobile terminals.
- the purpose of the embodiments of the present application is to propose a method, device, computer equipment and storage medium for blurred image restoration, so as to solve the problem of insufficient generalization ability and high computational cost in the existing image blur restoration scheme, which is not conducive to embedded in mobile terminals. technical problem.
- the embodiments of the present application provide a method for correcting blurred images, which adopts the following technical solutions:
- a method for blurring image correction comprising:
- Receive the image restoration instruction obtain the target video corresponding to the image restoration instruction, and obtain all image frames in the target video;
- Image fusion is performed on the image to be repaired and the previous frame of the image to be repaired based on the image fusion weight matrix to obtain a repaired image, and the image to be repaired is replaced by the repaired image.
- the target video corresponding to the image restoration instruction After receiving the image restoration instruction, acquiring the target video corresponding to the image restoration instruction, and acquiring all the image frames in the target video, it also includes:
- Grayscale processing is performed on all image frames respectively to obtain a grayscale image of each image frame
- the grayscale image of each image frame is normalized to obtain the normalized image matrix corresponding to each image frame.
- the embodiments of the present application also provide a device for correcting blurred images, which adopts the following technical solutions:
- a blurred image correction device comprising:
- the target video acquisition module is used to receive the image restoration instruction, acquire the target video corresponding to the image restoration instruction, and acquire all image frames in the target video;
- a to-be-repaired image determination module configured to determine the to-be-repaired image in all image frames of the target video based on a preset image detection strategy
- the deformation matrix acquisition module is used to import the detected image to be repaired into the pre-trained image deformation prediction model to obtain the deformation matrix of the to-be-repaired image;
- the deformation matrix optimization module is used to construct the deformation matrix optimization function, and iteratively updates the deformation matrix based on the deformation matrix optimization function to obtain the image fusion weight matrix;
- the image repairing module is used for image fusion based on the image fusion weight matrix of the image to be repaired and the previous frame of the image to be repaired to obtain the repaired image, and replace the image to be repaired with the repaired image.
- the embodiment of the present application also provides a computer device, which adopts the following technical solutions:
- a computer device includes a memory and a processor, wherein computer-readable instructions are stored in the memory, and when the processor executes the computer-readable instructions, the following blurred image correction method is implemented:
- Receive the image restoration instruction obtain the target video corresponding to the image restoration instruction, and obtain all image frames in the target video;
- Image fusion is performed on the image to be repaired and the previous frame of the image to be repaired based on the image fusion weight matrix to obtain a repaired image, and the image to be repaired is replaced by the repaired image.
- the embodiments of the present application also provide a computer-readable storage medium, which adopts the following technical solutions:
- a computer-readable storage medium where computer-readable instructions are stored on the computer-readable storage medium, and when the computer-readable instructions are executed by a processor, the following blurred image correction method is implemented:
- Receive the image restoration instruction obtain the target video corresponding to the image restoration instruction, and obtain all image frames in the target video;
- Image fusion is performed on the image to be repaired and the previous frame of the image to be repaired based on the image fusion weight matrix to obtain a repaired image, and the image to be repaired is replaced by the repaired image.
- the present application discloses a blurred image correction method, device, computer equipment and storage medium, which belong to the technical field of artificial intelligence. Since the image frames in the target video are often continuous, although the quality of the extracted target image frame is not excellent, the information contained in the blurred part of the target image is inferred from the image at the previous moment of the target image frame, and then the information contained in the blurred part of the target image is inferred. After the target image is fused, the information of the image can be recovered to achieve the purpose of image restoration.
- the present application determines the image to be repaired in all the image frames of the target video through a preset image detection strategy, imports the image to be repaired into a pre-trained image deformation prediction model, and obtains the deformation matrix of the image to be repaired.
- Matrix optimization function and iteratively update the deformation matrix based on the deformation matrix optimization function to obtain the image fusion weight matrix, and perform image fusion based on the image fusion weight matrix of the image to be repaired and the previous frame of the image to be repaired to obtain the repaired image.
- the repaired image replaces the image to be repaired.
- the technical solution of the present application can simultaneously correct image blur caused by high-speed or low-speed movement of objects, has high adaptability, does not increase the occupation of system resources, and does not increase system overhead, and is suitable for deployment in the mobile terminal.
- FIG. 1 shows an exemplary system architecture diagram to which the present application can be applied
- FIG. 2 shows a flowchart of an embodiment of a method for blurred image correction according to the present application
- FIG. 3 shows a schematic structural diagram of an embodiment of a blurred image correction device according to the present application
- FIG. 4 shows a schematic structural diagram of an embodiment of a computer device according to the present application.
- the system architecture 100 may include terminal devices 101 , 102 , and 103 , a network 104 and a server 105 .
- the network 104 is a medium used to provide a communication link between the terminal devices 101 , 102 , 103 and the server 105 .
- the network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
- the user can use the terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or send messages and the like.
- Various communication client applications may be installed on the terminal devices 101 , 102 and 103 , such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, and the like.
- the terminal devices 101, 102, and 103 may be various electronic devices that have a display screen and support web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III, dynamic Picture Experts Compression Standard Audio Layer 3), MP4 (Moving Picture Experts Group Audio Layer IV, Moving Picture Experts Compression Standard Audio Layer 4) Players, Laptops and Desktops, etc.
- MP3 players Moving Picture Experts Group Audio Layer III, dynamic Picture Experts Compression Standard Audio Layer 3
- MP4 Moving Picture Experts Group Audio Layer IV, Moving Picture Experts Compression Standard Audio Layer 4
- the server 105 may be a server that provides various services, such as a background server that provides support for the pages displayed on the terminal devices 101 , 102 , and 103 .
- the blurred image correction method provided in the embodiments of the present application is generally performed by a terminal device, and accordingly, the blurred image correction apparatus is generally set in the terminal device.
- terminal devices, networks and servers in FIG. 1 are merely illustrative. There can be any number of terminal devices, networks and servers according to implementation needs.
- the method for correcting blurred images includes the following steps:
- S201 Receive an image restoration instruction, acquire a target video corresponding to the image restoration instruction, and acquire all image frames in the target video.
- the user terminal may be a mobile terminal, such as a smart phone, and the target video may be a video already shot by the user and stored in the database of the mobile terminal, or may be a video shot by the user in real time through the camera of the mobile terminal.
- the electronic device (for example, the server/terminal device shown in FIG. 1 ) on which the blurred image correction method runs may receive the image restoration instruction through a wired connection or a wireless connection.
- the above wireless connection methods may include but are not limited to 3G/4G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or developed in the future .
- an image to be repaired is determined in all image frames of the target video based on a preset image detection strategy.
- the image restoration instruction includes a preset image detection strategy. After receiving the image restoration instruction, the preset image detection strategy corresponding to the image restoration instruction is extracted, and the preset image detection strategy is used in sequence in the target video. Determine the image to be repaired in all image frames.
- the preset image detection strategy may be to determine whether the current image frame is an image to be repaired by comparing two image frames connected in time series, wherein the two image frames connected in time series are the current image frame and the image frame before the current image frame. The previous image frame at the moment.
- the information contained in the blurred part of the target image is inferred from the image at the previous moment of the target image frame, and then the information contained in the blurred part of the target image is inferred. After the target image is fused, the information of the image can be recovered to achieve the purpose of image restoration.
- an image deformation prediction model can be pre-built.
- the image deformation prediction model is constructed based on the RNN cyclic neural network.
- a matrix of how the image is degraded can be obtained according to the input image sequence, that is, the input image.
- the deformation matrix of the sequence In a specific embodiment of the present application, an RNN cyclic neural network is used for image deformation prediction, the input parameter of the image deformation prediction model is the image sequence I_r to be repaired, and the image sequence I_r to be repaired includes at least the current image frame and the previous image frame.
- the target part of the deformation matrix output by the model is obtained by the following method: after converting the two images with time sequence into grayscale images, the grayscale difference value can be obtained by directly making the difference, and the grayscale difference value is filled into the initial deformation matrix.
- the target part of forms a deformation matrix.
- the training samples can be pre-collected image sequences that need to be repaired. After labeling the above image sequences, import the initial image deformation prediction model, and construct the corresponding image deformation prediction model. and based on the prediction results and the constructed loss function, the initial image deformation prediction model is iterated until the model is fitted, and the trained image deformation prediction model is output.
- the matrix difference between the image to be repaired and the previous image frame is calculated, and the matrix difference is the deformation matrix of the image to be repaired.
- the grayscale difference value of each element position on the image matrix can be obtained by directly making the difference, and the grayscale difference value of each element position is filled into the deformation matrix correspondingly.
- the target part of the position to obtain the deformation matrix of the image to be repaired is the target part of the position to obtain the deformation matrix of the image to be repaired.
- a deformation matrix optimization function is constructed, and the deformation matrix is iteratively updated based on the deformation matrix optimization function to obtain an image fusion weight matrix.
- the weight factor of the image to be repaired is calculated, wherein the weight factor of the image to be repaired represents the degree of deformation of the image to be repaired, and the initial image fusion weight matrix can be obtained by normalizing the deformation matrix, and the initial image fusion weight
- the specific parameters in the matrix are calculated.
- the deformation matrix optimization function is constructed based on the weight factor, the deformation matrix optimization function is iterated based on the Newton method, and the initial image fusion weight matrix is iteratively updated through the iterative deformation matrix optimization function to obtain the fusion weight matrix.
- the to-be-repaired image and the previous frame of the to-be-repaired image are fused to obtain a repaired image.
- the image to be repaired is M
- the previous image in the sequence of the image to be repaired is N
- the value of the p element on the M matrix of the image to be repaired is 1
- the value of the p element on the N matrix of the previous image to be repaired in the sequence is the same as that of the p element.
- the value of the element at the corresponding position is i
- the weight value of the p element in the fusion weight matrix is v.
- the following fusion ratio (1-v)*I+ v*i performs fusion of M and N to obtain a repaired image M', and replaces the original M with the repaired image M' obtained after fusion.
- the image quality of the repaired image M' is continuously detected. If the image quality of the repaired image M' does not meet the requirements, the above-mentioned image repairing process is repeated until the image quality of the finally obtained image M meets the requirements. until requested.
- the present application discloses a method for amending a blurred image, which belongs to the technical field of artificial intelligence. Since the image frames in the target video are often continuous, although the quality of the extracted target image frame is not excellent, the information contained in the blurred part of the target image is inferred from the image at the previous moment of the target image frame, and then the information contained in the blurred part of the target image is inferred. After the target image is fused, the information of the image can be recovered to achieve the purpose of image restoration. According to the above idea, the present application determines the image to be repaired in all the image frames of the target video through a preset image detection strategy, imports the image to be repaired into a pre-trained image deformation prediction model, and obtains the deformation matrix of the image to be repaired.
- Matrix optimization function and iteratively update the deformation matrix based on the deformation matrix optimization function to obtain the image fusion weight matrix, and perform image fusion based on the image fusion weight matrix of the image to be repaired and the previous frame of the image to be repaired to obtain the repaired image.
- the repaired image replaces the image to be repaired.
- the technical solution of the present application can simultaneously correct image blur caused by high-speed or low-speed movement of objects, has high adaptability, does not increase the occupation of system resources, and does not increase system overhead, and is suitable for deployment in the mobile terminal.
- the target video corresponding to the image restoration instruction After receiving the image restoration instruction, acquiring the target video corresponding to the image restoration instruction, and acquiring all the image frames in the target video, it also includes:
- Grayscale processing is performed on all image frames respectively to obtain a grayscale image of each image frame
- the grayscale image of each image frame is normalized to obtain the normalized image matrix corresponding to each image frame.
- X is called the gray value
- X is the A specific value between 0-255.
- grayscale processing is performed on all image frames to obtain a grayscale image of each image frame.
- Each pixel of the grayscale image is represented by a grayscale value.
- the grayscale image can be regarded as a matrix.
- the value range of the elements in the image matrix is between 0-255.
- Each pixel in the grayscale image matrix is normalized to obtain a normalized image matrix corresponding to the grayscale image, so that the pixel value of each pixel is between 0 and 1.
- the method further includes:
- the target video corresponding to the image restoration instruction is re-captured.
- a judgment condition for judging a standard image frame is preset in the image restoration instruction, and based on the above judgment condition, the first image frame of the target video is detected to determine whether the first image frame is a standard image frame, and if the first image frame is a standard image frame If one image frame is a standard image frame, then continue to perform the step of determining the image to be repaired in all image frames of the target video. If the first image frame is not a standard image frame, output the prompt information that the first image frame does not meet the requirements, and re-intercept the target video corresponding to the image restoration instruction to obtain a new target video.
- the step of determining an image to be repaired in all image frames of the target video based on a preset image detection strategy specifically includes:
- the preset defect detector is used to detect all image frames in the target video.
- the detection idea of the defect detector is described as follows: In a normal continuous video image frame, the image content of the two frames should be basically unchanged. Therefore, this content-based image quality discriminator is used to calculate the similarity between the two frames before and after, that is, when the former frame is used as the reference image and the latter frame is used as the image to be evaluated, the difference between the two is small due to the small difference in image content.
- the image quality evaluation requirements of the frame should be relatively high, and once the image content changes due to rapid motion and other reasons, the image to be evaluated is equivalent to a larger degradation on the basis of the reference image. The difference of a frame of images is enlarged, and a defect detector is constructed based on this.
- the current image frame and the previous image frame are imported into a preset defect detector, the previous image frame is regarded as a standard image frame that does not need to be repaired, and the difference between the current image frame and the previous image frame is calculated by calculating Similarity, by comparing the similarity with a preset similarity threshold, it is determined whether the current image frame is an image to be repaired.
- image quality detection is performed on all image frames in the target video by traversing. Therefore, the first image frame of the target video must be of high quality The standard image frame required by the condition.
- the defect detector every time the defect detector detects an image to be repaired, it repairs the image to be repaired until the repair of the image to be repaired is completed, and then the defect detector continues to detect subsequent image frames until the image to be repaired is repaired. All image frames in the target video are checked for image quality.
- the current image frame and the previous image frame are imported into a preset defect detector to obtain an image defect detection result
- the steps of determining whether the current image frame is an image to be repaired based on the image defect detection result specifically include:
- the first similarity is less than the preset similarity threshold, it is determined that the current image frame is the image to be repaired.
- the similarity between the current image frame and the previous image frame is calculated by comparing the pixel value of each pixel corresponding to each other on the current image frame and the previous image frame, and the first similarity is obtained.
- the size of the first similarity and the preset similarity threshold determines whether the current image frame is an image to be repaired. If the first similarity is less than the preset similarity threshold, it is determined that the current image frame is an image to be repaired. If the first similarity is an image to be repaired If it is greater than or equal to the preset similarity threshold, it is determined that the current image frame is an image that does not need to be repaired.
- steps of constructing a deformation matrix optimization function, iteratively updating the deformation matrix based on the deformation matrix optimization function, and obtaining an image fusion weight matrix specifically include:
- the initial fusion weight matrix is optimized by the iterative deformation matrix optimization function, and the fusion weight matrix is obtained.
- Newton's method is generally known as Newton's method, also known as Newton-Raphson method (Newton-Raphson method), Newton's method is an iterative solution method, An optimization problem can be solved by using Newton's method.
- the so-called optimization problem means that there are countless possible results for the solution of a problem, and solving an optimization problem is to find the most qualified result among the countless possible results.
- the deformation matrix is normalized to obtain an initial fusion weight matrix, and the weight factor of the image to be repaired is calculated based on the initial fusion weight matrix.
- the weight factor of the image to be repaired includes the weight factor in the x direction and the The weight factor in the y direction is specifically expressed as follows:
- ⁇ x,p is the weight factor in the x direction
- ⁇ y,p is the weight factor in the y direction
- p refers to a specific element on the initial fusion weight matrix
- L′ p refers to the element value of point p
- ⁇ is a constant
- T x,p , T y,p are RTV (relative total variation) operators, specifically expressed as follows:
- ⁇ refers to an image block patch
- the size of the image block patch is 15 ⁇ 15
- q refers to another specific element on the initial fusion weight matrix
- p and q are both in the above image block patch
- G ⁇ (p ,q) is a Gaussian function whose expression is as follows:
- G ⁇ (p, q) is a function of e
- D (p, q) is the straight-line distance between the two points p and q
- ⁇ refers to the overall variance of the patch of the image block.
- ⁇ is a constant parameter
- L p is the matrix value of the element corresponding to the above-mentioned p element in the matrix of the image to be repaired.
- the method further includes:
- the second similarity is compared with the preset similarity threshold. If the second similarity is greater than or equal to the preset similarity threshold, the restoration of the current image frame is stopped, and the next image to be restored is checked. If the second similarity is less than the preset similarity threshold, image restoration is continued on the repaired image frame until the second similarity is greater than or equal to the preset similarity threshold.
- the present application discloses a method for correcting blurred images, which belongs to the technical field of artificial intelligence. Since the image frames in the target video are often continuous, although the quality of the extracted target image frame is not excellent, the information contained in the blurred part of the target image is inferred from the image at the previous moment of the target image frame, and then the information contained in the blurred part of the target image is inferred. After the target image is fused, the information of the image can be recovered to achieve the purpose of image restoration.
- the present application determines the image to be repaired in all the image frames of the target video through a preset image detection strategy, imports the image to be repaired into a pre-trained image deformation prediction model, and obtains the deformation matrix of the image to be repaired.
- Matrix optimization function and iteratively update the deformation matrix based on the deformation matrix optimization function to obtain the image fusion weight matrix, and perform image fusion based on the image fusion weight matrix of the image to be repaired and the previous frame of the image to be repaired to obtain the repaired image.
- the repaired image replaces the image to be repaired.
- the technical solution of the present application can simultaneously correct image blur caused by high-speed or low-speed movement of objects, has high adaptability, does not increase the occupation of system resources, and does not increase system overhead, and is suitable for deployment in the mobile terminal.
- the above-mentioned target video can also be stored in a node of a blockchain.
- the blockchain referred to in this application is a new application mode of computer technologies such as distributed data storage, point-to-point transmission, consensus mechanism, and encryption algorithm.
- Blockchain essentially a decentralized database, is a series of data blocks associated with cryptographic methods. Each data block contains a batch of network transaction information to verify its Validity of information (anti-counterfeiting) and generation of the next block.
- the blockchain can include the underlying platform of the blockchain, the platform product service layer, and the application service layer.
- the aforementioned storage medium may be a non-volatile storage medium such as a magnetic disk, an optical disk, a read-only storage memory (Read-Only Memory, ROM), or a random access memory (Random Access Memory, RAM), etc.
- the present application provides an embodiment of an apparatus for correcting blurred images.
- the apparatus embodiment corresponds to the method embodiment shown in FIG. 2 .
- the apparatus Specifically, it can be applied to various electronic devices.
- the apparatus for correcting a blurred image in this embodiment includes:
- a target video acquisition module 301 configured to receive an image restoration instruction, acquire a target video corresponding to the image restoration instruction, and acquire all image frames in the target video;
- a to-be-repaired image determination module 302 configured to determine the to-be-repaired image in all image frames of the target video based on a preset image detection strategy
- the deformation matrix acquisition module 303 is used for importing the detected image to be repaired into a pre-trained image deformation prediction model to obtain a deformation matrix of the to-be-repaired image;
- the deformation matrix optimization module 304 is used for constructing a deformation matrix optimization function, and iteratively updates the deformation matrix based on the deformation matrix optimization function to obtain an image fusion weight matrix;
- the image repairing module 305 is configured to perform image fusion on the image to be repaired and the previous frame of the image to be repaired based on the image fusion weight matrix to obtain a repaired image, and replace the image to be repaired with the repaired image.
- the blurred image correction device also includes:
- the grayscale processing module is used to perform grayscale processing on all image frames respectively to obtain a grayscale image of each image frame;
- the normalization processing module is used to normalize the grayscale image of each image frame to obtain the normalized image matrix corresponding to each image frame.
- the blurred image correction device also includes:
- the standard image frame detection module is used to detect the first image frame of the target video based on the preset standard image index, and determine whether the first image frame is a standard image frame;
- the standard image frame detection result module is used to re-intercept the target video corresponding to the image restoration instruction when the first image frame is not a standard image frame.
- the to-be-repaired image determination module 302 specifically includes:
- the image frame acquisition unit is used to obtain two image frames connected in time series from the target video, wherein the two image frames connected in time series are the current image frame and the previous image frame at the previous moment of the current image frame;
- the image defect detection unit is used to import the current image frame and the previous image frame into a preset defect detector, obtain the image defect detection result, and determine whether the current image frame is an image to be repaired based on the image defect detection result.
- the image defect detection unit specifically includes:
- the first similarity calculation subunit is used to calculate the similarity between the current image frame and the previous image frame to obtain the first similarity
- a first similarity comparison subunit used for comparing the size of the first similarity and a preset similarity threshold
- the first similarity comparison result subunit is configured to determine that the current image frame is an image to be repaired when the first similarity is less than a preset similarity threshold.
- the deformation matrix optimization module 304 specifically includes:
- the matrix normalization unit is used to normalize the deformation matrix to obtain the initial fusion weight matrix
- the weight factor calculation unit is used to calculate the weight factor of the image to be repaired, and build the deformation matrix optimization function based on the weight factor;
- the function iteration unit is used to iterate the deformation matrix optimization function based on Newton's method
- the deformation matrix optimization unit is used to optimize the initial fusion weight matrix through the iterative deformation matrix optimization function to obtain the fusion weight matrix.
- the image defect detection unit also includes:
- the second similarity calculation subunit is used to import the obtained repaired image frame and the previous image frame into a preset defect detector, calculate the similarity between the repaired image frame and the previous image frame, and obtain the second similarity;
- the second similarity comparison subunit is used to compare the size of the second similarity with the preset similarity threshold
- the second similarity comparison result subunit is used for continuing to perform image restoration on the repaired image frame when the second similarity is less than the preset similarity threshold until the second similarity is greater than or equal to the preset similarity threshold.
- the application discloses a blurred image correction device, which belongs to the technical field of artificial intelligence. Since the image frames in the target video are often continuous, although the quality of the extracted target image frame is not excellent, the information contained in the blurred part of the target image is inferred from the image at the previous moment of the target image frame, and then the information contained in the blurred part of the target image is inferred. After the target image is fused, the information of the image can be recovered to achieve the purpose of image restoration. According to the above idea, the present application determines the image to be repaired in all the image frames of the target video through a preset image detection strategy, imports the image to be repaired into a pre-trained image deformation prediction model, and obtains the deformation matrix of the image to be repaired.
- Matrix optimization function and iteratively update the deformation matrix based on the deformation matrix optimization function to obtain the image fusion weight matrix, and perform image fusion based on the image fusion weight matrix of the image to be repaired and the previous frame of the image to be repaired to obtain the repaired image.
- the repaired image replaces the image to be repaired.
- the technical solution of the present application can simultaneously correct image blur caused by high-speed or low-speed movement of objects, has high adaptability, does not increase the occupation of system resources, and does not increase system overhead, and is suitable for deployment in the mobile terminal.
- FIG. 4 is a block diagram of a basic structure of a computer device according to this embodiment.
- the computer device 4 includes a memory 41, a processor 42, and a network interface 43 that communicate with each other through a system bus. It should be noted that only the computer device 4 with components 41-43 is shown in the figure, but it should be understood that it is not required to implement all of the shown components, and more or less components may be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculation and/or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, special-purpose Integrated circuit (Application Specific Integrated Circuit, ASIC), programmable gate array (Field-Programmable Gate Array, FPGA), digital processor (Digital Signal Processor, DSP), embedded equipment, etc.
- ASIC Application Specific Integrated Circuit
- FPGA Field-Programmable Gate Array
- DSP Digital Signal Processor
- the computer equipment may be a desktop computer, a notebook computer, a palmtop computer, a cloud server and other computing equipment.
- the computer device can perform human-computer interaction with the user through a keyboard, a mouse, a remote control, a touch pad or a voice control device.
- the memory 41 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static Random Access Memory (SRAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Programmable Read Only Memory (PROM), Magnetic Memory, Magnetic Disk, Optical Disk, etc.
- the memory 41 may be an internal storage unit of the computer device 4 , such as a hard disk or a memory of the computer device 4 .
- the memory 41 may also be an external storage device of the computer device 4, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, flash memory card (Flash Card), etc.
- the memory 41 may also include both the internal storage unit of the computer device 4 and its external storage device.
- the memory 41 is generally used to store the operating system and various application software installed on the computer device 4 , such as computer-readable instructions for a method for correcting blurred images.
- the memory 41 can also be used to temporarily store various types of data that have been output or will be output.
- the processor 42 may be a central processing unit (Central Processing Unit, CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. This processor 42 is typically used to control the overall operation of the computer device 4 . In this embodiment, the processor 42 is configured to execute computer-readable instructions stored in the memory 41 or process data, for example, computer-readable instructions for executing the blurred image correction method.
- CPU Central Processing Unit
- controller a controller
- microcontroller a microcontroller
- microprocessor microprocessor
- This processor 42 is typically used to control the overall operation of the computer device 4 .
- the processor 42 is configured to execute computer-readable instructions stored in the memory 41 or process data, for example, computer-readable instructions for executing the blurred image correction method.
- the network interface 43 may include a wireless network interface or a wired network interface, and the network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.
- the application discloses a computer device, which belongs to the technical field of artificial intelligence. Since the image frames in the target video are often continuous, although the quality of the extracted target image frame is not excellent, the information contained in the blurred part of the target image is inferred from the image at the previous moment of the target image frame, and then the information contained in the blurred part of the target image is inferred. After the target image is fused, the information of the image can be recovered to achieve the purpose of image restoration. According to the above idea, the present application determines the image to be repaired in all the image frames of the target video through a preset image detection strategy, imports the image to be repaired into a pre-trained image deformation prediction model, and obtains the deformation matrix of the image to be repaired.
- Matrix optimization function and iteratively update the deformation matrix based on the deformation matrix optimization function to obtain the image fusion weight matrix, and perform image fusion based on the image fusion weight matrix of the image to be repaired and the previous frame of the image to be repaired to obtain the repaired image.
- the repaired image replaces the image to be repaired.
- the technical solution of the present application can simultaneously correct image blur caused by high-speed or low-speed movement of objects, has high adaptability, does not increase the occupation of system resources, and does not increase system overhead, and is suitable for deployment in the mobile terminal.
- the present application also provides another implementation manner, that is, to provide a computer-readable storage medium
- the computer-readable storage medium may be non-volatile or volatile
- the computer-readable storage medium stores Computer readable instructions executable by at least one processor to cause the at least one processor to perform the steps of the method for blurred image correction as described above.
- the application discloses a storage medium, which belongs to the technical field of artificial intelligence. Since the image frames in the target video are often continuous, although the quality of the extracted target image frame is not excellent, the information contained in the blurred part of the target image is inferred from the image at the previous moment of the target image frame, and then the information contained in the blurred part of the target image is inferred. After the target image is fused, the information of the image can be recovered to achieve the purpose of image restoration. According to the above idea, the present application determines the image to be repaired in all the image frames of the target video through a preset image detection strategy, imports the image to be repaired into a pre-trained image deformation prediction model, and obtains the deformation matrix of the image to be repaired.
- Matrix optimization function and iteratively update the deformation matrix based on the deformation matrix optimization function to obtain the image fusion weight matrix, and perform image fusion based on the image fusion weight matrix of the image to be repaired and the previous frame of the image to be repaired to obtain the repaired image.
- the repaired image replaces the image to be repaired.
- the technical solution of the present application can simultaneously correct image blur caused by high-speed or low-speed movement of objects, has high adaptability, does not increase the occupation of system resources, and does not increase system overhead, and is suitable for deployment in the mobile terminal.
- the method of the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is better implementation.
- the technical solution of the present application can be embodied in the form of a software product in essence or in a part that contributes to the prior art, and the computer software product is stored in a storage medium (such as ROM/RAM, magnetic disk, CD-ROM), including several instructions to make a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) execute the methods described in the various embodiments of this application.
- a storage medium such as ROM/RAM, magnetic disk, CD-ROM
Abstract
一种模糊图像修正的方法、装置、计算机设备及存储介质，属于人工智能技术领域的中的计算机视觉技术，通过确定待修复图像，将待修复图像导入预先训练好的图像形变预测模型，得到待修复图像的形变矩阵，构建形变矩阵优化函数，基于形变矩阵优化函数对形变矩阵进行迭代更新，得到图像融合权重矩阵，基于图像融合权重矩阵对待修复图像和待修复图像的前一帧图像进行图像融合，得到修复图像。还涉及区块链技术，目标视频可存储于区块链中。可以同时针对物体高速移动或者低速移动所导致的图像模糊进行修正，具有较高的适应性，且不会增大系统资源的占用，适合部署在移动终端。
Description
本申请要求于2020年12月30日提交中国专利局、申请号为202011609062.4，发明名称为“一种模糊图像修正的方法、装置、计算机设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
本申请属于人工智能技术领域，具体涉及一种模糊图像修正的方法、装置、计算机设备及存储介质。
随着人工智能在金融领域中的广泛使用，在移动端开展金融行为的场景越来越丰富，这些场景中均涉及到较为严格的审批业务，如人脸识别审批业务。人脸识别审批业务中需要获取客户图像进行验证。出于体验感的考虑，用于人脸识别的客户图像往往是从用户移动端相机拍摄的视频中抽相应的图片帧所得到的图片，但在模糊图像修正过程中，发明人意识到通过这种方式得到的图片往往质量得不到保障，尤其是移动端本身的轻微移动就会造成图像的模糊，这样的图像缺陷对于人脸识别等任务来说是及其不利的。
因此，为了有效提高人脸识别的准确率，优化图像质量势在必行。针对由相机相对被拍摄物体的快速移动所造成的图像模糊修复，通常是采用一些基于卡尔曼滤波器的方法，这样的方法具有一定的局限性，例如基于卡尔曼滤波的图像修复方法的泛化能力不足，对于高速移动的图像无法实现修复，因此单纯采用卡尔曼滤波方法所设计的方案往往不具有较高的适应性，还需要额外添加其他修复方法，以实现高速移动图像修复，但这无疑会增大系统资源的占用，不利于部署在移动终端。另外，近年来兴起的深度学习卷积神经网络方法，虽然具有较好的适应性，但是计算开销大，也不利于嵌入移动终端中。
发明内容
本申请实施例的目的在于提出一种模糊图像修正的方法、装置、计算机设备及存储介质，以解决现有图像模糊修复方案存在的泛化能力不足，计算开销大，不利于嵌入移动终端中的技术问题。
为了解决上述技术问题，本申请实施例提供一种模糊图像修正的方法，采用了如下所述的技术方案：
一种模糊图像修正的方法，包括：
接收图像修复指令，获取与图像修复指令相对应的目标视频，并获取目标视频中的所有图像帧；
基于预设的图像检测策略在目标视频的所有图像帧中确定待修复图像；
将检测到待修复图像导入预先训练好的图像形变预测模型，得到待修复图像的形变矩阵；
构建形变矩阵优化函数，基于形变矩阵优化函数对形变矩阵进行迭代更新，得到图像融合权重矩阵；
基于图像融合权重矩阵对待修复图像和待修复图像的前一帧图像进行图像融合，得到修复图像，并通过修复图像替换待修复图像。
进一步地，在接收图像修复指令，获取与图像修复指令相对应的目标视频，并获取目标视频中的所有图像帧的步骤之后，还包括：
分别对所有图像帧进行灰度化处理，得到每一张图像帧的灰度图像；
对每一张图像帧的灰度图像进行归一化处理，得到每一张图像帧对应的归一化图像矩阵。
为了解决上述技术问题，本申请实施例还提供一种模糊图像修正的装置，采用了如下所述的技术方案：
一种模糊图像修正的装置，包括：
目标视频获取模块，用于接收图像修复指令，获取与图像修复指令相对应的目标视频，并获取目标视频中的所有图像帧；
待修复图像确定模块，用于基于预设的图像检测策略在目标视频的所有图像帧中确定待修复图像；
形变矩阵获取模块，用于将检测到待修复图像导入预先训练好的图像形变预测模型，得到待修复图像的形变矩阵；
形变矩阵优化模块，用于构建形变矩阵优化函数，基于形变矩阵优化函数对形变矩阵进行迭代更新，得到图像融合权重矩阵；
图像修复模块，用于基于图像融合权重矩阵对待修复图像和待修复图像的前一帧图像进行图像融合，得到修复图像，并通过修复图像替换待修复图像。
为了解决上述技术问题，本申请实施例还提供一种计算机设备，采用了如下所述的技术方案：
一种计算机设备，包括存储器和处理器，存储器中存储有计算机可读指令，处理器执行计算机可读指令时实现如下的模糊图像修正的方法：
接收图像修复指令，获取与图像修复指令相对应的目标视频，并获取目标视频中的所有图像帧；
基于预设的图像检测策略在目标视频的所有图像帧中确定待修复图像；
将检测到待修复图像导入预先训练好的图像形变预测模型，得到待修复图像的形变矩阵；
构建形变矩阵优化函数，基于形变矩阵优化函数对形变矩阵进行迭代更新，得到图像融合权重矩阵；
基于图像融合权重矩阵对待修复图像和待修复图像的前一帧图像进行图像融合，得到修复图像，并通过修复图像替换待修复图像。
为了解决上述技术问题，本申请实施例还提供一种计算机可读存储介质，采用了如下所述的技术方案：
一种计算机可读存储介质，计算机可读存储介质上存储有计算机可读指令，计算机可读指令被处理器执行时实现如下的模糊图像修正的方法的：
接收图像修复指令，获取与图像修复指令相对应的目标视频，并获取目标视频中的所有图像帧；
基于预设的图像检测策略在目标视频的所有图像帧中确定待修复图像；
将检测到待修复图像导入预先训练好的图像形变预测模型，得到待修复图像的形变矩阵；
构建形变矩阵优化函数，基于形变矩阵优化函数对形变矩阵进行迭代更新，得到图像融合权重矩阵；
基于图像融合权重矩阵对待修复图像和待修复图像的前一帧图像进行图像融合，得到修复图像，并通过修复图像替换待修复图像。
与现有技术相比，本申请实施例主要有以下有益效果：
本申请公开了一种模糊图像修正的方法、装置、计算机设备及存储介质，属于人工智能技术领域。由于目标视频中的图像帧往往是连续的，因而尽管抽取的目标图像帧质量并不优秀，但是通过目标图像帧上一时刻的图像推断出目标图像模糊部分所包含的信息，再将这些信息与目标图像进行融合，即可恢复该张图像的信息，达到图像修复的目的。根据上述思想本申请通过预设的图像检测策略在目标视频的所有图像帧中确定待修复图像，将待修复图像导入预先训练好的图像形变预测模型，得到待修复图像的形变矩阵，通过构建形变矩阵优化函数，并基于形变矩阵优化函数对形变矩阵进行迭代更新，得到图像融合权重矩阵，基于图像融合权重矩阵对待修复图像和待修复图像的前一帧图像进行图像融合，得到修复图像，并用得到的修复图像替换待修复图。本申请的技术方案可以同时针对物体高速移动或者低速移动所导致的图像模糊进行修正，具有较高的适应性，且不会增大系统资源的占用，也不会增大系统的开销，适合部署在移动终端。
为了更清楚地说明本申请中的方案，下面将对本申请实施例描述中所需要使用的附图作一个简单介绍，显而易见地，下面描述中的附图是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1示出了本申请可以应用于其中的示例性系统架构图；
图2示出了根据本申请的模糊图像修正的方法的一个实施例的流程图；
图3示出了根据本申请的模糊图像修正的装置的一个实施例的结构示意图；
图4示出了根据本申请的计算机设备的一个实施例的结构示意图。
除非另有定义，本文所使用的所有的技术和科学术语与属于本申请的技术领域的技术人员通常理解的含义相同；本文中在申请的说明书中所使用的术语只是为了描述具体的实施例的目的，不是旨在于限制本申请；本申请的说明书和权利要求书及上述附图说明中的术语“包括”和“具有”以及它们的任何变形，意图在于覆盖不排他的包含。本申请的说明书和权利要求书或上述附图中的术语“第一”、“第二”等是用于区别不同对象，而不是用于描述特定顺序。
在本文中提及“实施例”意味着，结合实施例描述的特定特征、结构或特性可以包含在本申请的至少一个实施例中。在说明书中的各个位置出现该短语并不一定均是指相同的实施例，也不是与其它实施例互斥的独立的或备选的实施例。本领域技术人员显式地和隐式地理解的是，本文所描述的实施例可以与其它实施例相结合。
为了使本技术领域的人员更好地理解本申请方案，下面将结合附图，对本申请实施例中的技术方案进行清楚、完整地描述。
如图1所示，系统架构100可以包括终端设备101、102、103，网络104和服务器105。网络104用以在终端设备101、102、103和服务器105之间提供通信链路的介质。网络104可以包括各种连接类型，例如有线、无线通信链路或者光纤电缆等等。
用户可以使用终端设备101、102、103通过网络104与服务器105交互，以接收或发送消息等。终端设备101、102、103上可以安装有各种通讯客户端应用，例如网页浏览器应用、购物类应用、搜索类应用、即时通信工具、邮箱客户端、社交平台软件等。
终端设备101、102、103可以是具有显示屏并且支持网页浏览的各种电子设备，包括但不限于智能手机、平板电脑、电子书阅读器、MP3播放器(Moving Picture Experts Group Audio Layer III，动态影像专家压缩标准音频层面3)、MP4(Moving Picture Experts Group Audio Layer IV，动态影像专家压缩标准音频层面4)播放器、膝上型便携计算机和台式计算机等等。
服务器105可以是提供各种服务的服务器，例如对终端设备101、102、103上显示的页面提供支持的后台服务器。
需要说明的是，本申请实施例所提供的模糊图像修正的方法一般由终端设备执行，相应地，模糊图像修正的装置一般设置于终端设备中。
应该理解，图1中的终端设备、网络和服务器的数目仅仅是示意性的。根据实现需要，可以具有任意数目的终端设备、网络和服务器。
继续参考图2，示出了根据本申请的模糊图像修正的的方法的一个实施例的流程图。所述的模糊图像修正的方法，包括以下步骤：
S201，接收图像修复指令，获取与图像修复指令相对应的目标视频，并获取目标视频中的所有图像帧。
具体的，当存在图像需要进行修复时，接收用户从用户终端输入的图像修复指令，获取与图像修复指令相对应的目标视频。其中，用户终端可以是移动终端，如智能手机，目标视频可以是用户已拍摄好的存在移动终端数据库的视频，也可以是用户通过移动终端的摄像头即时拍摄的视频。
在本实施例中，模糊图像修正的方法运行于其上的电子设备(例如图1所示的服务器/终端设备)可以通过有线连接方式或者无线连接方式接收图像修复指令。需要指出的是，上述无线连接方式可以包括但不限于3G/4G连接、WiFi连接、蓝牙连接、WiMAX连接、Zigbee连接、UWB(ultra wideband)连接、以及其他现在已知或将来开发的无线连接方式。
S202，基于预设的图像检测策略在目标视频的所有图像帧中确定待修复图像。
具体的，图像修复指令中包含有预设的图像检测策略，接收到图像修复指令后，提取图像修复指令对应的预设的图像检测策略，并通过预设的图像检测策略依次在目标视频中的所有图像帧中确定待修复图像。其中，预设的图像检测策略可以是通过比对时序相连的两个图像帧来确定当前图像帧是否为待修复图像，其中，时序相连的两个图像帧为当前图像帧和当前图像帧前一时刻的前一图像帧。由于目标视频中的图像帧往往是连续的，因而尽管抽取的目标图像帧质量并不优秀，但是通过目标图像帧上一时刻的图像推断出目标图像模糊部分所包含的信息，再将这些信息与目标图像进行融合，即可恢复该张图像的信息，达到图像修复的目的。
S203，将检测到待修复图像导入预先训练好的图像形变预测模型，得到待修复图像的形变矩阵。
其中，可以预先构建图像形变预测模型，该图像形变预测模型基于RNN循环神经网络进行构建，在预先构建图像形变预测模型中，可以根据输入的图像序列，获得图像如何退化的矩阵，即输入的图像序列的形变矩阵。在本申请具体的实施例中，采用RNN循环神经网络进行图像形变预测，图像形变预测模型的输入参量是待修复图像序列I_r，待修复图像序列I_r至少包括当前图像帧和前一图像帧，预测模型输出的形变矩阵的target部分是由以下方法获得：将具有时序顺序的两张图转变为灰度图后，通过直接做差可以获得灰度差值，将灰度差值填入初始形变矩阵的target部分，形成形变矩阵。在形变预测模型训练的过程中，训练样本可以是预先收集的需要进行需要进行图像修复的图像序列，对上述图像序列进行标注后导入初始的图像形变预测模型，并构建初始的图像形变预测模型相应的损失函数，并基于预测结果和构建的损失函数对初始的图像形变预测模型进行迭代，直至模型拟合，输出训练好的图像形变预测模型。
具体的，计算待修复图像和前一图像帧之间的矩阵差值，该矩阵差值即为待修复图像的形变矩阵。将具有时序顺序的两张图转变为灰度图后，通过直接做差可以获得图像矩阵 上每一个元素位置的灰度差值，将每一个元素位置的灰度差值均填入形变矩阵相应位置的target部分，得到待修复图像的形变矩阵。
S204，构建形变矩阵优化函数，基于形变矩阵优化函数对形变矩阵进行迭代更新，得到图像融合权重矩阵。
具体的，计算待修复图像的权重因子，其中，待修复图像的权重因子表征的是待修复图像的形变程度，可以通过对形变矩阵进行归一化得到初始图像融合权重矩阵，通过初始图像融合权重矩阵中的具体参量进行计算。基于权重因子构建所述形变矩阵优化函数，基于牛顿法对形变矩阵优化函数进行迭代，通过迭代后的形变矩阵优化函数对初始图像融合权重矩阵进行迭代更新，得到融合权重矩阵。
S205，基于图像融合权重矩阵对待修复图像和待修复图像的前一帧图像进行图像融合，得到修复图像，并通过修复图像替换待修复图像。
具体的，在上述得到的融合权重矩阵后，基于计算得到的融合权重矩阵以及alpha图像融合算法对待修复图像和待修复图像的前一帧图像进行融合，得到修复图像。例如，待修复图像为M，待修复图像时序的前一张图为N，待修复图像M矩阵上的p元素的值为I，待修复图像时序上的前一张图N矩阵上与p元素对应位置的元素的值为i，融合权重矩阵中p元素的权重值为v，基于计算得到的融合权重矩阵以及alpha图像融合算法对M和N，按照以下融合比例(1-v)*I+v*i进行融合M和N的融合，得到修复图像M’，并将融合后得到的修复图像M’替换原有的M。在本申请一种具体的实施例中，继续检测修复图像M’的图像质量，若修复图像M’的图像质量不满足要求，则重复上述图像修复过程，直至最终得到的图像M的图像质量满足要求为止。
本申请公开了一种模糊图像修正的方法，属于人工智能技术领域。由于目标视频中的图像帧往往是连续的，因而尽管抽取的目标图像帧质量并不优秀，但是通过目标图像帧上一时刻的图像推断出目标图像模糊部分所包含的信息，再将这些信息与目标图像进行融合，即可恢复该张图像的信息，达到图像修复的目的。根据上述思想本申请通过预设的图像检测策略在目标视频的所有图像帧中确定待修复图像，将待修复图像导入预先训练好的图像形变预测模型，得到待修复图像的形变矩阵，通过构建形变矩阵优化函数，并基于形变矩阵优化函数对形变矩阵进行迭代更新，得到图像融合权重矩阵，基于图像融合权重矩阵对待修复图像和待修复图像的前一帧图像进行图像融合，得到修复图像，并用得到的修复图像替换待修复图。本申请的技术方案可以同时针对物体高速移动或者低速移动所导致的图像模糊进行修正，具有较高的适应性，且不会增大系统资源的占用，也不会增大系统的开销，适合部署在移动终端。
进一步地，在接收图像修复指令，获取与图像修复指令相对应的目标视频，并获取目标视频中的所有图像帧的步骤之后，还包括：
分别对所有图像帧进行灰度化处理，得到每一张图像帧的灰度图像；
对每一张图像帧的灰度图像进行归一化处理，得到每一张图像帧对应的归一化图像矩阵。
具体的，灰度化就是让图像中的每一个像素点都满足下面的关系：R＝G＝B＝X，其中，R、G、B为三原色通道，此时X叫做灰度值，X是一个具体的数值，取值在0-255之间。其中，对所有图像帧进行灰度化处理，得到每一张图像帧的灰度图像，灰度图像的每一个像素点都通过灰度值表示，灰度图像可以看做是一个矩阵，灰度图像矩阵中元素取值范围在0-255之间。对灰度图像矩阵中的每一个像素点都进行归一化处理，得到灰度图像对应的归一化图像矩阵，使得每一个像素点的像素值的取值在0和1之间。
进一步地，基于预设的图像检测策略在目标视频的所有图像帧中确定待修复图像的步骤之前，还包括：
基于预设的标准图像指标对目标视频的第一张图像帧进行检测，判断第一张图像帧是否为标准图像帧；
若第一张图像帧不是标准图像帧，则重新截取与图像修复指令相对应的目标视频。
具体的，图像修复指令中预设有判断标准图像帧的判断条件，基于上述判断条件对所述目标视频的第一张图像帧进行检测，判断第一张图像帧是否为标准图像帧，如果第一张图像帧是标准图像帧，则继续执行在目标视频的所有图像帧中确定待修复图像的步骤。如果第一张图像帧不是标准图像帧，输出第一张图像帧不满足要求的提示信息，并重新截取与图像修复指令相对应的目标视频，得到新的目标视频。
进一步地，基于预设的图像检测策略在目标视频的所有图像帧中确定待修复图像的步骤，具体包括：
从目标视频中获取时序相连的两个图像帧，其中，时序相连的两个图像帧为当前图像帧和当前图像帧前一时刻的前一图像帧；
将当前图像帧和前一图像帧导入预设的缺陷检测器，得到图像缺陷检测结果，基于图像缺陷检测结果确定当前图像帧是否为待修复图像。
其中，使用预设的缺陷检测器对目标视频中的所有图像帧进行检测，缺陷检测器的检测思路描述如下：在一段正常的连续视频图像帧中，前后两帧的图像内容应当基本不变，因而采用这种基于内容的图像质量判别器，对前后两帧进行相似度计算，即将前一帧作为参考图像而后一帧作为待评估图像时，两者由于图像内容差别较小，因而对后一帧的图像质量评估要求应当较高，而一旦出现因为快速运动等原因造成的图像内容发生变化时，即待评估图像相当于在参考图像基础上发生较大的退化时，后一帧图像与前一帧图像的差异拉大，基于此构建缺陷检测器。因此，本申请通过设计一个相似度阈值，后一帧图像与前一帧图像的相似度当高于相似度阈值时，认为待修复图像不需要进行修正，反之，则认为待修复图像需要进行修正。
具体的，将当前图像帧和前一图像帧导入预设的缺陷检测器，将前一图像帧作为不需要进行修复的标准图像帧，通过计算当前图像帧与所述前一图像帧之间的相似度，通过比对相似度与预设相似度阈值，来确定当前图像帧是否为待修复图像。在本申请具体的实施例中，从目标视频的第一张图像帧开始，采用遍历的方式对目标视频中的所有图像帧进行图像质量检测，因此目标视频的第一张图像帧必须为符合质量条件要求的标准图像帧。在图像质量检测过程中，缺陷检测器每检测到一张待修复图像，都先对该待修复图像进行修复，直至该待修复图像修复完成，然后缺陷检测器继续对后续图像帧进行检测，直至标视频中的所有图像帧完成图像质量检测。
进一步地，将当前图像帧和前一图像帧导入预设的缺陷检测器，得到图像缺陷检测结果，基于图像缺陷检测结果确定当前图像帧是否为待修复图像的步骤，具体包括：
计算当前图像帧与前一图像帧之间的相似度，得到第一相似度；
比对第一相似度与预设相似度阈值的大小；
若第一相似度小于预设相似度阈值，则确定当前图像帧为待修复图像。
具体的，通过比对当前图像帧和前一图像帧上每一个相互对应的像素点的像素值，计算当前图像帧与前一图像帧之间的相似度，得到第一相似度，通过比对第一相似度与预设相似度阈值的大小，确定当前图像帧是否为待修复图像，如果第一相似度小于预设相似度阈值，则确定当前图像帧为待修复图像，如果第一相似度大于或等于预设相似度阈值，则确定当前图像帧为不需要进行修复的图像。
进一步地，构建形变矩阵优化函数，基于形变矩阵优化函数对形变矩阵进行迭代更新，得到图像融合权重矩阵的步骤，具体包括：
对形变矩阵进行归一化处理，得到初始融合权重矩阵；
计算待修复图像的权重因子，并基于权重因子构建形变矩阵优化函数；
基于牛顿法对形变矩阵优化函数进行迭代；
通过迭代后的形变矩阵优化函数对初始融合权重矩阵进行优化，得到融合权重矩阵。
其中，需要说明的是，牛顿法一般又称为牛顿迭代法(Newton's method)，也称为牛顿-拉夫逊(拉弗森)方法(Newton-Raphson method)，牛顿法是一种迭代求解方法，采用牛顿法可以解决一个优化问题，所谓优化问题就是指某个问题的解有无数种可能的结果，求解优化问题则是在无数种可能的结果中找到最符合条件的结果。
具体的，在得到形变矩阵后，对形变矩阵进行归一化处理，得到初始融合权重矩阵，基于初始融合权重矩阵计算待修复图像的权重因子，待修复图像的权重因子包括x方向的权重因子和y方向的权重因子，具体表示如下：
其中，ω
x,p是在x方向的权重因子，ω
y,p是在y方向的权重因子，p是指初始融合权重矩阵上某一个具体元素，L′
p是指p点的元素值，
是指对初始融合权重矩阵的x方向求偏微分,
是指对初始融合权重矩阵的y方向求偏微分,ε是常数，T
x,p、T
y,p为RTV(relative total variation)算子，具体表示如下：
其中，Ω是指一个图像块patch，在本申请中，图像块patch的大小为15x15，q指初始融合权重矩阵上另一个具体元素，p和q均处于上述图像块patch中，G
σ(p,q)为高斯函数，其表现形式如下：
其中，G
σ(p,q)是一个e指函数，D(p,q)为p、q两点之间的直线距离，σ是指图像块patch的整体方差，这样的设计是出于防止计算中出现畸变的考虑，因而设计的一种基于高斯加权的平滑处理。
至此，上述形变矩阵优化函数的各项参数均已经构建完毕，具体形变矩阵优化函数表示如下：
其中，λ是常数参数，L
p是在待修复图像的矩阵中与上述p元素对应的元素的矩阵值。在本申请具体的实施例中，在构建完成形变矩阵优化函数之后，需要采用一种方法来找到符合这样条件的权重矩阵，根据上述的优化函数的形式可以采用牛顿法来求解上述形变矩阵优化函数，得到融合权重矩阵。
进一步地，在基于图像融合权重矩阵对待修复图像和待修复图像的前一帧图像进行图像融合，得到修复图像，并通过修复图像替换待修复图之后，还包括：
将得到的修复图像帧和前一图像帧导入预设的缺陷检测器，计算修复图像帧与前一图像帧之间的相似度，得到第二相似度；
比对第二相似度与预设相似度阈值的大小；
若第二相似度小于预设相似度阈值，则持续对修复图像帧进行图像修复，直至第二相似度大于或等于预设相似度阈值为止。
具体的，用修复图像替换待修复图像之后，将得到的修复图像帧和前一图像帧导入预设的缺陷检测器，计算修复图像帧与前一图像帧之间的相似度，得到第二相似度，比对第二相似度与预设相似度阈值的大小，如果第二相似度大于或等于预设相似度阈值，则停止对当前图像帧的修复，并检查下一待修复图像。如果第二相似度小于预设相似度阈值，则持续对修复图像帧进行图像修复，直至第二相似度大于或等于预设相似度阈值为止。
目前，在移动端开展金融行为的场景越来越丰富，这些场景中均涉及到较为严格的审批业务，如人脸识别审批业务。人脸识别过程需要获取客户图像进行验证，出于体验感的考虑，用于人脸识别的客户图像往往是从用户移动端相机拍摄的视频中抽相应的图片帧所得到的图片，但通过这种方式得到的图片往往质量得不到保障，尤其是移动端本身的轻微移动就会造成图像的模糊，这样的图像缺陷对于人脸识别等任务来说是及其不利的。
针对上述技术问题，本申请公开了一种模糊图像修正的方法，属于人工智能技术领域。由于目标视频中的图像帧往往是连续的，因而尽管抽取的目标图像帧质量并不优秀，但是通过目标图像帧上一时刻的图像推断出目标图像模糊部分所包含的信息，再将这些信息与目标图像进行融合，即可恢复该张图像的信息，达到图像修复的目的。根据上述思想本申请通过预设的图像检测策略在目标视频的所有图像帧中确定待修复图像，将待修复图像导入预先训练好的图像形变预测模型，得到待修复图像的形变矩阵，通过构建形变矩阵优化函数，并基于形变矩阵优化函数对形变矩阵进行迭代更新，得到图像融合权重矩阵，基于图像融合权重矩阵对待修复图像和待修复图像的前一帧图像进行图像融合，得到修复图像，并用得到的修复图像替换待修复图。本申请的技术方案可以同时针对物体高速移动或者低速移动所导致的图像模糊进行修正，具有较高的适应性，且不会增大系统资源的占用，也不会增大系统的开销，适合部署在移动终端。
需要强调的是，为进一步保证上述目标视频的私密和安全性，上述目标视频还可以存储于一区块链的节点中。
本申请所指区块链是分布式数据存储、点对点传输、共识机制、加密算法等计算机技术的新型应用模式。区块链(Blockchain)，本质上是一个去中心化的数据库，是一串使用密码学方法相关联产生的数据块，每一个数据块中包含了一批次网络交易的信息，用于验证其信息的有效性(防伪)和生成下一个区块。区块链可以包括区块链底层平台、平台产品服务层以及应用服务层等。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机可读指令来指令相关的硬件来完成，该计算机可读指令可存储于一计算机可读取存储介质中，该计算机可读指令在执行时，可包括如上述各方法的实施例的流程。其中，前 述的存储介质可为磁碟、光盘、只读存储记忆体(Read-Only Memory，ROM)等非易失性存储介质，或随机存储记忆体(Random Access Memory，RAM)等。
应该理解的是，虽然附图的流程图中的各个步骤按照箭头的指示依次显示，但是这些步骤并不是必然按照箭头指示的顺序依次执行。除非本文中有明确的说明，这些步骤的执行并没有严格的顺序限制，其可以以其他的顺序执行。而且，附图的流程图中的至少一部分步骤可以包括多个子步骤或者多个阶段，这些子步骤或者阶段并不必然是在同一时刻执行完成，而是可以在不同的时刻执行，其执行顺序也不必然是依次进行，而是可以与其他步骤或者其他步骤的子步骤或者阶段的至少一部分轮流或者交替地执行。
进一步参考图3，作为对上述图2所示方法的实现，本申请提供了一种模糊图像修正的装置的一个实施例，该装置实施例与图2所示的方法实施例相对应，该装置具体可以应用于各种电子设备中。
如图3所示，本实施例所述的模糊图像修正的装置包括：
目标视频获取模块301，用于接收图像修复指令，获取与图像修复指令相对应的目标视频，并获取目标视频中的所有图像帧；
待修复图像确定模块302，用于基于预设的图像检测策略在目标视频的所有图像帧中确定待修复图像；
形变矩阵获取模块303，用于将检测到待修复图像导入预先训练好的图像形变预测模型，得到待修复图像的形变矩阵；
形变矩阵优化模块304，用于构建形变矩阵优化函数，基于形变矩阵优化函数对形变矩阵进行迭代更新，得到图像融合权重矩阵；
图像修复模块305，用于基于图像融合权重矩阵对待修复图像和待修复图像的前一帧图像进行图像融合，得到修复图像，并通过修复图像替换待修复图像。
进一步地，该模糊图像修正的装置还包括：
灰度化处理模块，用于分别对所有图像帧进行灰度化处理，得到每一张图像帧的灰度图像；
归一化处理模块，用于对每一张图像帧的灰度图像进行归一化处理，得到每一张图像帧对应的归一化图像矩阵。
进一步地，该模糊图像修正的装置还包括：
标准图像帧检测模块，用于基于预设的标准图像指标对目标视频的第一张图像帧进行检测，判断第一张图像帧是否为标准图像帧；
标准图像帧检测结果模块，用于当第一张图像帧不是标准图像帧时，重新截取与图像修复指令相对应的目标视频。
进一步地，待修复图像确定模块302具体包括：
图像帧获取单元，用于从目标视频中获取时序相连的两个图像帧，其中，时序相连的两个图像帧为当前图像帧和当前图像帧前一时刻的前一图像帧；
图像缺陷检测单元，用于将当前图像帧和前一图像帧导入预设的缺陷检测器，得到图像缺陷检测结果，基于图像缺陷检测结果确定当前图像帧是否为待修复图像。
进一步地，图像缺陷检测单元具体包括：
第一相似度计算子单元，用于计算当前图像帧与前一图像帧之间的相似度，得到第一相似度；
第一相似度比对子单元，用于比对第一相似度与预设相似度阈值的大小；
第一相似度比对结果子单元，用于当第一相似度小于预设相似度阈值时，确定当前图像帧为待修复图像。
进一步地，形变矩阵优化模块304具体包括：
矩阵归一化单元，用于对形变矩阵进行归一化处理，得到初始融合权重矩阵；
权重因子计算单元，用于计算待修复图像的权重因子，并基于权重因子构建形变矩阵优化函数；
函数迭代单元，用于基于牛顿法对形变矩阵优化函数进行迭代；
形变矩阵优化单元，用于通过迭代后的形变矩阵优化函数对初始融合权重矩阵进行优化，得到融合权重矩阵。
进一步地，图像缺陷检测单元还包括：
第二相似度计算子单元，用于将得到的修复图像帧和前一图像帧导入预设的缺陷检测器，计算修复图像帧与前一图像帧之间的相似度，得到第二相似度；
第二相似度比对子单元，用于比对第二相似度与预设相似度阈值的大小；
第二相似度比对结果子单元，用于当第二相似度小于预设相似度阈值时，持续对修复图像帧进行图像修复，直至第二相似度大于或等于预设相似度阈值为止。
本申请公开了一种模糊图像修正的装置，属于人工智能技术领域。由于目标视频中的图像帧往往是连续的，因而尽管抽取的目标图像帧质量并不优秀，但是通过目标图像帧上一时刻的图像推断出目标图像模糊部分所包含的信息，再将这些信息与目标图像进行融合，即可恢复该张图像的信息，达到图像修复的目的。根据上述思想本申请通过预设的图像检测策略在目标视频的所有图像帧中确定待修复图像，将待修复图像导入预先训练好的图像形变预测模型，得到待修复图像的形变矩阵，通过构建形变矩阵优化函数，并基于形变矩阵优化函数对形变矩阵进行迭代更新，得到图像融合权重矩阵，基于图像融合权重矩阵对待修复图像和待修复图像的前一帧图像进行图像融合，得到修复图像，并用得到的修复图像替换待修复图。本申请的技术方案可以同时针对物体高速移动或者低速移动所导致的图像模糊进行修正，具有较高的适应性，且不会增大系统资源的占用，也不会增大系统的开销，适合部署在移动终端。
为解决上述技术问题，本申请实施例还提供计算机设备。具体请参阅图4，图4为本实施例计算机设备基本结构框图。
所述计算机设备4包括通过系统总线相互通信连接存储器41、处理器42、网络接口43。需要指出的是，图中仅示出了具有组件41-43的计算机设备4，但是应理解的是，并不要求实施所有示出的组件，可以替代的实施更多或者更少的组件。其中，本技术领域技术人员可以理解，这里的计算机设备是一种能够按照事先设定或存储的指令，自动进行数值计算和/或信息处理的设备，其硬件包括但不限于微处理器、专用集成电路(Application Specific Integrated Circuit，ASIC)、可编程门阵列(Field－Programmable Gate Array，FPGA)、数字处理器(Digital Signal Processor，DSP)、嵌入式设备等。
所述计算机设备可以是桌上型计算机、笔记本、掌上电脑及云端服务器等计算设备。所述计算机设备可以与用户通过键盘、鼠标、遥控器、触摸板或声控设备等方式进行人机交互。
所述存储器41至少包括一种类型的可读存储介质，所述可读存储介质包括闪存、硬盘、多媒体卡、卡型存储器(例如，SD或DX存储器等)、随机访问存储器(RAM)、静态随机访问存储器(SRAM)、只读存储器(ROM)、电可擦除可编程只读存储器(EEPROM)、可编程只读存储器(PROM)、磁性存储器、磁盘、光盘等。在一些实施例中，所述存储器41可以是所述计算机设备4的内部存储单元，例如该计算机设备4的硬盘或内存。在另一些实施例中，所述存储器41也可以是所述计算机设备4的外部存储设备，例如该计算机设备4上配备的插接式硬盘，智能存储卡(Smart Media Card,SMC)，安全数字(Secure Digital,SD)卡，闪存卡(Flash Card)等。当然，所述存储器41还可以既包括所述计算机设备4的内部存储单元也包括其外部存储设备。本实施例中，所述存储器41通常用于存储安装于所述计算机设备4的操作系统和各类应用软件，例如模糊图像修正的方法的计算机可读指令等。此外，所述存储器41还可以用于暂时地存储已经输出或者将要输出的各类数据。
所述处理器42在一些实施例中可以是中央处理器(Central Processing Unit，CPU)、控制器、微控制器、微处理器、或其他数据处理芯片。该处理器42通常用于控制所述计算机设备4的总体操作。本实施例中，所述处理器42用于运行所述存储器41中存储的计算机可读指令或者处理数据，例如运行所述模糊图像修正的方法的计算机可读指令。
所述网络接口43可包括无线网络接口或有线网络接口，该网络接口43通常用于在所述计算机设备4与其他电子设备之间建立通信连接。
本申请公开了一种计算机设备，属于人工智能技术领域。由于目标视频中的图像帧往往是连续的，因而尽管抽取的目标图像帧质量并不优秀，但是通过目标图像帧上一时刻的图像推断出目标图像模糊部分所包含的信息，再将这些信息与目标图像进行融合，即可恢复该张图像的信息，达到图像修复的目的。根据上述思想本申请通过预设的图像检测策略在目标视频的所有图像帧中确定待修复图像，将待修复图像导入预先训练好的图像形变预测模型，得到待修复图像的形变矩阵，通过构建形变矩阵优化函数，并基于形变矩阵优化函数对形变矩阵进行迭代更新，得到图像融合权重矩阵，基于图像融合权重矩阵对待修复图像和待修复图像的前一帧图像进行图像融合，得到修复图像，并用得到的修复图像替换待修复图。本申请的技术方案可以同时针对物体高速移动或者低速移动所导致的图像模糊进行修正，具有较高的适应性，且不会增大系统资源的占用，也不会增大系统的开销，适合部署在移动终端。
本申请还提供了另一种实施方式，即提供一种计算机可读存储介质，所述计算机可读存储介质可以是非易失性，也可以是易失性，所述计算机可读存储介质存储有计算机可读指令，所述计算机可读指令可被至少一个处理器执行，以使所述至少一个处理器执行如上述的模糊图像修正的方法的步骤。
本申请公开了一种存储介质，属于人工智能技术领域。由于目标视频中的图像帧往往是连续的，因而尽管抽取的目标图像帧质量并不优秀，但是通过目标图像帧上一时刻的图像推断出目标图像模糊部分所包含的信息，再将这些信息与目标图像进行融合，即可恢复该张图像的信息，达到图像修复的目的。根据上述思想本申请通过预设的图像检测策略在目标视频的所有图像帧中确定待修复图像，将待修复图像导入预先训练好的图像形变预测模型，得到待修复图像的形变矩阵，通过构建形变矩阵优化函数，并基于形变矩阵优化函数对形变矩阵进行迭代更新，得到图像融合权重矩阵，基于图像融合权重矩阵对待修复图像和待修复图像的前一帧图像进行图像融合，得到修复图像，并用得到的修复图像替换待修复图。本申请的技术方案可以同时针对物体高速移动或者低速移动所导致的图像模糊进行修正，具有较高的适应性，且不会增大系统资源的占用，也不会增大系统的开销，适合部署在移动终端。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到上述实施例方法可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件，但很多情况下前者是更佳的实施方式。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质(如ROM/RAM、磁碟、光盘)中，包括若干指令用以使得一台终端设备(可以是手机，计算机，服务器，空调器，或者网络设备等)执行本申请各个实施例所述的方法。
显然，以上所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例，附图中给出了本申请的较佳实施例，但并不限制本申请的专利范围。本申请可以以许多不同的形式来实现，相反地，提供这些实施例的目的是使对本申请的公开内容的理解更加透彻全面。尽管参照前述实施例对本申请进行了详细的说明，对于本领域的技术人员来而言，其依然可以对前述各具体实施方式所记载的技术方案进行修改，或者对其中部分技术特征进行等效替换。凡是利用本申请说明书及附图内容所做的等效结构，直接或间接运用在其他相关的技术领域，均同理在本申请专利保护范围之内。
Claims (20)
- 一种模糊图像修正的方法，包括：接收图像修复指令，获取与所述图像修复指令相对应的目标视频，并获取所述目标视频中的所有图像帧；基于预设的图像检测策略在所述目标视频的所有图像帧中确定待修复图像；将检测到所述待修复图像导入预先训练好的图像形变预测模型，得到所述待修复图像的形变矩阵；构建形变矩阵优化函数，基于所述形变矩阵优化函数对所述形变矩阵进行迭代更新，得到图像融合权重矩阵；基于所述图像融合权重矩阵对所述待修复图像和所述待修复图像的前一帧图像进行图像融合，得到修复图像，并通过所述修复图像替换所述待修复图像。
- 如权利要求1所述的模糊图像修正的方法，其中，在所述接收图像修复指令，获取与所述图像修复指令相对应的目标视频，并获取所述目标视频中的所有图像帧的步骤之后，还包括：分别对所述所有图像帧进行灰度化处理，得到每一张所述图像帧的灰度图像；对每一张所述图像帧的灰度图像进行归一化处理，得到每一张所述图像帧对应的归一化图像矩阵。
- 如权利要求1所述的模糊图像修正的方法，其中，所述基于预设的图像检测策略在所述目标视频的所有图像帧中确定待修复图像的步骤之前，还包括：基于预设的标准图像指标对所述目标视频的第一张图像帧进行检测，判断所述第一张图像帧是否为标准图像帧；若所述第一张图像帧不是标准图像帧，则重新截取与所述图像修复指令相对应的目标视频。
- 如权利要求3所述的模糊图像修正的方法，其中，所述基于预设的图像检测策略在所述目标视频的所有图像帧中确定待修复图像的步骤，具体包括：从所述目标视频中获取时序相连的两个图像帧，其中，所述时序相连的两个图像帧为当前图像帧和所述当前图像帧前一时刻的前一图像帧；将所述当前图像帧和所述前一图像帧导入预设的缺陷检测器，得到图像缺陷检测结果，基于所述图像缺陷检测结果确定所述当前图像帧是否为待修复图像。
- 如权利要求4所述的模糊图像修正的方法，其中，所述将所述当前图像帧和所述前一图像帧导入预设的缺陷检测器，得到图像缺陷检测结果，基于所述图像缺陷检测结果确定所述当前图像帧是否为待修复图像的步骤，具体包括：计算所述当前图像帧与所述前一图像帧之间的相似度，得到第一相似度；比对所述第一相似度与预设相似度阈值的大小；若所述第一相似度小于预设相似度阈值，则确定所述当前图像帧为待修复图像。
- 如权利要求1至5任意一项所述的模糊图像修正的方法，其中，所述构建形变矩阵优化函数，基于所述形变矩阵优化函数对所述形变矩阵进行迭代更新，得到图像融合权重矩阵的步骤，具体包括：对所述形变矩阵进行归一化处理，得到初始融合权重矩阵；计算所述待修复图像的权重因子，并基于所述权重因子构建所述形变矩阵优化函数；基于牛顿法对所述形变矩阵优化函数进行迭代；通过迭代后的所述形变矩阵优化函数对所述初始融合权重矩阵进行优化，得到融合权重矩阵。
- 如权利要求5所述的模糊图像修正的方法，其中，在所述基于所述图像融合权重矩阵对所述待修复图像和所述待修复图像的前一帧图像进行图像融合，得到修复图像，并通过所述修复图像替换所述待修复图之后，还包括：将得到的所述修复图像帧和所述前一图像帧导入预设的缺陷检测器，计算所述修复图像帧与所述前一图像帧之间的相似度，得到第二相似度；比对所述第二相似度与预设相似度阈值的大小；若所述第二相似度小于预设相似度阈值，则持续对所述修复图像帧进行图像修复，直至所述第二相似度大于或等于预设相似度阈值为止。
- 一种模糊图像修正的装置，包括：目标视频获取模块，用于接收图像修复指令，获取与所述图像修复指令相对应的目标视频，并获取所述目标视频中的所有图像帧；待修复图像确定模块，用于基于预设的图像检测策略在所述目标视频的所有图像帧中确定待修复图像；形变矩阵获取模块，用于将检测到所述待修复图像导入预先训练好的图像形变预测模型，得到所述待修复图像的形变矩阵；形变矩阵优化模块，用于构建形变矩阵优化函数，基于所述形变矩阵优化函数对所述形变矩阵进行迭代更新，得到图像融合权重矩阵；图像修复模块，用于基于所述图像融合权重矩阵对所述待修复图像和所述待修复图像的前一帧图像进行图像融合，得到修复图像，并通过所述修复图像替换所述待修复图像。
- 一种计算机设备，包括存储器和处理器，所述存储器中存储有计算机可读指令，所述处理器执行所述计算机可读指令时实现如下所述的模糊图像修正的方法：接收图像修复指令，获取与所述图像修复指令相对应的目标视频，并获取所述目标视频中的所有图像帧；基于预设的图像检测策略在所述目标视频的所有图像帧中确定待修复图像；将检测到所述待修复图像导入预先训练好的图像形变预测模型，得到所述待修复图像的形变矩阵；构建形变矩阵优化函数，基于所述形变矩阵优化函数对所述形变矩阵进行迭代更新，得到图像融合权重矩阵；基于所述图像融合权重矩阵对所述待修复图像和所述待修复图像的前一帧图像进行图像融合，得到修复图像，并通过所述修复图像替换所述待修复图像。
- 如权利要求9所述的计算机设备，其中，在所述接收图像修复指令，获取与所述图像修复指令相对应的目标视频，并获取所述目标视频中的所有图像帧的步骤之后，还包括：分别对所述所有图像帧进行灰度化处理，得到每一张所述图像帧的灰度图像；对每一张所述图像帧的灰度图像进行归一化处理，得到每一张所述图像帧对应的归一化图像矩阵。
- 如权利要求9所述的计算机设备，其中，所述基于预设的图像检测策略在所述目标视频的所有图像帧中确定待修复图像的步骤之前，还包括：基于预设的标准图像指标对所述目标视频的第一张图像帧进行检测，判断所述第一张图像帧是否为标准图像帧；若所述第一张图像帧不是标准图像帧，则重新截取与所述图像修复指令相对应的目标视频。
- 如权利要求11所述的计算机设备，其中，所述基于预设的图像检测策略在所述目标视频的所有图像帧中确定待修复图像的步骤，具体包括：从所述目标视频中获取时序相连的两个图像帧，其中，所述时序相连的两个图像帧为当前图像帧和所述当前图像帧前一时刻的前一图像帧；将所述当前图像帧和所述前一图像帧导入预设的缺陷检测器，得到图像缺陷检测结果，基于所述图像缺陷检测结果确定所述当前图像帧是否为待修复图像。
- 如权利要求12所述的计算机设备，其中，所述将所述当前图像帧和所述前一图像帧导入预设的缺陷检测器，得到图像缺陷检测结果，基于所述图像缺陷检测结果确定所述当前图像帧是否为待修复图像的步骤，具体包括：计算所述当前图像帧与所述前一图像帧之间的相似度，得到第一相似度；比对所述第一相似度与预设相似度阈值的大小；若所述第一相似度小于预设相似度阈值，则确定所述当前图像帧为待修复图像。
- 如权利要求9至13任意一项所述的计算机设备，其中，所述构建形变矩阵优化函数，基于所述形变矩阵优化函数对所述形变矩阵进行迭代更新，得到图像融合权重矩阵的步骤，具体包括：对所述形变矩阵进行归一化处理，得到初始融合权重矩阵；计算所述待修复图像的权重因子，并基于所述权重因子构建所述形变矩阵优化函数；基于牛顿法对所述形变矩阵优化函数进行迭代；通过迭代后的所述形变矩阵优化函数对所述初始融合权重矩阵进行优化，得到融合权重矩阵。
- 一种计算机可读存储介质，所述计算机可读存储介质上存储有计算机可读指令，所述计算机可读指令被处理器执行时实现如下所述的模糊图像修正的方法：接收图像修复指令，获取与所述图像修复指令相对应的目标视频，并获取所述目标视频中的所有图像帧；基于预设的图像检测策略在所述目标视频的所有图像帧中确定待修复图像；将检测到所述待修复图像导入预先训练好的图像形变预测模型，得到所述待修复图像的形变矩阵；构建形变矩阵优化函数，基于所述形变矩阵优化函数对所述形变矩阵进行迭代更新，得到图像融合权重矩阵；基于所述图像融合权重矩阵对所述待修复图像和所述待修复图像的前一帧图像进行图像融合，得到修复图像，并通过所述修复图像替换所述待修复图像。
- 如权利要求15所述的计算机可读存储介质，其中，在所述接收图像修复指令，获取与所述图像修复指令相对应的目标视频，并获取所述目标视频中的所有图像帧的步骤之后，还包括：分别对所述所有图像帧进行灰度化处理，得到每一张所述图像帧的灰度图像；对每一张所述图像帧的灰度图像进行归一化处理，得到每一张所述图像帧对应的归一化图像矩阵。
- 如权利要求15所述的计算机可读存储介质，其中，所述基于预设的图像检测策略在所述目标视频的所有图像帧中确定待修复图像的步骤之前，还包括：基于预设的标准图像指标对所述目标视频的第一张图像帧进行检测，判断所述第一张图像帧是否为标准图像帧；若所述第一张图像帧不是标准图像帧，则重新截取与所述图像修复指令相对应的目标视频。
- 如权利要求17所述的计算机可读存储介质，其中，所述基于预设的图像检测策略在所述目标视频的所有图像帧中确定待修复图像的步骤，具体包括：从所述目标视频中获取时序相连的两个图像帧，其中，所述时序相连的两个图像帧为当前图像帧和所述当前图像帧前一时刻的前一图像帧；将所述当前图像帧和所述前一图像帧导入预设的缺陷检测器，得到图像缺陷检测结果，基于所述图像缺陷检测结果确定所述当前图像帧是否为待修复图像。
- 如权利要求18所述的计算机可读存储介质，其中，所述将所述当前图像帧和所述前一图像帧导入预设的缺陷检测器，得到图像缺陷检测结果，基于所述图像缺陷检测结果确定所述当前图像帧是否为待修复图像的步骤，具体包括：计算所述当前图像帧与所述前一图像帧之间的相似度，得到第一相似度；比对所述第一相似度与预设相似度阈值的大小；若所述第一相似度小于预设相似度阈值，则确定所述当前图像帧为待修复图像。
- 如权利要求15至19任意一项所述的计算机可读存储介质，其中，所述构建形变矩阵优化函数，基于所述形变矩阵优化函数对所述形变矩阵进行迭代更新，得到图像融合权重矩阵的步骤，具体包括：对所述形变矩阵进行归一化处理，得到初始融合权重矩阵；计算所述待修复图像的权重因子，并基于所述权重因子构建所述形变矩阵优化函数；基于牛顿法对所述形变矩阵优化函数进行迭代；通过迭代后的所述形变矩阵优化函数对所述初始融合权重矩阵进行优化，得到融合权重矩阵。
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011609062.4A CN112634170B (zh) | 2020-12-30 | 2020-12-30 | 一种模糊图像修正的方法、装置、计算机设备及存储介质 |
| CN202011609062.4 | 2020-12-30 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2022142009A1 true WO2022142009A1 (zh) | 2022-07-07 |
Family
ID=75286638
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2021/090420 WO2022142009A1 (zh) | 2020-12-30 | 2021-04-28 | 一种模糊图像修正的方法、装置、计算机设备及存储介质 |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN112634170B (zh) |
| WO (1) | WO2022142009A1 (zh) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115829524A (zh) * | 2023-02-15 | 2023-03-21 | 山东铁路投资控股集团有限公司 | 隐蔽工程的远程审核方法、装置、设备及可读存储介质 |
| CN116069646A (zh) * | 2023-02-10 | 2023-05-05 | 中国人民解放军海军航空大学 | 一种多目标测试性优化的测试集确定方法及系统 |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112634170B (zh) * | 2020-12-30 | 2023-11-21 | 平安科技（深圳）有限公司 | 一种模糊图像修正的方法、装置、计算机设备及存储介质 |
| CN113344826B (zh) * | 2021-07-06 | 2023-12-19 | 北京锐安科技有限公司 | 图像处理方法、装置、电子设备及存储介质 |
| CN114283091B (zh) * | 2021-12-27 | 2022-08-09 | 国网黑龙江省电力有限公司伊春供电公司 | 基于视频融合的电力设备图像恢复系统 |
| CN114913468A (zh) * | 2022-06-16 | 2022-08-16 | 阿里巴巴（中国）有限公司 | 对象修复方法、修复评估方法、电子设备和存储介质 |
| CN115100209B (zh) * | 2022-08-28 | 2022-11-08 | 电子科技大学 | 一种基于摄像机的图像质量的修正方法及修正系统 |
| CN115760609B (zh) * | 2022-11-14 | 2023-07-28 | 林雅霞 | 一种图像优化方法及系统 |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101976429A (zh) * | 2010-10-27 | 2011-02-16 | 南京大学 | 基于游弋图像的水面鸟瞰图成像方法 |
| CN103279939A (zh) * | 2013-04-27 | 2013-09-04 | 北京工业大学 | 一种图像拼接处理系统 |
| US20190035149A1 (en) * | 2015-08-14 | 2019-01-31 | Metail Limited | Methods of generating personalized 3d head models or 3d body models |
| CN109670503A (zh) * | 2018-12-19 | 2019-04-23 | 北京旷视科技有限公司 | 标识检测方法、装置和电子系统 |
| CN111553864A (zh) * | 2020-04-30 | 2020-08-18 | 深圳市商汤科技有限公司 | 图像修复方法及装置、电子设备和存储介质 |
| CN112634170A (zh) * | 2020-12-30 | 2021-04-09 | 平安科技（深圳）有限公司 | 一种模糊图像修正的方法、装置、计算机设备及存储介质 |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6583823B1 (en) * | 1997-08-01 | 2003-06-24 | Sony Corporation | Methods, apparatuses, and mediums for repairing a pixel associated with motion-picture processes |
| KR101080846B1 (ko) * | 2010-03-30 | 2011-11-07 | 중앙대학교 산학협력단 | 보정 행렬의 추정에 의한 영상의 색상 손상 보정장치 및 방법 |
| CN103595980B (zh) * | 2013-10-25 | 2015-08-05 | 西安电子科技大学 | 基于轮廓非局部均值的色彩滤波阵列图像去马赛克方法 |
| CN108921898B (zh) * | 2018-06-28 | 2021-08-10 | 北京旷视科技有限公司 | 摄像机位姿确定方法、装置、电子设备和计算机可读介质 |
| WO2020231016A1 (en) * | 2019-05-16 | 2020-11-19 | Samsung Electronics Co., Ltd. | Image optimization method, apparatus, device and storage medium |
| CN111415319A (zh) * | 2020-03-20 | 2020-07-14 | 成科扬 | 一种基于时序网络预测与金字塔融合的视频图像修复方法 |
-
2020
- 2020-12-30 CN CN202011609062.4A patent/CN112634170B/zh active Active
-
2021
- 2021-04-28 WO PCT/CN2021/090420 patent/WO2022142009A1/zh active Application Filing
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101976429A (zh) * | 2010-10-27 | 2011-02-16 | 南京大学 | 基于游弋图像的水面鸟瞰图成像方法 |
| CN103279939A (zh) * | 2013-04-27 | 2013-09-04 | 北京工业大学 | 一种图像拼接处理系统 |
| US20190035149A1 (en) * | 2015-08-14 | 2019-01-31 | Metail Limited | Methods of generating personalized 3d head models or 3d body models |
| CN109670503A (zh) * | 2018-12-19 | 2019-04-23 | 北京旷视科技有限公司 | 标识检测方法、装置和电子系统 |
| CN111553864A (zh) * | 2020-04-30 | 2020-08-18 | 深圳市商汤科技有限公司 | 图像修复方法及装置、电子设备和存储介质 |
| CN112634170A (zh) * | 2020-12-30 | 2021-04-09 | 平安科技（深圳）有限公司 | 一种模糊图像修正的方法、装置、计算机设备及存储介质 |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116069646A (zh) * | 2023-02-10 | 2023-05-05 | 中国人民解放军海军航空大学 | 一种多目标测试性优化的测试集确定方法及系统 |
| CN116069646B (zh) * | 2023-02-10 | 2023-08-11 | 中国人民解放军海军航空大学 | 一种多目标测试性优化的测试集确定方法及系统 |
| CN115829524A (zh) * | 2023-02-15 | 2023-03-21 | 山东铁路投资控股集团有限公司 | 隐蔽工程的远程审核方法、装置、设备及可读存储介质 |
| CN115829524B (zh) * | 2023-02-15 | 2023-05-09 | 山东铁路投资控股集团有限公司 | 隐蔽工程的远程审核方法、装置、设备及可读存储介质 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112634170B (zh) | 2023-11-21 |
| CN112634170A (zh) | 2021-04-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2022142009A1 (zh) | 一种模糊图像修正的方法、装置、计算机设备及存储介质 | |
| WO2021143267A1 (zh) | 基于图像检测的细粒度分类模型处理方法、及其相关设备 | |
| CN109118447B (zh) | 一种图片处理方法、图片处理装置及终端设备 | |
| CN108734106B (zh) | 基于对比的快速暴恐视频识别方法 | |
| WO2023035531A1 (zh) | 文本图像超分辨率重建方法及其相关设备 | |
| CN113436100B (zh) | 用于修复视频的方法、装置、设备、介质和产品 | |
| CN114677565B (zh) | 特征提取网络的训练方法和图像处理方法、装置 | |
| WO2023159746A1 (zh) | 基于图像分割的图像抠图方法、装置、计算机设备及介质 | |
| WO2021174877A1 (zh) | 基于智能决策的目标检测模型的处理方法、及其相关设备 | |
| CN112231592A (zh) | 基于图的网络社团发现方法、装置、设备以及存储介质 | |
| CN114663952A (zh) | 对象分类方法、深度学习模型的训练方法、装置和设备 | |
| CN116403083A (zh) | 图像的处理方法、装置、电子设备及存储介质 | |
| EP4343616A1 (en) | Image classification method, model training method, device, storage medium, and computer program | |
| CN111104941B (zh) | 图像方向纠正方法、装置及电子设备 | |
| CN114882321A (zh) | 深度学习模型的训练方法、目标对象检测方法和装置 | |
| WO2022001233A1 (zh) | 基于层次化迁移学习的预标注方法及其相关设备 | |
| WO2024041108A1 (zh) | 图像矫正模型训练及图像矫正方法、装置和计算机设备 | |
| CN112634182A (zh) | 一种基于光场的图像校正方法、装置、设备及存储介质 | |
| CN112633200A (zh) | 基于人工智能的人脸图像比对方法、装置、设备及介质 | |
| US11232616B2 (en) | Methods and systems for performing editing operations on media | |
| WO2022178975A1 (zh) | 基于噪声场的图像降噪方法、装置、设备及存储介质 | |
| WO2022105120A1 (zh) | 图片文字检测方法、装置、计算机设备及存储介质 | |
| CN114863162A (zh) | 对象分类方法、深度学习模型的训练方法、装置和设备 | |
| CN113362249A (zh) | 文字图像合成方法、装置、计算机设备及存储介质 | |
| CN114663418A (zh) | 图像处理的方法及装置、存储介质及电子设备 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 21912771 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 21912771 Country of ref document: EP Kind code of ref document: A1 |