WO2020107214A1 - 图像处理方法及装置 - Google Patents

图像处理方法及装置 Download PDF

Info

Publication number
WO2020107214A1
WO2020107214A1 PCT/CN2018/117692 CN2018117692W WO2020107214A1 WO 2020107214 A1 WO2020107214 A1 WO 2020107214A1 CN 2018117692 W CN2018117692 W CN 2018117692W WO 2020107214 A1 WO2020107214 A1 WO 2020107214A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
information
image block
compensation
compensation information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2018/117692
Other languages
English (en)
French (fr)
Inventor
裴继红
邵旭慧
杨烜
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen University
Original Assignee
Shenzhen University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen University filed Critical Shenzhen University
Priority to PCT/CN2018/117692 priority Critical patent/WO2020107214A1/zh
Priority to US16/586,979 priority patent/US11140419B2/en
Publication of WO2020107214A1 publication Critical patent/WO2020107214A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
    • H04N19/86—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving reduction of coding artifacts, e.g. of blockiness
    • H04N19/865—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving reduction of coding artifacts, e.g. of blockiness with detection of the former encoding block subdivision in decompressed video
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
    • H04N19/86—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving reduction of coding artifacts, e.g. of blockiness
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/105—Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/625—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding using discrete cosine transform [DCT]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/80—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation
    • H04N19/82—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation involving filtering within a prediction loop
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/124—Quantisation
    • H04N19/126—Details of normalisation or weighting functions, e.g. normalisation matrices or variable uniform quantisers

Definitions

  • the invention relates to the technical field of image processing, and in particular to an image processing method and device.
  • infrared imaging technology has been used in various aspects such as biology, medicine, geoscience, and military reconnaissance. Compared with visible light imaging, infrared imaging has the advantages of high resolution, long range, and all-weather work at night or under insufficient light.
  • compression technology is mainly used for storage and transmission, such as image and video compression standards based on block discrete cosine transform (Block Discrete Cosine Transform, BDCT).
  • BDCT Block Discrete Cosine Transform
  • BDCT performs individual quantization and coding in block units, which destroys the correlation of pixels in adjacent blocks. Therefore, at low bit rates, coarse quantization will cause grayscale discontinuities on the boundaries of adjacent blocks, resulting in The obvious block effect seriously reduces the image quality.
  • blockiness removal algorithms are usually used to remove blockiness in infrared images.
  • the current blockiness removal algorithm usually solves the problem of boundary discontinuity caused by blockiness by smoothing the pixels at the boundary of the image block, which causes the problem that the DC component of the image block is not compensated.
  • the main purpose of the embodiments of the present invention is to provide an image processing method and device that can compensate for the DC component that has been lost in an image with blockiness, thereby eliminating the blockiness of the image, and thereby improving the image quality.
  • a first aspect of an embodiment of the present invention provides an image processing method, the method includes: dividing an infrared decompressed image with block effect into image blocks distributed in a rectangular array; acquiring boundary grayscale information of the image block, According to the boundary grayscale information of the image block, determine the boundary drop information between each image block and the adjacent image block, and obtain the drop information of the image block according to the boundary drop information; based on the boundary drop information, the Constructing a target objective optimization function of the infrared decompressed image based on the drop information and preset compensation information; determining a target compensation matrix according to the constrained optimization objective function, each element value in the target compensation matrix being the infrared solution The target value of the compensation information of the image block in the compressed image whose position is the same as the position of the element value in the target compensation matrix, and each element value in the target compensation matrix satisfies the constraint optimization objective function The condition of the minimum value; according to the element value in the target compensation matrix, compensate the DC component of the image block in the infrare
  • a second aspect of an embodiment of the present invention provides an image processing apparatus.
  • the apparatus includes: a dividing unit for dividing an infrared decompressed image with block effect into image blocks distributed in a rectangular array; and an acquiring unit for acquiring The boundary grayscale information of the image block, according to the boundary grayscale information of the image block, determine the boundary drop information of each image block and the adjacent image block, and obtain the drop of the image block according to the boundary drop information Information; a construction unit for constructing a constrained optimization target function of the infrared decompressed image based on the boundary drop information, the drop information and preset compensation information; a determining unit for determining according to the constrained optimization target function A target compensation matrix, each element value in the target compensation matrix is a target value of compensation information for an image block in the infrared decompressed image whose position is the same as the position of the element value in the target compensation matrix, and Each element value in the target compensation matrix satisfies the condition that the constrained optimization objective function obtains the minimum value;
  • the target compensation matrix is determined by constructing a constrained optimization target function, and the element values in the target compensation matrix are used to compensate the image with the same position as the infrared decompressed image and the element value in the target compensation matrix
  • the DC component of the block and then remove the blocking effect present in the infrared decompressed image, instead of smoothing the pixels at the boundary of the image block of the infrared decompressed image, which can compensate for the DC component lost in each image block, and It can adaptively remove the blockiness of decompressed images compressed by various bit rates and improve work efficiency.
  • FIG. 1 is a schematic diagram of an implementation process of the image processing method in the first embodiment provided by the present invention
  • FIG. 2 is a schematic diagram of image blocks of the image processing method in the first embodiment provided by the present invention.
  • FIG. 3 is a schematic diagram of drop information of each image block in the image processing method in the first embodiment provided by the present invention.
  • FIG. 5 is a schematic structural diagram of an image processing device in a second embodiment provided by the present invention.
  • FIG. 1 is a schematic diagram of an implementation process of the image processing method in the first embodiment provided by the present invention. As shown in Figure 1, the method mainly includes the following steps:
  • the terminal uses the BDCT-based image compression standard to compress the infrared image to obtain an infrared compressed image, and then decompress the infrared compressed image to obtain an infrared decompressed image. Since the BDCT-based image compression standard performs individual quantization and coding in units of image blocks, it will destroy the correlation of pixels between adjacent image blocks, so there is a block effect in the infrared decompressed image.
  • the infrared decompressed image of size 8m ⁇ 8n is divided into mn images distributed in a matrix array Block x i,j (1 ⁇ i ⁇ m,1 ⁇ j ⁇ n), where the subscript (i,j) represents the image block located in the i-th row and j-th column in the infrared decompressed image.
  • the terminal obtains the boundary grayscale information of each image block according to the divided infrared decompressed image.
  • each image block is divided into an image block located in the middle of the image, and images located at the four corners of the image One of the block and the image block located at the image boundary.
  • the image block 1 is an image block located in the middle of the image
  • the image blocks 2, 3, 4 and 5 are an upper block, a right block, a lower block and a left block adjacent to the image block 1, respectively.
  • the image block 6 is an image block located at the four corners of the image
  • the image blocks 7 and 8 are the right block and the lower block adjacent to the image block 6, respectively.
  • Both image block 9 and image block 13 are image blocks located at the image boundary.
  • the image block 9 is an image block located on the left border of the image
  • the image blocks 10, 11 and 12 are an upper block, a right block and a lower block adjacent to the image block 9, respectively.
  • the image block 13 is an image block located at the lower boundary of the image
  • the image blocks 14, 15 and 16 are the left block, the upper block and the right block adjacent to the image block 13, respectively.
  • the terminal obtains boundary drop information of each image block according to a preset boundary drop information operation template.
  • a preset boundary drop information operation template For the image block in the middle of the image, four preset boundary drop information operation templates are used to obtain the boundary drop information at the boundary of the middle block and the upper block, the middle block and the lower block, the middle block and the left block, and the middle block and the right block respectively .
  • the image block located at the image boundary the image block located at the upper boundary of the image is taken as an example for description, and the middle block and the lower block, the middle block and the left block, and the middle block and the Boundary drop information of the right block at the boundary. It can be understood that the determination method of the boundary drop information of the image block located at the left border of the image, the image block located at the lower border of the image, and the image block located at the right border of the image is the same as the determination method of the boundary drop information of the image block located at the upper border of the image .
  • boundary drop information for the image blocks located at the four corners of the image, taking the image block located at the upper left corner of the image as an example, two preset boundary drop information calculation templates are used to obtain the middle block and the lower block, and the middle block and the right block at the boundary. Boundary drop information. It can be understood that the boundary drop information of the image block located in the lower left corner of the image, the image block located in the upper right corner of the image, and the image block located in the lower right corner of the image is determined in the same manner as the boundary drop information of the image block located in the upper left corner of the image .
  • the size of the preset boundary drop information operation template corresponds to two rows/column pixels of adjacent boundary blocks. Therefore, the size of the preset boundary drop information operation template is related to the position of the adjacent block relative to the intermediate block.
  • the specific operation process of the preset boundary drop information calculation template is: along the direction of the middle block to the surroundings, first calculate the intra-block difference and the inter-block difference of two adjacent blocks, and then based on the three The first-order difference performs the second-order difference, and finally takes the 1/16 times the cumulative sum of the second-order difference.
  • the average information of the boundary drop between the image block and the adjacent image block is determined according to the information of the boundary drop between the image block and the adjacent image block, which is recorded as ⁇ U,i,j , ⁇ D,i,j , ⁇ L,i ,j , ⁇ R,i,j ⁇ , and use the sum of the average information of the boundary drop between the image block and the adjacent image block as the drop information of the image block, as shown in the following formula:
  • ⁇ i,j ⁇ U,i,j + ⁇ D,i,j + ⁇ L,i,j + ⁇ R,i,j .
  • ⁇ i,j is the drop information of the image block in row i and column j in the infrared decompressed image
  • ⁇ U,i,j is the image block in row i and column j in the infrared decompressed image and the upper block
  • Average information of the boundary drop of ⁇ , ⁇ D,i,j is the average information of the boundary drop of the image block in the i-th row and j-th column of the infrared decompressed image and the lower block
  • ⁇ L,i,j is the i-th of the infrared decompressed image
  • ⁇ R,i,j is the average information about the boundary drop between the image block in row i and j in the infrared decompressed image and the right block.
  • the quantization error mainly comes from the excessive quantization of the DC component and the AC component in the DCT domain.
  • the DC component is the average grayscale of the image block, which represents the overall brightness of the image block. Due to excessive quantization, the adjacent image blocks with similar brightness originally have uneven grayscale drops, which visually appear as block effects. .
  • a constraint optimization target function is constructed to determine a target compensation matrix, and the DC component of the infrared decompressed image is compensated according to the target compensation matrix to eliminate blockiness.
  • the constraint optimization objective function of the infrared decompressed image is constructed based on the boundary drop information, the drop information and the compensation information of the image block.
  • the constrained optimization objective function is as follows:
  • J represents the objective function of constrained optimization
  • E represents the total squared error of the blockiness after compensation
  • C 1 and C 2 represent the overall offset constraint term of the compensation information and the penalty term of the compensation information
  • C 3 and C 4 represent the upper and lower boundaries, respectively
  • the compensation information constraint item and the left and right boundary compensation value constraint items, ⁇ 1 , ⁇ 2 , ⁇ 3 and ⁇ 4 are the first weight parameter, the second weight parameter, the third weight parameter and the fourth weight parameter, respectively, used to adjust C
  • the weights of 1 , C 2 , C 3 and C 4 , ⁇ 1 , ⁇ 2 , ⁇ 3 and ⁇ 4 are all greater than 0.
  • the total square error E of the blockiness after compensation is the square error of the blockiness after the DC component compensation of all image blocks x i,j in the infrared decompressed image The sum is as follows:
  • the block error square error of the image block x i,j after DC component compensation It is the square sum of the blockiness errors of the middle block and its four adjacent blocks above, below, right, and left.
  • the method of obtaining the block effect error at the boundary between the intermediate block and the adjacent block is specifically: the boundary drop information of the intermediate block plus the compensation information of the intermediate block and then the compensation information of the adjacent block minus, the size of which is the same as the boundary of the adjacent block and the intermediate block Blocking errors are opposite to each other.
  • the blockiness error at the boundary between the middle block and the four adjacent blocks above, below, left, and right thereof can be summed to obtain the blockiness error of the middle block.
  • the square of the sum of the compensation information of all image blocks in the infrared decompressed image is taken as the overall offset constraint term C 1 of the compensation information, in order to avoid a single compensation If the information is too large or too small, the square sum of the compensation information of all image blocks in the image is used as the compensation information penalty term C 2 , as shown in the following formula.
  • v i,j is the compensation information of the image block in row i and column j in the infrared decompressed image.
  • the compensation information of the image blocks in the infrared decompressed image may gradually accumulate in a certain direction, resulting in a gradation of the grayscale of the image after compensation.
  • the drop gradients corresponding to the blocks are defined as two compensation value boundary stability constraints C 3 and C 4 respectively , as shown in the following formula.
  • ⁇ m,j is the drop information of the image block in the mth row and jth column in the infrared decompressed image
  • v m,j is the compensation information for the image block in the mth row and jth column in the infrared decompressed image
  • ⁇ 1,j Infrared decompressed image
  • the image block information in the first row and jth column, v 1,j is the compensation information for the image block in the first row and jth column in the infrared decompressed image
  • ⁇ i,n is the infrared
  • the drop information of the image block in the i-th row and n-th column in the decompressed image, v i,n is the compensation information for the image block in the i-th row and n-th column in the infrared decompressed image, and ⁇ i,1 in the infrared decompressed image
  • the drop information of the image block in row i and column 1, v i,1 is
  • the derivative of the compensation information for each image block according to the constraint optimization objective function is zero, as shown in the following formula:
  • the optimization formula of the compensation information of each image block can be determined, and then the iterative optimization formula of the compensation information of each image block can be determined.
  • the difference between the compensation information iterative optimization formula and the compensation information optimization formula of each image block is that the compensation information iterative optimization formula adds an iteration variable k. Therefore, the value of the compensation information obtained after the image block performs the iterative operation according to the iterative optimization formula of the compensation information constrains the condition that the objective function obtains the minimum value.
  • the constrained optimization objective function is used to differentiate the compensation information of each image block, and the derivative of the constrained optimization objective function to the compensation information of each image block is zero to determine the compensation information of each image block
  • the iterative optimization formula includes:
  • the constrained optimization objective function is used to differentiate the compensation information of each image block, and the derivative of the constrained optimization objective function to the compensation information of each image block is zero to determine each An iterative optimization formula for the first compensation information of the image block;
  • the constrained optimization objective function is used to differentiate the compensation information of each image block, and the derivative of the constrained optimization objective function to the compensation information of each image block is zero to determine each An iterative optimization formula for the second compensation information of the image block;
  • the constrained optimization objective function is used to differentiate the compensation information of each image block, and the derivative of the constrained optimization objective function to the compensation information of each image block is zero to determine each Iterative optimization formula for the third compensation information of the image block.
  • the iterative optimization formula of the first compensation information is:
  • the iterative optimization formula of the second compensation information is:
  • the iterative optimization formula of the third compensation information is:
  • the compensation information after performing the k-th iterative calculation for the image block in the i-th row and the j-th column in the infrared decompressed image For the sum of the image block compensation information adjacent to the image block in the i-th row and j-th column in the infrared decompressed image before performing the k-th iterative operation, ⁇ i,j is the i-th line in the infrared decompressed image
  • the drop information of the image blocks in the column, ⁇ 1 , ⁇ 2 , ⁇ 3 and ⁇ 4 are the first weight parameter, the second weight parameter, the third weight parameter and the fourth weight parameter, ⁇ 1 , ⁇ 2 , ⁇ 3 and ⁇ 4 are all greater than 0,
  • V k is the sum of the compensation information of several image blocks before the kth iterative operation, with For the upper and lower boundary compensation gradients and the left and right boundary compensation gradients before performing the k-th iteration operation, D ⁇ r and D ⁇ c are
  • the boundary stability constraint is added, and the compensation value gradient related to the image block is increased accordingly.
  • the compensation information of the image block located at the lower boundary of the image and the image block located in the same column among the image blocks located at the upper boundary of the image are each subjected to first-order difference, and then averaged.
  • the compensation information of the image block located on the right border of the image and the image block located on the same line among the image blocks located on the left border of the image are first-differenced and then averaged.
  • D ⁇ r and D ⁇ c are two constants, D ⁇ r is the average difference between the compensation information of the image block located at the lower boundary of the image and the compensation information of the image block located at the upper boundary of the image, and D ⁇ c is the image block located at the right boundary of the image The average difference between the compensation information and the compensation information of the image block located on the left border of the image.
  • the initial value and iteration value of the compensation information determine the amount of change in compensation information.
  • an initial matrix of compensation information of the infrared decompressed image is obtained, and the element value of the initial matrix of compensation information corresponding to the position of the image block in the infrared decompressed image is the initial value of the compensation information of the image block.
  • the initial matrix of compensation information may be a zero matrix.
  • the terminal obtains the initial value of the weight parameter.
  • the initial values of ⁇ 1 , ⁇ 2 , ⁇ 3 and ⁇ 4 may be 2 , 2 , 0.1 and 0.1, respectively.
  • the iterative value of the compensation information of the image block and the initial value of the compensation information of the image block are subtracted to determine the change amount of the compensation information.
  • the maximum value is selected from the amount of change in compensation information of several image blocks as the maximum amount of change in compensation information of the infrared decompressed image. Since the initial value of the compensation information of the image block is obtained by human initialization, in the process of reversely calculating the compensation information, the iterative value matrix of the compensation information composed of the iterative values of the compensation information of several image blocks and the initial matrix of the compensation information have certain values The deviation is described by the amount of change in compensation information. Among them, the smaller the deviation, the closer the compensation information iterative value matrix is to the stable state.
  • the preset threshold can be set to 0.0001.
  • step 107 a matrix composed of iteration values of the compensation information of the image blocks is a target compensation matrix. If the change amount of the maximum compensation information is less than the preset threshold, the matrix composed of the iteration values of the compensation information of several image blocks reaches a stable state according to the position of the image block in the infrared decompressed image, and the iteration values of the compensation information of several image blocks The formed matrix is the target compensation matrix.
  • step 108 is performed: the iteration value of the compensation information of the image block is used as the new initial value, and the initial value and the compensation based on the compensation information of the image block are executed
  • the information iterative optimization formula performs the steps of iterative operation until the target compensation matrix is determined. If the change amount of the maximum compensation information is not less than the preset threshold, the matrix composed of the iterative values of the compensation information of several image blocks has not reached a stable state, and the matrix composed of the iterative values of the compensation information of several image blocks is continuously iterated. Obtain the target compensation matrix.
  • each element value in the target compensation matrix is the target value of the compensation information of the image block whose position in the infrared decompressed image is the same as the position of the element value in the target compensation matrix, and the Each element value satisfies the condition that the constraint optimization objective function obtains the minimum value.
  • the element value in the target compensation matrix compensate the DC component of the image block in the infrared decompressed image with the same position as the element value in the target compensation matrix, and perform the compensation on the infrared decompressed image Smoothing to eliminate the blockiness of the infrared decompressed image.
  • the average brightness of the image block will increase or decrease.
  • the larger the compensated DC component the greater the average brightness change of the image block, and vice versa, so , Compensate the DC component of the image block at the corresponding position in the infrared decompressed image according to the element value in the target compensation matrix, so that the brightness of the compensated image block is the closest to the brightness of the adjacent image block, thereby reducing the difference between the image blocks Continuity.
  • the image block compensation process can be expressed by the following formula:
  • y i,j compensates the DC component of the image block
  • I is an 8 ⁇ 8 unit matrix, which means that each pixel in the 8 ⁇ 8 image block is compensated for the DC component of the same size.
  • the pixels at the boundary of the image block are used as processing objects, and the pixels at the boundary of the block are selected by using a priori method of quantization to select random weight values to modify the gray value. So as to achieve the effect of image smoothing. As shown in FIG. 4, the left image is the image before processing, and the right image on the same line is the processed image. It can be clearly seen that after processing by the image processing method in this embodiment, the block effect The phenomenon has been greatly improved.
  • the target compensation matrix is determined by constructing a constrained optimization target function, and the element values in the target compensation matrix are used to compensate the image blocks whose positions and element values in the infrared decompressed image are the same in the target compensation matrix DC component, and then remove the blocking effect in the infrared decompressed image, instead of smoothing the pixels at the boundary of the image block of the infrared decompressed image, which can compensate for the DC component lost in each image block, and It can adaptively remove the blockiness of decompressed images compressed by various bit rates, and improve work efficiency.
  • the filtering unit is used to smooth the compensated infrared compressed image to further improve the image quality.
  • FIG. 5 is a schematic structural diagram of an image processing apparatus in a second embodiment provided by the present invention. As shown in Figure 5, the device includes:
  • the dividing unit 201 is used to divide the infrared decompressed image with block effect into image blocks distributed in a rectangular array;
  • the obtaining unit 202 is used to obtain the boundary grayscale information of the image block, determine the boundary drop information of each image block and the adjacent image block according to the boundary grayscale information of the image block, and obtain the drop information of the image block according to the boundary drop information.
  • the construction unit 203 is configured to construct a constraint optimization objective function of the infrared decompressed image based on the boundary fall information, the fall information, and preset compensation information.
  • the determining unit 204 is configured to determine a target compensation matrix according to the constraint optimization target function, each element value in the target compensation matrix is a position in the infrared decompressed image and the element value is in the target compensation matrix The target value of the compensation information of the image block at the same position, and each element value in the target compensation matrix satisfies the condition that the constrained optimization objective function obtains the minimum value.
  • the compensation unit 205 is configured to compensate the DC component of the image block in the infrared decompressed image whose position is the same as the position of the element value in the target compensation matrix according to the element value in the target compensation matrix.
  • the filtering unit 206 is configured to perform smoothing processing on the compensated infrared decompressed image to eliminate the blockiness of the infrared decompressed image.
  • the determining unit 204 is also used to derive the constraint optimization objective function for the compensation information of each image block respectively, and make the derivative of the constraint optimization objective function for the compensation information of each image block to zero, to determine the Iterative optimization formula for compensation information.
  • the determining unit 204 is also used to iteratively optimize the formula based on the compensation information of several image blocks to determine the target compensation matrix.
  • the determining unit 204 is also used to obtain the initial value of the compensation information of the image block, and perform an iterative operation based on the initial value of the compensation information of the image block and the iterative optimization formula of the compensation information to obtain the iterative value of the compensation information of the image block.
  • the initial value and iteration value of the compensation information determine the amount of change in compensation information.
  • the determining unit 204 is further configured to determine the maximum compensation information change amount of the infrared decompressed image according to the compensation information change amount of several image blocks, and determine whether the maximum compensation information change amount is less than a preset threshold.
  • the determining unit 204 is further configured to: if the change amount of the maximum compensation information is less than a preset threshold, the matrix composed of the iteration values of the compensation information of several image blocks is the target compensation matrix.
  • the determining unit 204 is also used to execute the initial value of the compensation information based on the image block and the iterative optimization formula of the compensation information based on the iterative value of the compensation information of the image block as the new initial value if the maximum amount of change in the compensation information is not less than the preset threshold. The steps of iterative operation are performed until the target compensation matrix is determined.
  • the dividing unit 201 is also used to divide each image block into an image block located in the middle of the image, an image block located at the four corners of the image, and an image block located at the image boundary according to the position information of the image block in the infrared decompressed image One of them.
  • the determining unit 204 is also used to differentiate the compensation information of each image block according to the image block located in the middle of the image, and make the derivative of the compensation information of each image block by the constraint optimization target function When it is zero, the iterative optimization formula for the first compensation information of each image block is determined.
  • the determining unit 204 is also used to differentiate the compensation information of each image block based on the image blocks located in the four corners of the image, and to make the derivative of the compensation information of each image block of the constraint optimization target function to zero, Determine the iterative optimization formula for the second compensation information of each image block.
  • the determining unit 204 is also used to differentiate the compensation information of each image block from the constrained optimization objective function according to the image blocks located at the image boundary, and make the derivative of the constrained optimization objective function to the compensation information of each image block to zero, Determine the iterative optimization formula for the third compensation information of each image block.
  • the target compensation matrix is determined by constructing a constrained optimization target function, and the element values in the target compensation matrix are used to compensate the image blocks whose positions and element values in the infrared decompressed image are in the target compensation matrix DC component, and then remove the blocking effect in the infrared decompressed image, instead of smoothing the pixels at the boundary of the image block of the infrared decompressed image, which can compensate for the DC component lost in each image block, and It can adaptively remove the blockiness of decompressed images compressed by various bit rates, and improve work efficiency.
  • the filtering unit is used to smooth the compensated infrared compressed image to further improve the image quality.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Discrete Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Image Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

本发明公开了一种图像处理方法及装置，应用于图像处理技术领域。该方法包括：将存在块效应的红外解压缩图像划分为呈矩形阵列分布的图像块。获取图像块的边界灰度信息，进而确定各图像块与相邻图像块的边界落差信息，进而得到图像块的落差信息。基于边界落差信息、落差信息以及预设的补偿信息构建图像的约束优化目标函数。根据约束优化目标函数确定目标补偿矩阵。目标补偿矩阵中的每个元素值为图像中位置与元素值在目标补偿矩阵中的位置相同的图像块的补偿信息的目标值，且均满足约束优化目标函数取得最小值的条件。根据目标补偿矩阵中的元素值，补偿图像的直流分量，并对补偿后的图像进行平滑处理。该方法可补偿图像中已损失的直流分量。

Description

图像处理方法及装置 技术领域
本发明涉及图像处理技术领域，尤其涉及一种图像处理方法及装置。
背景技术
随着我国科学技术的快速发展，红外成像技术已被应用于生物、医学、地学、军事侦察等各个方面。相比于可见光成像，红外成像具有分辨率高，作用距离远，在夜间或光照不足的情况下全天候工作等优点，但是，由于红外热像仪采集的视频数据量较大，受存储容量与传输带宽的限制，在实际应用中，主要使用压缩技术进行存储与传输，如基于块离散余弦变换(Block Discrete Cosine Transform，BDCT)的图像及视频压缩标准。然而，BDCT以块为单位进行单独量化和编码，破坏了像素在相邻块中的相关性，因此在低比特率时由于粗糙量化会使得相邻块边界上出现灰度不连续的现象，产生明显的块效应，严重降低了图像质量。
因此，为了提高图像质量，通常采用块效应去除算法将红外图像中的块效应去除。但是，目前的块效应去除算法通常是通过平滑图像块边界处的像素点来解决块效应导致的边界不连续问题，这就会存在未对图像块的直流分量进行补偿的问题。
发明内容
本发明实施例的主要目的在于提供一种图像处理方法及装置，可补偿存在块效应的图像中已损失的直流分量，从而消除图像的块效应，进而提高图像质 量。
本发明实施例第一方面提供了一种图像处理方法，所述方法包括：将存在块效应的红外解压缩图像划分为呈矩形阵列分布的图像块；获取所述图像块的边界灰度信息，根据所述图像块的边界灰度信息，确定各所述图像块与相邻图像块的边界落差信息，根据所述边界落差信息得到所述图像块的落差信息；基于所述边界落差信息、所述落差信息以及预设的补偿信息构建所述红外解压缩图像的约束优化目标函数；根据所述约束优化目标函数确定目标补偿矩阵，所述目标补偿矩阵中的每个元素值为所述红外解压缩图像中位置与所述元素值在所述目标补偿矩阵中的位置相同的图像块的补偿信息的目标值，并且所述目标补偿矩阵中的每个元素值均满足所述约束优化目标函数取得最小值的条件；根据所述目标补偿矩阵中的元素值，补偿所述红外解压缩图像中位置与所述元素值在所述目标补偿矩阵中的位置相同的图像块的直流分量，并对补偿后的红外解压缩图像进行平滑处理，以消除所述红外解压缩图像的块效应。
本发明实施例第二方面提供了一种图像处理装置，所述装置包括：划分单元，用于将存在块效应的红外解压缩图像划分为呈矩形阵列分布的图像块；获取单元，用于获取所述图像块的边界灰度信息，根据所述图像块的边界灰度信息，确定各所述图像块与相邻图像块的边界落差信息，根据所述边界落差信息得到所述图像块的落差信息；构建单元，用于基于所述边界落差信息、所述落差信息以及预设的补偿信息构建所述红外解压缩图像的约束优化目标函数；确定单元，用于根据所述约束优化目标函数确定目标补偿矩阵，所述目标补偿矩阵中的每个元素值为所述红外解压缩图像中位置与所述元素值在所述目标补偿矩阵中的位置相同的图像块的补偿信息的目标值，并且所述目标补偿矩阵中的每个元素值均满足所述约束优化目标函数取得最小值的条件；补偿单元，用于根据所述目标补偿矩阵中的元素值，补偿所述红外解压缩图像中位置与所述元素值在所述目标补偿矩阵中的位置相同的图像块的直流分量；滤波单元，用于 对补偿后的红外解压缩图像进行平滑处理，以消除所述红外解压缩图像的块效应。
从上述实施例可知，通过构建约束优化目标函数来确定目标补偿矩阵，利用目标补偿矩阵中的元素值来补偿与红外解压缩图像中位置与元素值在所述目标补偿矩阵中的位置相同的图像块的直流分量，进而去除红外解压缩图像中存在的块效应，而不是通过对红外解压缩图像的图像块的边界处的像素点进行平滑处理得到，可补偿各图像块损失的直流分量、并且可自适应去除各种比特率压缩的解压缩图像的块效应，提高工作效率。
附图说明
图1是本发明提供的第一实施例中的图像处理方法的实现流程示意图；
图2是本发明提供的第一实施例中的图像处理方法的图像块示意图；
图3是本发明提供的第一实施例中的图像处理方法中各图像块的落差信息示意图；
图4是本发明提供的第一实施例中的图像处理方法的对比图；
图5是本发明提供的第二实施例中的图像处理装置的结构示意图。
具体实施方式
为使得本发明的发明目的、特征、优点能够更加的明显和易懂，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而非全部实施例。基于本发明中的实施例，本领域技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
请参阅图1，图1是本发明提供的第一实施例中的图像处理方法的实现流程示意图。如图1所示，该方法主要包括以下步骤：
101、将存在块效应的红外解压缩图像划分为呈矩形阵列分布的图像块。
具体的，终端利用基于BDCT的图像压缩标准对红外图像进行压缩，得到红外压缩图像，接着将红外压缩图像进行解压缩，得到红外解压缩图像。由于基于BDCT的图像压缩标准以图像块为单位进行单独量化和编码，会破坏像素在相邻图像块之间的相关性，因此红外解压缩图像中存在块效应。
示例性的，若基于BDCT的图像压缩标准是对大小为8×8的图像块进行单独量化和编码的，则将尺寸为8m×8n的红外解压缩图像划分为mn个呈矩阵阵列分布的图像块x i,j(1≤i≤m,1≤j≤n)，其中，下标(i,j)表示位于在红外解压缩图像中第i行第j列的图像块。
102、获取该图像块的边界灰度信息，根据该图像块的边界灰度信息，确定各该图像块与相邻图像块的边界落差信息，根据该边界落差信息得到该图像块的落差信息。
具体的，终端根据划分后的红外解压缩图像获取各图像块的边界灰度信息。
进一步地，获取图像块的边界灰度信息之后，包括：根据该图像块在该红外解压缩图像中的位置信息，将各该图像块分别划分为位于图像中间的图像块，位于图像四角的图像块以及位于图像边界的图像块中的一种。
具体的，图像块的位置示意图如图2所示。其中，图像块1为位于图像中间的图像块，图像块2、3、4和5分别是与图像块1相邻的上边块、右边块、下边块及左边块。图像块6为位于图像四角的图像块，图像块7和8分别是与图像块6相邻的右边块和下边块。图像块9和图像块13均为位于图像边界的图像块。其中，图像块9为位于图像左边界的图像块，图像块10、11和12分别是与图像块9相邻的上边块、右边块及下边块。图像块13为位于图像下边界的图像块，图像块14、15和16分别是与图像块13相邻的左边块、上边块及右边块。
其中，终端根据预设边界落差信息运算模板获取各图像块的边界落差信息。 对于位于图像中间的图像块，利用四个预设边界落差信息运算模板分别获取中间块与上边块、中间块与下边块、中间块与左边块与中间块与右边块在边界处的边界落差信息。
其中，对于位于图像边界的图像块，以位于图像上边界的图像块为例进行说明，利用三个预设边界落差信息运算模板分别获取中间块与下边块、中间块与左边块以及中间块与右边块在边界处的边界落差信息。可以理解的，位于图像左边界的图像块、位于图像下边界的图像块以及位于图像右边界的图像块的边界落差信息的确定方式与位于图像上边界的图像块的边界落差信息的确定方式相同。
其中，对于位于图像四角的图像块，以位于图像左上角的图像块为例进行说明，利用两个预设边界落差信息运算模板分别获取中间块与下边块以及中间块与右边块在边界处的边界落差信息。可以理解的，位于图像左下角的图像块、位于图像右上角的图像块以及位于图像右下角的图像块的边界落差信息的确定方式与位于图像左上角的图像块的边界落差信息的确定方式相同。
其中，预设边界落差信息运算模板的大小对应于相邻边界块的两行/列像素点，因此，预设边界落差信息运算模板的大小与相邻块相对于中间块的位置有关。如图3所示，该预设边界落差信息运算模板的具体运算过程为：沿着中间块向四周的方向，首先计算两个相邻块的块内差分和块间差分，随后基于三个一阶差分进行二阶差分，最后取二阶差分的1/16倍累加和。
其中，根据图像块与相邻图像块的边界落差信息，确定图像块与相邻图像块的边界落差平均信息，记作{ε U,i,j,ε D,i,j,ε L,i,j,ε R,i,j}，并将图像块与相邻图像块的边界落差平均信息的和作为图像块的落差信息，如下式所示：
ε i,j＝ε U,i,j+ε D,i,j+ε L,i,j+ε R,i,j。
式中，ε i,j为红外解压缩图像中第i行第j列的图像块的落差信息，ε U,i,j为红外解压缩图像中第i行第j列的图像块与上边块的边界落差平均信息，ε D,i,j为红 外解压缩图像中第i行第j列的图像块与下边块的边界落差平均信息，ε L,i,j为红外解压缩图像中第i行第j列的图像块与左边块的边界落差平均信息，ε R,i,j为红外解压缩图像中第i行第j列的图像块与右边块的边界落差平均信息。
103、基于该边界落差信息、该落差信息以及预设的补偿信息构建该红外解压缩图像的约束优化目标函数。
具体的，在图像处理过程中，量化误差主要来源于DCT域直流分量和交流分量的过度量化。其中，直流分量为图像块的灰度平均值，代表图像块的整体亮度，由于过度量化，使得原本亮度相近的相邻图像块，出现明暗不均匀的灰度落差，在视觉上表现为块效应。为了降低相邻图像块的亮度差异，本实施例通过构建约束优化目标函数以确定目标补偿矩阵，并根据目标补偿矩阵对红外解压缩图像的直流分量进行补偿，以消除块效应。
其中，基于该图像块的边界落差信息、落差信息以及补偿信息构建该红外解压缩图像的约束优化目标函数。该约束优化目标函数如下式所示：
Figure PCTCN2018117692-appb-000001
式中，J表示约束优化目标函数，E表示补偿后块效应的总平方误差，C 1和C 2分别表示补偿信息总体偏移约束项和补偿信息惩罚项，C 3和C 4分别表示上下边界补偿信息约束项和左右边界补偿值约束项，λ 1、λ 2、λ 3和λ 4分别为第一权重参数、第二权重参数、第三权重参数和第四权重参数，分别用于调节C 1、C 2、C 3和C 4的权重，λ 1、λ 2、λ 3和λ 4均大于0。
在本实施例中，补偿后块效应总平方误差E为红外解压缩图像中所有图像块x i,j经直流分量补偿后块效应的平方误差
Figure PCTCN2018117692-appb-000002
的和，如下式所示：
Figure PCTCN2018117692-appb-000003
其中，经直流分量补偿后图像块x i,j的块效应平方误差
Figure PCTCN2018117692-appb-000004
为中间块与其上下左右四个相邻块的块效应误差的平方和。中间块与相邻块边界处块效应误差的获取方法具体为：中间块的边界落差信息加上中间块的补偿信息再减去相邻 块的补偿信息，其大小与相邻块和中间块边界处块效应误差互为相反数。中间块与其上下左右四个相邻块边界处的块效应误差求和可获得中间块的块效应误差。
在本实施例中，为了避免得到的补偿信息整体过大或者过小，将红外解压缩图像中所有图像块的补偿信息的和的平方作为补偿信息总体偏移约束项C 1，为了避免单个补偿信息过大或过小，将图像中所有图像块的补偿信息的平方和作为补偿信息惩罚项C 2，如下式所示。
Figure PCTCN2018117692-appb-000005
Figure PCTCN2018117692-appb-000006
式中，v i,j为红外解压缩图像中第i行第j列的图像块的补偿信息。
其中，红外解压缩图像中图像块的补偿信息可能沿着某一方向逐渐累积，造成补偿后图像灰度出现渐变，为克服这一问题，将补偿后图像上下边界的图像块和左右边界的图像块所对应的落差梯度分别定义为两个补偿值边界稳定约束项C 3和C 4，如下式所示。
Figure PCTCN2018117692-appb-000007
Figure PCTCN2018117692-appb-000008
式中，ε m,j为红外解压缩图像中第m行第j列的图像块的落差信息，v m,j为红外解压缩图像中第m行第j列的图像块的补偿信息，ε 1,j红外解压缩图像中第1行第j列的图像块的落差信息，v 1,j为红外解压缩图像中第1行第j列的图像块的补偿信息，ε i,n为红外解压缩图像中第i行第n列的图像块的落差信息，v i,n为红外解压缩图像中第i行第n列的图像块的补偿信息，ε i,1红外解压缩图像中第i行第1列的图像块的落差信息，v i,1为红外解压缩图像中第i行第1列的图像块的补偿信息。
104、将该约束优化目标函数分别对各该图像块的补偿信息进行求导，并令该约束优化目标函数对各该图像块的补偿信息的导数为零，确定各该图像块的补偿信息迭代优化公式。
具体的，根据约束优化目标函数对各图像块的补偿信息的导数为零，如下式所示：
Figure PCTCN2018117692-appb-000009
根据上式可确定各图像块的补偿信息优化公式，进而确定各图像块的补偿信息迭代优化公式。其中，各图像块的补偿信息迭代优化公式与补偿信息优化公式的区别在于，补偿信息迭代优化公式多了迭代变量k。因此，图像块根据补偿信息迭代优化公式进行迭代运算后得到的补偿信息的值均约束目标函数取得最小值的条件。
进一步地，对于位于红外解压缩图像中不同位置的图像块，其相邻图像块的个数及方向都不相同。因此，对于位于红外解压缩图像中不同位置的图像块的补偿信息进行迭代计算的思路相同，但补偿信息迭代优化公式并不相同。
因此，该将该约束优化目标函数分别对各该图像块的补偿信息进行求导，并令该约束优化目标函数对各该图像块的补偿信息的导数为零，确定各该图像块的补偿信息迭代优化公式包括：
根据该位于图像中间的图像块，将该约束优化目标函数分别对各该图像块的补偿信息进行求导，并令该约束优化目标函数对各该图像块的补偿信息的导数为零，确定各该图像块的第一补偿信息迭代优化公式；
根据该位于图像四角的图像块，将该约束优化目标函数分别对各该图像块的补偿信息进行求导，并令该约束优化目标函数对各该图像块的补偿信息的导数为零，确定各该图像块的第二补偿信息迭代优化公式；
根据该位于图像边界的图像块，将该约束优化目标函数分别对各该图像块的补偿信息进行求导，并令该约束优化目标函数对各该图像块的补偿信息的导 数为零，确定各该图像块的第三补偿信息迭代优化公式。
进一步地，该第一补偿信息迭代优化公式为：
Figure PCTCN2018117692-appb-000010
该第二补偿信息迭代优化公式为：
Figure PCTCN2018117692-appb-000011
该第三补偿信息迭代优化公式为：
Figure PCTCN2018117692-appb-000012
Figure PCTCN2018117692-appb-000013
式中，
Figure PCTCN2018117692-appb-000014
为该红外解压缩图像中第i行第j列的图像块进行第k次迭代运算后的补偿信息，
Figure PCTCN2018117692-appb-000015
为进行第k次迭代运算前与该红外解压缩图像中第i行第j列的图像块相邻的图像块补偿信息的和，ε i,j为该红外解压缩图像中第i行第j列的图像块的落差信息，λ 1、λ 2、λ 3和λ 4分别为第一权重参数、第二权重参数、第三权重参数和第四权重参数，λ 1、λ 2、λ 3和λ 4均大于0，V k为进行第k次迭代运算前的若干该图像块的补偿信息的和，
Figure PCTCN2018117692-appb-000016
和
Figure PCTCN2018117692-appb-000017
为进行第k次迭代运算前的上下边界补偿梯度和左右边界补偿梯度，D εr和D εc为上下边界落差差分信息和左右边界落差差分信息，
Figure PCTCN2018117692-appb-000018
为该红外解压缩图像中第i行第j列的图像块进行第k次迭代运算前的补偿信息。
其中，对于位于图像边界上的图像块，加入边界稳定约束项，并相应增加了与图像块相关的补偿值梯度。
其中，
Figure PCTCN2018117692-appb-000019
为进行第k次迭代运算前位于图像下边界的图像块的补偿信息与位于图像上边界的图像块中位于同一列的图像块各进行一阶差分，然后取平均值得到。同理，
Figure PCTCN2018117692-appb-000020
位进行第k次迭代运算前位于图像右边界的图像块的补偿信 息与位于图像左边界的图像块中位于同一行的图像块各进行一阶差分，然后取平均值得到。
其中，D εr和D εc为两个常数，D εr是位于图像下边界的图像块的补偿信息与位于图像上边界的图像块的补偿信息平均差分，D εc是位于图像右边界的图像块的补偿信息与位于图像左边界的图像块的补偿信息平均差分。
105、获取该图像块的补偿信息的初始值，并基于该图像块的补偿信息的初始值和该补偿信息迭代优化公式进行迭代运算，得到该图像块的补偿信息的迭代值，根据该图像块的补偿信息的初始值和迭代值确定补偿信息变化量。
具体的，获取红外解压缩图像的补偿信息初始矩阵，与图像块在红外解压缩图像中的位置相对应的补偿信息初始矩阵的元素值为该图像块的补偿信息的初始值。在实际应用中，补偿信息初始矩阵可为零矩阵。同时，终端获取权重参数的初始值。在实际应用中，λ 1、λ 2、λ 3和λ 4的初始值可分别为2、2、0.1和0.1。其中，将该图像块的补偿信息的迭代值和该图像块的补偿信息的初始值相减可确定补偿信息变化量。
可选的，依照如图2所示的箭头方向依次对图像块进行迭代运算，待所有的图像块均依照各自的补偿信息迭代优化公式进行一次迭代计算后，计算各图像块的补偿信息变化量。
106、根据若干该图像块的补偿信息变化量，确定该红外解压缩图像的最大补偿信息变化量，并判断该最大补偿信息变化量是否小于预设阈值。
具体的，从若干图像块的补偿信息变化量中选取最大值，作为该红外解压缩图像的最大补偿信息变化量。由于图像块的补偿信息的初始值时人为初始化得到的，因此在逆向计算补偿信息的过程中，若干图像块的补偿信息的迭代值构成的补偿信息迭代值矩阵与补偿信息初始矩阵的值存在一定的偏差，该偏差利用补偿信息变化量描述。其中，偏差越小，说明补偿信息迭代值矩阵越接近稳定状态。因此，通过判断最大补偿信息变化量是否小于预设阈值的方式来确 定进行迭代运算后，若干图像块的补偿信息的迭代值构成的矩阵是否达到稳定状态。可选的，预设阈值可设置为0.0001。
其中，若该最大补偿信息变化量小于该预设阈值，则执行步骤107：若干该图像块的补偿信息的迭代值构成的矩阵为目标补偿矩阵。若该最大补偿信息变化量小于该预设阈值，依据图像块在红外解压缩图像中的位置，若干图像块的补偿信息的迭代值构成的矩阵达到稳定状态，若干图像块的补偿信息的迭代值构成的矩阵为目标补偿矩阵。
其中，若该最大补偿信息变化量不小于该预设阈值，则执行步骤108：将该图像块的补偿信息的迭代值作为新初始值，执行基于该图像块的补偿信息的初始值和该补偿信息迭代优化公式进行迭代运算的步骤，直至确定该目标补偿矩阵。若该最大补偿信息变化量不小于该预设阈值，若干图像块的补偿信息的迭代值构成的矩阵未达到稳定状态，若干图像块的补偿信息的迭代值构成的矩阵通过不断迭代的方式，最终得到目标补偿矩阵。
其中，该目标补偿矩阵中的每个元素值为该红外解压缩图像中位置与该元素值在该目标补偿矩阵中的位置相同的图像块的补偿信息的目标值，并且该目标补偿矩阵中的每个元素值均满足该约束优化目标函数取得最小值的条件。
109、根据该目标补偿矩阵中的元素值，补偿该红外解压缩图像中位置与该元素值在该目标补偿矩阵中的位置相同的图像块的直流分量，并对补偿后的红外解压缩图像进行平滑处理，以消除该红外解压缩图像的块效应。
具体的，对于图像块补偿一个非零值的直流分量，该图像块其平均亮度会增大或减小，补偿的直流分量越大，则图像块的平均亮度变化越大，反之亦然，因此，根据目标补偿矩阵中的元素值补偿红外解压缩图像中相应位置的图像块的直流分量，使得补偿后的图像块的亮度与相邻图像块的亮度最相近，从而降低图像块之间的不连续程度。
其中，图像块补偿过程可用下式进行表示：
y i,j＝x i,j+v i,jI i＝1,...,m,j＝1,...,n。
式中，，y i,j补偿直流分量后的图像块，I为8×8单位矩阵，表示对8×8图像块中每个像素点补偿相同大小的直流分量。
其中，为避免平滑后的图像整体模糊，本实施例中以图像块边界处的像素点为处理对象，对块边界处的像素点使用量化先验方式选取随机权重值进行灰度值的修正，从而达到图像平滑的效果。如图4所示，左侧的图像为处理前的图像，位于同一行的右侧图像为处理后的图像，可清楚看到，经过本实施例中的图像处理方法进行处理后，块效应的现象获得极大改善。
在本发明实施例中，通过构建约束优化目标函数来确定目标补偿矩阵，利用目标补偿矩阵中的元素值来补偿与红外解压缩图像中位置与元素值在目标补偿矩阵中的位置相同的图像块的直流分量，进而去除红外解压缩图像中存在的块效应，而不是通过对红外解压缩图像的图像块的边界处的像素点进行平滑处理得到，可补偿各图像块损失的直流分量、并且可自适应去除各种比特率压缩的解压缩图像的块效应，提高工作效率。并且利用滤波单元对补偿后的红外压缩图像进行平滑处理，进一步提高图像质量。
参照图5，图5是本发明提供的第二实施例中的图像处理装置的结构示意图。如图5所示，该装置包括：
划分单元201，用于将存在块效应的红外解压缩图像划分为呈矩形阵列分布的图像块；
获取单元202，用于获取图像块的边界灰度信息，根据图像块的边界灰度信息，确定各图像块与相邻图像块的边界落差信息，根据边界落差信息得到图像块的落差信息。
构建单元203，用于基于所述边界落差信息、所述落差信息以及预设的补偿信息构建所述红外解压缩图像的约束优化目标函数。
确定单元204，用于根据所述约束优化目标函数确定目标补偿矩阵，所述 目标补偿矩阵中的每个元素值为所述红外解压缩图像中位置与所述元素值在所述目标补偿矩阵中的位置相同的图像块的补偿信息的目标值，并且所述目标补偿矩阵中的每个元素值均满足所述约束优化目标函数取得最小值的条件。
补偿单元205，用于根据所述目标补偿矩阵中的元素值，补偿所述红外解压缩图像中位置与所述元素值在所述目标补偿矩阵中的位置相同的图像块的直流分量。
滤波单元206，用于对补偿后的红外解压缩图像进行平滑处理，以消除所述红外解压缩图像的块效应。
进一步地，确定单元204，还用于将约束优化目标函数分别对各图像块的补偿信息进行求导，并令约束优化目标函数对各图像块的补偿信息的导数为零，确定各图像块的补偿信息迭代优化公式。
确定单元204，还用于基于若干图像块的补偿信息迭代优化公式，确定目标补偿矩阵。
确定单元204，还用于获取图像块的补偿信息的初始值，并基于图像块的补偿信息的初始值和补偿信息迭代优化公式进行迭代运算，得到图像块的补偿信息的迭代值，根据图像块的补偿信息的初始值和迭代值确定补偿信息变化量。
确定单元204，还用于根据若干图像块的补偿信息变化量，确定红外解压缩图像的最大补偿信息变化量，并判断最大补偿信息变化量是否小于预设阈值。
确定单元204，还用于若最大补偿信息变化量小于预设阈值，则若干图像块的补偿信息的迭代值构成的矩阵为目标补偿矩阵。
确定单元204，还用于若最大补偿信息变化量不小于预设阈值，则将图像块的补偿信息的迭代值作为新初始值，执行基于图像块的补偿信息的初始值和补偿信息迭代优化公式进行迭代运算的步骤，直至确定目标补偿矩阵。
进一步地，划分单元201，还用于根据图像块在红外解压缩图像中的位置信息，将各图像块分别划分为位于图像中间的图像块，位于图像四角的图像块 以及位于图像边界的图像块中的一种。
进一步地，确定单元204，还用于根据位于图像中间的图像块，将约束优化目标函数分别对各图像块的补偿信息进行求导，并令约束优化目标函数对各图像块的补偿信息的导数为零，确定各图像块的第一补偿信息迭代优化公式。
确定单元204，还用于根据位于图像四角的图像块，将约束优化目标函数分别对各图像块的补偿信息进行求导，并令约束优化目标函数对各图像块的补偿信息的导数为零，确定各图像块的第二补偿信息迭代优化公式。
确定单元204，还用于根据位于图像边界的图像块，将约束优化目标函数分别对各图像块的补偿信息进行求导，并令约束优化目标函数对各图像块的补偿信息的导数为零，确定各图像块的第三补偿信息迭代优化公式。
本实施例未尽之细节，请参阅前述图1至图4所示的实施例的描述，此处不再赘述。
在本发明实施例中，通过构建约束优化目标函数来确定目标补偿矩阵，利用目标补偿矩阵中的元素值来补偿与红外解压缩图像中位置与元素值在目标补偿矩阵中的位置相同的图像块的直流分量，进而去除红外解压缩图像中存在的块效应，而不是通过对红外解压缩图像的图像块的边界处的像素点进行平滑处理得到，可补偿各图像块损失的直流分量、并且可自适应去除各种比特率压缩的解压缩图像的块效应，提高工作效率。并且利用滤波单元对补偿后的红外压缩图像进行平滑处理，进一步提高图像质量。
在上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述的部分，可以参见其他实施例的相关描述。
以上为本发明所提供的图像处理方法及装置的描述，对于本领域的一般技术人员，依据本发明实施例的思想，在具体实施方式及应用范围上均有改变之处，综上，本说明书内容不应理解为对本发明的限制。

Claims (10)

  1. 一种图像处理方法，其特征在于，所述方法包括：
    将存在块效应的红外解压缩图像划分为呈矩形阵列分布的图像块；
    获取所述图像块的边界灰度信息，根据所述图像块的边界灰度信息，确定各所述图像块与相邻图像块的边界落差信息，根据所述边界落差信息得到所述图像块的落差信息；
    基于所述边界落差信息、所述落差信息以及预设的补偿信息构建所述红外解压缩图像的约束优化目标函数；
    根据所述约束优化目标函数确定目标补偿矩阵，所述目标补偿矩阵中的每个元素值为所述红外解压缩图像中位置与所述元素值在所述目标补偿矩阵中的位置相同的图像块的补偿信息的目标值，并且所述目标补偿矩阵中的每个元素值均满足所述约束优化目标函数取得最小值的条件；
    根据所述目标补偿矩阵中的元素值，补偿所述红外解压缩图像中位置与所述元素值在所述目标补偿矩阵中的位置相同的图像块的直流分量，并对补偿后的红外解压缩图像进行平滑处理，以消除所述红外解压缩图像的块效应。
  2. 如权利要求1所述的图像处理方法，其特征在于，所述根据所述约束优化目标函数确定目标补偿矩阵包括：
    将所述约束优化目标函数分别对各所述图像块的补偿信息进行求导，并令所述约束优化目标函数对各所述图像块的补偿信息的导数为零，确定各所述图像块的补偿信息迭代优化公式；
    基于若干所述图像块的补偿信息迭代优化公式，确定目标补偿矩阵。
  3. 如权利要求2所述的图像处理方法，其特征在于，所述基于若干所述图像块的补偿信息迭代优化公式，确定目标补偿矩阵包括：
    获取所述图像块的补偿信息的初始值，并基于所述图像块的补偿信息的初始值和所述补偿信息迭代优化公式进行迭代运算，得到所述图像块的补偿信息 的迭代值，根据所述图像块的补偿信息的初始值和迭代值确定补偿信息变化量；
    根据若干所述图像块的补偿信息变化量，确定所述红外解压缩图像的最大补偿信息变化量，并判断所述最大补偿信息变化量是否小于预设阈值；
    若所述最大补偿信息变化量小于所述预设阈值，则若干所述图像块的补偿信息的迭代值构成的矩阵为目标补偿矩阵；
    若所述最大补偿信息变化量不小于所述预设阈值，则将所述图像块的补偿信息的迭代值作为新初始值，执行基于所述图像块的补偿信息的初始值和所述补偿信息迭代优化公式进行迭代运算的步骤，直至确定所述目标补偿矩阵。
  4. 如权利要求2所述的图像处理方法，其特征在于，所述获取所述图像块的边界灰度信息之后，包括：
    根据所述图像块在所述红外解压缩图像中的位置信息，将各所述图像块分别划分为位于图像中间的图像块，位于图像四角的图像块以及位于图像边界的图像块中的一种。
  5. 如权利要求4所述的图像处理方法，其特征在于，所述将所述约束优化目标函数分别对各所述图像块的补偿信息进行求导，并令所述约束优化目标函数对各所述图像块的补偿信息的导数为零，确定各所述图像块的补偿信息迭代优化公式包括：
    根据所述位于图像中间的图像块，将所述约束优化目标函数分别对各所述图像块的补偿信息进行求导，并令所述约束优化目标函数对各所述图像块的补偿信息的导数为零，确定各所述图像块的第一补偿信息迭代优化公式；
    根据所述位于图像四角的图像块，将所述约束优化目标函数分别对各所述图像块的补偿信息进行求导，并令所述约束优化目标函数对各所述图像块的补偿信息的导数为零，确定各所述图像块的第二补偿信息迭代优化公式；
    根据所述位于图像边界的图像块，将所述约束优化目标函数分别对各所述图像块的补偿信息进行求导，并令所述约束优化目标函数对各所述图像块的补 偿信息的导数为零，确定各所述图像块的第三补偿信息迭代优化公式。
  6. 如权利要求5所述的图像处理方法，其特征在于，所述第一补偿信息迭代优化公式为：
    Figure PCTCN2018117692-appb-100001
    所述第二补偿信息迭代优化公式为：
    Figure PCTCN2018117692-appb-100002
    所述第三补偿信息迭代优化公式为：
    Figure PCTCN2018117692-appb-100003
    Figure PCTCN2018117692-appb-100004
    式中，
    Figure PCTCN2018117692-appb-100005
    为所述红外解压缩图像中第i行第j列的图像块进行第k次迭代运算后的补偿信息，
    Figure PCTCN2018117692-appb-100006
    为进行第k次迭代运算前与所述红外解压缩图像中第i行第j列的图像块相邻的图像块补偿信息的和，ε i,j为所述红外解压缩图像中第i行第j列的图像块的落差信息，λ 1、λ 2、λ 3和λ 4分别为第一权重参数、第二权重参数、第三权重参数和第四权重参数，λ 1、λ 2、λ 3和λ 4均大于0，V k为进行第k次迭代运算前的若干所述图像块的补偿信息的和，
    Figure PCTCN2018117692-appb-100007
    和
    Figure PCTCN2018117692-appb-100008
    为进行第k次迭代运算前的上下边界补偿梯度和左右边界补偿梯度，D εr和D εc为上下边界落差差分信息和左右边界落差差分信息，
    Figure PCTCN2018117692-appb-100009
    为所述红外解压缩图像中第i行第j列的图像块进行第k次迭代运算前的补偿信息。
  7. 一种图像处理装置，其特征在于，所述装置包括：
    划分单元，用于将存在块效应的红外解压缩图像划分为呈矩形阵列分布的图像块；
    获取单元，用于获取所述图像块的边界灰度信息，根据所述图像块的边界 灰度信息，确定各所述图像块与相邻图像块的边界落差信息，根据所述边界落差信息得到所述图像块的落差信息；
    构建单元，用于基于所述边界落差信息、所述落差信息以及预设的补偿信息构建所述红外解压缩图像的约束优化目标函数；
    确定单元，用于根据所述约束优化目标函数确定目标补偿矩阵，所述目标补偿矩阵中的每个元素值为所述红外解压缩图像中位置与所述元素值在所述目标补偿矩阵中的位置相同的图像块的补偿信息的目标值，并且所述目标补偿矩阵中的每个元素值均满足所述约束优化目标函数取得最小值的条件；
    补偿单元，用于根据所述目标补偿矩阵中的元素值，补偿所述红外解压缩图像中位置与所述元素值在所述目标补偿矩阵中的位置相同的图像块的直流分量；
    滤波单元，用于对补偿后的红外解压缩图像进行平滑处理，以消除所述红外解压缩图像的块效应。
  8. 如权利要求7所述的图像处理装置，其特征在于，
    所述确定单元，还用于将所述约束优化目标函数分别对各所述图像块的补偿信息进行求导，并令所述约束优化目标函数对各所述图像块的补偿信息的导数为零，确定各所述图像块的补偿信息迭代优化公式；
    所述确定单元，还用于基于若干所述图像块的补偿信息迭代优化公式，确定目标补偿矩阵；
    所述确定单元，还用于获取所述图像块的补偿信息的初始值，并基于所述图像块的补偿信息的初始值和所述补偿信息迭代优化公式进行迭代运算，得到所述图像块的补偿信息的迭代值，根据所述图像块的补偿信息的初始值和迭代值确定补偿信息变化量；
    所述确定单元，还用于根据若干所述图像块的补偿信息变化量，确定所述红外解压缩图像的最大补偿信息变化量，并判断所述最大补偿信息变化量是否 小于预设阈值；
    所述确定单元，还用于若所述最大补偿信息变化量小于所述预设阈值，则若干所述图像块的补偿信息的迭代值构成的矩阵为目标补偿矩阵；
    所述确定单元，还用于若所述最大补偿信息变化量不小于所述预设阈值，则将所述图像块的补偿信息的迭代值作为新初始值，执行基于所述图像块的补偿信息的初始值和所述补偿信息迭代优化公式进行迭代运算的步骤，直至确定所述目标补偿矩阵。
  9. 如权利要求8所述的图像处理装置，其特征在于，
    所述划分单元，还用于根据所述图像块在所述红外解压缩图像中的位置信息，将各所述图像块分别划分为位于图像中间的图像块，位于图像四角的图像块以及位于图像边界的图像块中的一种。
  10. 如权利要求9所述的图像处理装置，其特征在于，
    所述确定单元，还用于根据所述位于图像中间的图像块，将所述约束优化目标函数分别对各所述图像块的补偿信息进行求导，并令所述约束优化目标函数对各所述图像块的补偿信息的导数为零，确定各所述图像块的第一补偿信息迭代优化公式；
    所述确定单元，还用于根据所述位于图像四角的图像块，将所述约束优化目标函数分别对各所述图像块的补偿信息进行求导，并令所述约束优化目标函数对各所述图像块的补偿信息的导数为零，确定各所述图像块的第二补偿信息迭代优化公式；
    所述确定单元，还用于根据所述位于图像边界的图像块，将所述约束优化目标函数分别对各所述图像块的补偿信息进行求导，并令所述约束优化目标函数对各所述图像块的补偿信息的导数为零，确定各所述图像块的第三补偿信息迭代优化公式。
PCT/CN2018/117692 2018-11-27 2018-11-27 图像处理方法及装置 Ceased WO2020107214A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2018/117692 WO2020107214A1 (zh) 2018-11-27 2018-11-27 图像处理方法及装置
US16/586,979 US11140419B2 (en) 2018-11-27 2019-09-28 Image processing method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/117692 WO2020107214A1 (zh) 2018-11-27 2018-11-27 图像处理方法及装置

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/586,979 Continuation US11140419B2 (en) 2018-11-27 2019-09-28 Image processing method and device

Publications (1)

Publication Number Publication Date
WO2020107214A1 true WO2020107214A1 (zh) 2020-06-04

Family

ID=70771019

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/117692 Ceased WO2020107214A1 (zh) 2018-11-27 2018-11-27 图像处理方法及装置

Country Status (2)

Country Link
US (1) US11140419B2 (zh)
WO (1) WO2020107214A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116128759B (zh) * 2023-02-08 2024-01-09 爱芯元智半导体(上海)有限公司 图像的光照补偿方法及其装置

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH10336650A (ja) * 1997-05-15 1998-12-18 Mitsubishi Electric Inf Technol Center America Inc ディジタル信号の周波数逓降変換装置及びディジタル信号の周波数逓降変換方法
CN101459839A (zh) * 2007-12-10 2009-06-17 三星电子株式会社 去块效应滤波方法及实现该方法的装置
CN101605257A (zh) * 2008-06-11 2009-12-16 北京中创信测科技股份有限公司 一种块效应分析方法及系统
CN103108182A (zh) * 2013-01-18 2013-05-15 北京航空航天大学 多源异类无人机侦察图像通用压缩方法

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7539248B2 (en) * 2004-04-29 2009-05-26 Mediatek Incorporation Adaptive de-blocking filtering apparatus and method for MPEG video decoder
US20100027909A1 (en) * 2008-08-04 2010-02-04 The Hong Kong University Of Science And Technology Convex optimization approach to image deblocking

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH10336650A (ja) * 1997-05-15 1998-12-18 Mitsubishi Electric Inf Technol Center America Inc ディジタル信号の周波数逓降変換装置及びディジタル信号の周波数逓降変換方法
CN101459839A (zh) * 2007-12-10 2009-06-17 三星电子株式会社 去块效应滤波方法及实现该方法的装置
CN101605257A (zh) * 2008-06-11 2009-12-16 北京中创信测科技股份有限公司 一种块效应分析方法及系统
CN103108182A (zh) * 2013-01-18 2013-05-15 北京航空航天大学 多源异类无人机侦察图像通用压缩方法

Also Published As

Publication number Publication date
US20200169759A1 (en) 2020-05-28
US11140419B2 (en) 2021-10-05

Similar Documents

Publication Publication Date Title
US7570832B2 (en) Image clean-up and pre-coding
Zhang et al. DMCNN: Dual-domain multi-scale convolutional neural network for compression artifacts removal
CN111866521B (zh) 一种视频图像压缩伪影去除方法
Monika et al. Underwater image compression using energy based adaptive block compressive sensing for IoUT applications
CN108564544A (zh) 基于边缘感知的图像盲去模糊组合稀疏优化方法
CN106296591B (zh) 基于马氏距离的非局部均匀数字图像去噪方法
CN111105357B (zh) 一种失真图像的去失真方法、装置及电子设备
CN110830043A (zh) 一种基于混合加权全变分和非局部低秩的图像压缩感知重构方法
Lin et al. Residual in residual based convolutional neural network in-loop filter for AVS3
Ye et al. Accelir: Task-aware image compression for accelerating neural restoration
CN109495759B (zh) 图像处理方法及装置
WO2020107214A1 (zh) 图像处理方法及装置
Liu et al. Non-local extension of total variation regularization for image restoration
Wu et al. An improved guided filtering algorithm for image enhancement
Abd-Elhafiez Image compression algorithm using a fast curvelet transform
Kimishima et al. Frame adaptive rate control scheme for video compressive sensing
Manjunatha et al. An enhanced video compression approach through RLAH encoding and KDENN algorithms
CN106534866A (zh) 图像压缩方法及装置
CN113784130B (zh) 一种基于梯度模相似度离差最小化的比特分配方法
CN120769045B (zh) 一种基于jpeg图像压缩标准的优化方法和图像质量评估方法
Gomathi et al. A multiresolution image completion algorithm for compressing digital color images
Pei et al. A constrained optimization method for blocking artifact compensation in JPEG infrared image
Ahmadi et al. Lossless Compression of Angiogram Foreground with Visual Quality Preservation of Background
Wang et al. Parts for the whole: the dct norm for extreme visual recovery
Zhang et al. A novel in-loop filtering mechanism of HEVC based on 3D sub-bands and CNN processing

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18941719

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 28/10/2021)

122 Ep: pct application non-entry in european phase

Ref document number: 18941719

Country of ref document: EP

Kind code of ref document: A1