WO2024039332A1 - Partial reconstruction method based on sub-band components of jpeg2000 compressed images - Google Patents

Partial reconstruction method based on sub-band components of jpeg2000 compressed images Download PDF

Info

Publication number
WO2024039332A1
WO2024039332A1 PCT/TR2023/050767 TR2023050767W WO2024039332A1 WO 2024039332 A1 WO2024039332 A1 WO 2024039332A1 TR 2023050767 W TR2023050767 W TR 2023050767W WO 2024039332 A1 WO2024039332 A1 WO 2024039332A1
Authority
WO
WIPO (PCT)
Prior art keywords
band
sub
bands
pixel values
images
Prior art date
Application number
PCT/TR2023/050767
Other languages
French (fr)
Inventor
Berk ARICAN
Erdem Safa AKKUL
Levent ÇARKACIOĞLU
Behçet Uğur TÖREYİN
Original Assignee
Aselsan Elektroni̇k Sanayi̇ Ve Ti̇caret Anoni̇m Şi̇rketi̇
İstanbul Tekni̇k Üni̇versi̇tesi̇
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from TR2022/012873 external-priority patent/TR2022012873A1/en
Application filed by Aselsan Elektroni̇k Sanayi̇ Ve Ti̇caret Anoni̇m Şi̇rketi̇, İstanbul Tekni̇k Üni̇versi̇tesi̇ filed Critical Aselsan Elektroni̇k Sanayi̇ Ve Ti̇caret Anoni̇m Şi̇rketi̇
Publication of WO2024039332A1 publication Critical patent/WO2024039332A1/en

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/63—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding using sub-band based transform, e.g. wavelets

Definitions

  • the invention relates to a method for minimizing the performance degradation of JPEG2000 compressed images and maximizing the benefits of Partial Decompression.
  • JPEG2000 compressed images are that they take up a lot of memory, decompression times are quite long after compression, and the sub-images used in partial decompression of JPEG2000 compressed images do not achieve sufficient performance during image analysis.
  • the partial decompression approach for JPEG2000 compressed images provides speed and space savings, it is not preferred for critical architectures due to the decrease in performance. This results in the system running slower and requiring more memory, as well as the need for components with more processing capacity.
  • the sub-images of the images are extracted and the pre-evaluation pixel value of the original image is found.
  • This value represents the median of the pixel values that the image can take. For example, an 8-bit image is represented by 256 pixels, so the pre-evaluation pixel value will be 128.
  • all subimages are divided into 4x4 segments and the peak pixel in this segment is found.
  • the pixel average of the 4x4 partition is calculated. If this average is greater than the pre-evaluation value, the peak pixel is assigned as the largest pixel value of this segment. If it is below, the smallest pixel value is assigned as the peak pixel.
  • the locations of the peak pixel values are divided into 3x3 areas. When images are compressed, they are stored in memory according to their peak pixel positions. Based on the peak pixel position, images are decompressed and faster decompression is achieved.
  • averaging is used to find the pixel with the largest absolute value in 4x4 sections.
  • the values of different sub-images at the same pixel positions are averaged and a high-frequency average sub-image is created.
  • the invention aims to provide a structure with different technical features that brings a new decompression method in this field, unlike the structures used in the current technique.
  • the primary objective of the invention is to provide a method that minimizes the image analysis performance degradation that occurs during the partial decompression of JPEG2000 compressed images and maximizes the benefits of the partial decompression approach.
  • the objective of the invention is to provide a method that maximizes performance gains by blending the high-frequency HL, LH, and HH sub-bands in JPEG2000 compressed images with the LL sub-band and by including "maximization” and “averaging” processes.
  • the inventive method was applied to the landscape classification task with the DenseNet-121 architecture.
  • a performance gain of approximately 2% was achieved compared to the architecture trained with only the LL subband.
  • the objective of the invention is to smooth the edgecorner information in the image by transferring the information contained in the high-frequency sub-images to the LL band.
  • Figure 1 Illustration of the summary of the Hybrid Maximum Sub-band and Hybrid Average Sub-band methods.
  • R Number of lines of sub-band images.
  • XLL The partially decompressed Low-Low (LL) sub-band of the image. It is a 14 reduction of the original image.
  • the pixel values of the image vary as integers between [0,255],
  • XLH The partially decompressed Low-High (LH) sub-band of the image. This is an image that has been reduced by 14 of the original image and contains detail information only in the vertical axis.
  • the pixel values of the image vary as integers between [-128, +128],
  • XHL The partially decompressed High-Low (HL) sub-band of the image. This is an image that has been reduced by 14 of the original image and contains detail information only in the horizontal axis.
  • the pixel values of the image vary as integers between [-128, +128],
  • XHH The partially decompressed High-High (HH) sub-band of the image. It is an image that has been reduced by 14 of the original image and contains detail information only on the diagonal axis.
  • the pixel values of the image vary as integers between [-128, +128],
  • XMAX The high-band matrix created by the maximization process. Each element is represented by 32 bits.
  • XAVG The high-band matrix created by the averaging process. Each element is represented by 32 bits.
  • HMAX Hybrid maximum sub-band.
  • Min(HMAx) The minimum value of the HMAX band.
  • Max(HMAx) The maximum value of the HMAX band.
  • HAVG Hybrid average sub-band.
  • Min(HAVG) The minimum value of the HAVG band.
  • Max(HAvc) The maximum value of the HAVG band.
  • the invention relates to a method for reducing the degradation in performance of JPEG2000 compressed images, while maximizing the benefits of the partial decompression method (10).
  • a new sub-band is created by blending HL, LH and HH sub-bands with the LL sub-band. Maximization and averaging processes are applied on HL, LH and HH sub-bands. The intermediate sub-bands resulting from the maximization and averaging processes are blended with the LL sub-band.
  • the LL sub-band is updated to be represented by 8 bits and a hybrid sub-band with a size four times smaller than the original image is obtained.
  • the resulting hybrid sub-image has higher performance than the LL sub-band itself when used in image analysis (object classification, object detection, object segmentation, etc.).
  • the invention has been tested for classification on remote sensing landscape images and found to have about 2% higher performance than the LL sub-band.
  • LL band Comprises low-frequency details. It is a low resolution image of the original image.
  • LH band Comprises high-frequency vertical details.
  • HL band Comprises high-frequency horizontal details.
  • HH band Comprises high-frequency diagonal details).
  • the XLL band and the XMAX band are summed as follows to form HMAX.
  • the HMAX band is converted to be represented by 8 bits with the following process.
  • All elements of the XLL band are divided by 256. This way all pixel values have 32 bit double values with a real number distribution between [0,1], 5.
  • the XLL band and the XAVG band are summed as follows to form HAVG.
  • the HAVG band is converted to be represented by 8 bits with the following process.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Compression Of Band Width Or Redundancy In Fax (AREA)

Abstract

The invention relates to a method for minimizing the performance degradation of JPEG2000 compressed images and maximizing the benefits of the partial decompression approach. In the inventive method, a new sub-image is created by blending the HL, LH, and HH sub-images with the LL band. Maximization and averaging processes are applied on HL, LH, and HH sub-images. The intermediate images resulting from the maximization and averaging processes are blended with the LL band. The LL band is updated to be represented by 8 bits and a hybrid sub-image with a size four times smaller than the original image is obtained. The resulting hybrid sub-band performs better than the LL sub-band when used in image analysis (object classification, object detection, object segmentation, etc.). The invention has been tested for classifying remote sensing scene images and found to have about 2% higher performance than the LL sub-band.

Description

PARTIAL RECONSTRUCTION METHOD BASED ON SUB-BAND COMPONENTS OF JPEG2000 COMPRESSED IMAGES
Technical Field
The invention relates to a method for minimizing the performance degradation of JPEG2000 compressed images and maximizing the benefits of Partial Decompression.
State of the Art
Disadvantages of JPEG2000 compressed images are that they take up a lot of memory, decompression times are quite long after compression, and the sub-images used in partial decompression of JPEG2000 compressed images do not achieve sufficient performance during image analysis. Although the partial decompression approach for JPEG2000 compressed images provides speed and space savings, it is not preferred for critical architectures due to the decrease in performance. This results in the system running slower and requiring more memory, as well as the need for components with more processing capacity.
In the application numbered JP5175820B2, the sub-images of the images are extracted and the pre-evaluation pixel value of the original image is found. (This value represents the median of the pixel values that the image can take. For example, an 8-bit image is represented by 256 pixels, so the pre-evaluation pixel value will be 128.) When compressing images, all subimages are divided into 4x4 segments and the peak pixel in this segment is found. When calculating the peak pixel, the pixel average of the 4x4 partition is calculated. If this average is greater than the pre-evaluation value, the peak pixel is assigned as the largest pixel value of this segment. If it is below, the smallest pixel value is assigned as the peak pixel. The locations of the peak pixel values are divided into 3x3 areas. When images are compressed, they are stored in memory according to their peak pixel positions. Based on the peak pixel position, images are decompressed and faster decompression is achieved.
In the above-mentioned application, averaging is used to find the pixel with the largest absolute value in 4x4 sections. In the inventive method, unlike this application, the values of different sub-images at the same pixel positions are averaged and a high-frequency average sub-image is created.
Consequently, an improvement in the related technical field has become necessary due to the drawbacks mentioned above and the inadequacy of the existing solutions. Objective of the Invention
The invention aims to provide a structure with different technical features that brings a new decompression method in this field, unlike the structures used in the current technique.
The primary objective of the invention is to provide a method that minimizes the image analysis performance degradation that occurs during the partial decompression of JPEG2000 compressed images and maximizes the benefits of the partial decompression approach.
The objective of the invention is to provide a method that maximizes performance gains by blending the high-frequency HL, LH, and HH sub-bands in JPEG2000 compressed images with the LL sub-band and by including "maximization" and "averaging" processes.
In tests on landscape images obtained by remote sensing, the inventive method was applied to the landscape classification task with the DenseNet-121 architecture. A performance gain of approximately 2% was achieved compared to the architecture trained with only the LL subband.
Rather than increasing the resolution, the objective of the invention is to smooth the edgecorner information in the image by transferring the information contained in the high-frequency sub-images to the LL band.
The structural and characteristic features of the invention and all its advantages will be more clearly understood from the following figures and the detailed description with references to these figures. Therefore, the evaluation should be made with reference to these figures and the detailed description.
Description of the Figures
Figure 1, Illustration of the summary of the Hybrid Maximum Sub-band and Hybrid Average Sub-band methods.
The drawings are not necessarily to scale and details that are not necessary to understand the present invention may be omitted. Furthermore, elements that are at least substantially identical or have at least substantially identical functions are indicated by the same number.
Description of Part References
10. Partial decompression method
20. Average sub-band method
21 . Hybrid average sub-band method
30. Maximum sub-band method
31. Hybrid maximum sub-band method Abbreviations
X: Fully decompressed version of the image.
R: Number of lines of sub-band images.
C: Number of columns of sub-band images.
XLL: The partially decompressed Low-Low (LL) sub-band of the image. It is a 14 reduction of the original image. The pixel values of the image vary as integers between [0,255],
XLH: The partially decompressed Low-High (LH) sub-band of the image. This is an image that has been reduced by 14 of the original image and contains detail information only in the vertical axis. The pixel values of the image vary as integers between [-128, +128],
XHL: The partially decompressed High-Low (HL) sub-band of the image. This is an image that has been reduced by 14 of the original image and contains detail information only in the horizontal axis. The pixel values of the image vary as integers between [-128, +128],
XHH: The partially decompressed High-High (HH) sub-band of the image. It is an image that has been reduced by 14 of the original image and contains detail information only on the diagonal axis. The pixel values of the image vary as integers between [-128, +128],
XMAX: The high-band matrix created by the maximization process. Each element is represented by 32 bits.
XAVG: The high-band matrix created by the averaging process. Each element is represented by 32 bits.
HMAX : Hybrid maximum sub-band.
Min(HMAx) : The minimum value of the HMAX band.
Max(HMAx) : The maximum value of the HMAX band.
HAVG : Hybrid average sub-band.
Min(HAVG) : The minimum value of the HAVG band.
Max(HAvc) : The maximum value of the HAVG band.
HH : High-High
LH : Low-High
HL : High-Low
LL : Low-Low
Detailed Description of the Invention
In this detailed description, the preferred embodiments of the invention are described only for the purpose of better understanding the subject matter and without limitation.
The invention relates to a method for reducing the degradation in performance of JPEG2000 compressed images, while maximizing the benefits of the partial decompression method (10). In the inventive method, a new sub-band is created by blending HL, LH and HH sub-bands with the LL sub-band. Maximization and averaging processes are applied on HL, LH and HH sub-bands. The intermediate sub-bands resulting from the maximization and averaging processes are blended with the LL sub-band. The LL sub-band is updated to be represented by 8 bits and a hybrid sub-band with a size four times smaller than the original image is obtained. The resulting hybrid sub-image has higher performance than the LL sub-band itself when used in image analysis (object classification, object detection, object segmentation, etc.). The invention has been tested for classification on remote sensing landscape images and found to have about 2% higher performance than the LL sub-band.
In the inventive method, images compressed with the JPEG2000 algorithm are partially decompressed to access sub-bands four times smaller than the original size. This results in XLL, XLH, XHL and XHH. (LL band: Comprises low-frequency details. It is a low resolution image of the original image. LH band: Comprises high-frequency vertical details. HL band: Comprises high-frequency horizontal details. HH band: Comprises high-frequency diagonal details).
The following steps are then followed for the two different algorithms.
Hybrid Maximum Sub-Band Method (31)
1 . The absolute value of the pixel values in the XLH , XHL and XHH bands are taken. This way, the distribution of pixel values in these bands is reflected between [0,+128],
2. The XLH, XHL and XHH bands are combined with the maximum pixel extraction to create
Figure imgf000006_0001
3. All elements of the XMAX matrix are divided by 128. Thus, all pixel values are real numbers with 32-bit double values distributed between [0,1],
4. All elements of the XLL band are divided by 256. Thus, all pixel values are real numbers with 32-bit double values distributed between [0,1],
5. The XLL band and the XMAX band are summed as follows to form HMAX.
Figure imgf000006_0002
6. The HMAX band is converted to be represented by 8 bits with the following process.
Figure imgf000006_0003
Hybrid Average Sub-Band Method (21)
1. 128 is added to all pixel values of the XLH, XHL and XHH bands. This results in an integer distribution of pixel values in these bands in the range [0,255],
2. The
Figure imgf000007_0004
and
Figure imgf000007_0005
bands are averaged and XAVG is generated as follows.
Figure imgf000007_0003
3. All elements of the XAVG matrix are divided by 255. This way all pixel values have 32 bit double values distributed as real numbers between [0,1],
4. All elements of the XLL band are divided by 256. This way all pixel values have 32 bit double values with a real number distribution between [0,1], 5. The XLL band and the XAVG band are summed as follows to form HAVG.
Figure imgf000007_0002
6. The HAVG band is converted to be represented by 8 bits with the following process.
Figure imgf000007_0001

Claims

CLAIMS 1. A method that reduces the performance degradation in JPEG2000 compressed images and provides more efficiency with the benefits of partial decompression method (10), and is characterized in that it comprises the process steps of:
• Partial decompressing of images compressed with the JPEG2000 algorithm to access their sub-bands at 4 times the original size and obtaining XLL, XLH, XHL and XHH,
• With Hybrid Maximum Sub-Band Method (31):
> Taking the absolute value of the pixel values in the XLH, XHL and XHH bands and projecting the distribution of pixel values in these bands as integers between [0,+128],
> Forming the XMAX by combining the XLH, XHL and XHH bands with the maximum pixel retrieval process,
Figure imgf000008_0001
> Dividing all elements of the XMAX matrix by 128 and having all pixel values as 32-bit double values distributed as real numbers between [0,1],
> Dividing all elements of the XLL band by 256 and having all pixel values as 32-bit double values distributed as real numbers between [0,1],
> Forming the HMAX by combining the XLL band and the XMAX band,
Figure imgf000008_0002
> Converting the HMAX band to be represented by 8 bits with the following process,
Figure imgf000008_0003
• With Hybrid Average Sub-Band Method (21):
> Adding 128 to all pixel values of XLH , XHL and XHH bands and obtaining the distribution of pixel values in these bands as integers in the range [0,255],
> Averaging the XLH , XHL and XHH bands and forming the XAVG,
Figure imgf000009_0001
> Dividing all elements of the XAVG matrix by 255 and having all pixel values as
32-bit double values distributed as real numbers between [0,1],
> Dividing all elements of the XLL band by 256 and having all pixel values as
32-bit double values distributed as real numbers between [0,1],
> Forming the HAVG by combining the XLL band and the XAVG band,
Figure imgf000009_0002
> Converting the HAVG band to be represented by 8 bits with the following process,
Figure imgf000009_0003
PCT/TR2023/050767 2022-08-15 2023-08-03 Partial reconstruction method based on sub-band components of jpeg2000 compressed images WO2024039332A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
TR2022012873 2022-08-15
TR2022/012873 TR2022012873A1 (en) 2022-08-15 Partial Reconstruction Method Based on Subband Components of JPEG2000 Compressed Images

Publications (1)

Publication Number Publication Date
WO2024039332A1 true WO2024039332A1 (en) 2024-02-22

Family

ID=89942132

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/TR2023/050767 WO2024039332A1 (en) 2022-08-15 2023-08-03 Partial reconstruction method based on sub-band components of jpeg2000 compressed images

Country Status (1)

Country Link
WO (1) WO2024039332A1 (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102867187A (en) * 2012-07-04 2013-01-09 西安电子科技大学 NSST (NonsubsampledShearlet Transform) domain MRF (Markov Random Field) and adaptive threshold fused remote sensing image change detection method
CN103207999A (en) * 2012-11-07 2013-07-17 中国矿业大学(北京) Method and system for coal and rock boundary dividing based on coal and rock image feature extraction and classification and recognition
CN107292256A (en) * 2017-06-14 2017-10-24 西安电子科技大学 Depth convolved wavelets neutral net expression recognition method based on secondary task

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102867187A (en) * 2012-07-04 2013-01-09 西安电子科技大学 NSST (NonsubsampledShearlet Transform) domain MRF (Markov Random Field) and adaptive threshold fused remote sensing image change detection method
CN103207999A (en) * 2012-11-07 2013-07-17 中国矿业大学(北京) Method and system for coal and rock boundary dividing based on coal and rock image feature extraction and classification and recognition
CN107292256A (en) * 2017-06-14 2017-10-24 西安电子科技大学 Depth convolved wavelets neutral net expression recognition method based on secondary task

Non-Patent Citations (6)

* Cited by examiner, † Cited by third party
Title
ARICAN BERK ET AL: "Compressed domain classification of remote sensing scene images based on sub-band data fusion", PROCEEDINGS OF THE SPIE, SPIE, US, vol. 12267, 26 October 2022 (2022-10-26), pages 122670A - 122670A, XP060167509, ISSN: 0277-786X, ISBN: 978-1-5106-5738-0, DOI: 10.1117/12.2635862 *
LI X ET AL: "Detecting faces in the wavelet compressed domain", VISUAL COMMUNICATIONS AND IMAGE PROCESSING; 12-7-2005 - 15-7-2005; BEIJING,, 12 July 2005 (2005-07-12), XP030080922 *
SALIM LAHMIRI ET AL: "Classification of brain MRI using the LH and HL wavelet transform sub-bands", CIRCUITS AND SYSTEMS (ISCAS), 2011 IEEE INTERNATIONAL SYMPOSIUM ON, IEEE, 15 May 2011 (2011-05-15), pages 1025 - 1028, XP031997806, ISBN: 978-1-4244-9473-6, DOI: 10.1109/ISCAS.2011.5937743 *
TANG JIEXIONG ET AL: "Compressed-Domain Ship Detection on Spaceborne Optical Image Using Deep Neural Network and Extreme Learning Machine", IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, IEEE, USA, vol. 53, no. 3, 1 March 2015 (2015-03-01), pages 1174 - 1185, XP011559177, ISSN: 0196-2892, [retrieved on 20140916], DOI: 10.1109/TGRS.2014.2335751 *
TAO GUIHUA ET AL: "Incorporating Discrete Wavelet Transformation Decomposition Convolution into Deep Network to Achieve Light Training", 7 September 2021, TOPICS IN CRYPTOLOGY - CT-RSA 2020 : THE CRYPTOGRAPHERS' TRACK AT THE RSA CONFERENCE 2020, SAN FRANCISCO, CA, USA, FEBRUARY 24-28, 2020, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, PAGE(S) 307 - 318, XP047607137 *
YAO-HONG TSAI ET AL: "Wavelet-Based Image Fusion by Adaptive Decomposition", INTELLIGENT SYSTEMS DESIGN AND APPLICATIONS, 2008. ISDA '08. EIGHTH INTERNATIONAL CONFERENCE ON, IEEE, PISCATAWAY, NJ, USA, 26 November 2008 (2008-11-26), pages 283 - 287, XP031368505, ISBN: 978-0-7695-3382-7 *

Similar Documents

Publication Publication Date Title
CN100510773C (en) Single satellite remote sensing image small target super resolution ratio reconstruction method
US7437013B2 (en) Directional spatial video noise reduction
US7684632B2 (en) Estimating image compression quantization parameter values
CN100510772C (en) Small target super resolution reconstruction method for remote sensing image
JP4097873B2 (en) Image compression method and image compression apparatus for multispectral image
CN112465846B (en) Cloud-containing remote sensing image compression method based on filling strategy
CN115984110A (en) Swin-transform-based second-order spectral attention hyperspectral image super-resolution method
WO2024039332A1 (en) Partial reconstruction method based on sub-band components of jpeg2000 compressed images
CN108156461B (en) Bayer image compression method and device
CN116797462B (en) Real-time video super-resolution reconstruction method based on deep learning
CN111385582A (en) Hyperspectral remote sensing image lossless compression algorithm based on spectrum-space joint prediction
CN116170589A (en) Image compression method and system based on dimension-increasing bilateral filtering enhancement
CN1874421A (en) Device for eliminating flicker noise between frames in image sequence
CN111107360B (en) Spectrum-space dimension combined hyperspectral image lossless compression method and system
CN113542771A (en) Video high-efficiency compression processing method based on content weight
TR2022012873A1 (en) Partial Reconstruction Method Based on Subband Components of JPEG2000 Compressed Images
US20020039448A1 (en) Image transformation method and apparatus, and storage medium
Zhu et al. Lossless compression for hyperspectral images based on adaptive band selection and adaptive predictor selection
Belyaev et al. Lossless image compression algorithm based on haar transform
CN117915087A (en) Multispectral remote sensing image compression method, storage medium and device based on Bayer encoding
Narasimhulu et al. Hybrid LWT and DCT based High-Quality Image Compression
Song et al. The research of a new Auto Target Recognition directed Image compression
Fernàndez et al. Region based KLT for multispectral image compression
CN115396670B (en) Image data compression method for local processing
Zemliachenko et al. Peculiarities of hyperspectral image lossy compression for sub-band groups

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23765336

Country of ref document: EP

Kind code of ref document: A1