CN113743518B - Approximate reversible image translation method based on joint inter-frame coding and embedding - Google Patents
Approximate reversible image translation method based on joint inter-frame coding and embedding Download PDFInfo
- Publication number
- CN113743518B CN113743518B CN202111055395.1A CN202111055395A CN113743518B CN 113743518 B CN113743518 B CN 113743518B CN 202111055395 A CN202111055395 A CN 202111055395A CN 113743518 B CN113743518 B CN 113743518B
- Authority
- CN
- China
- Prior art keywords
- image
- source image
- embedding
- translation
- coding
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000013519 translation Methods 0.000 title claims abstract description 58
- 238000000034 method Methods 0.000 title claims abstract description 33
- 230000002441 reversible effect Effects 0.000 title claims abstract description 24
- 230000001131 transforming effect Effects 0.000 claims description 6
- 238000011084 recovery Methods 0.000 claims description 3
- 230000007547 defect Effects 0.000 abstract description 3
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000013441 quality evaluation Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 241000283070 Equus zebra Species 0.000 description 1
- -1 carrier Substances 0.000 description 1
- 239000000306 component Substances 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000009472 formulation Methods 0.000 description 1
- 239000004615 ingredient Substances 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 239000007858 starting material Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000011426 transformation method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computational Linguistics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Evolutionary Biology (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
The invention discloses an approximate reversible image translation method based on joint inter-frame coding and embedding. The image translation method can ensure that the translation image is approximately and reversibly inversely transformed into the reconstructed source image while the source image is transformed into the translation image by carrying out inter-frame coding on the pseudo video consisting of the reconstructed source image and the source image based on the generated countermeasure network and embedding the generated coded bit stream into the translation image. The image translation method overcomes the defect that the reversibility cannot be ensured by the existing image translation method, constructs an approximately reversible image translation technical frame, and improves the image quality of the reconstructed source image in the image translation.
Description
Technical Field
The invention relates to the technical field of image coding, in particular to an approximate reversible image translation method based on joint inter-frame coding and embedding.
Background
Image translation based on a generative countermeasure network can transform a source image in a source domain into a translated image in a target domain. And, the translated image may be inversely transformed into a reconstructed source image. Image translation plays an important role in the fields of content transformation, image style conversion, image saliency detection and the like. However, the existing image translation method based on the generation type countermeasure network cannot realize reversible image translation, namely, unavoidable distortion exists between the reconstructed source image and the source image, so that the application of image translation is limited to a certain extent.
Disclosure of Invention
The invention aims to provide an approximate reversible image translation method based on joint inter-frame coding and embedding, which overcomes the defect that the reversibility cannot be ensured in the conventional image translation method, constructs an approximate reversible image translation technical frame and improves the image quality of a reconstructed source image in image translation.
The invention aims at realizing the following technical scheme:
an approximately reversible image translation method based on joint inter-frame coding and embedding, comprising:
transforming the source image into a translated image using a generative countermeasure network, and inversely transforming the translated image into a reconstructed source image based on the generative countermeasure network;
forming a pseudo video by the reconstructed source image based on the generated type countermeasure network and the source image, performing inter-frame coding on the source image by taking the reconstructed source image based on the generated type countermeasure network as a reference frame based on a joint inter-frame coding and embedding mode, and reversibly embedding a generated coding bit stream into the translation image to generate a secret-carrying translation image;
extracting the coding bit stream from the encrypted translation image to obtain a translation image;
and decoding the coded bit stream by taking the reconstructed source image based on the generated countermeasure network as a reference frame to obtain a final reconstructed source image.
According to the technical scheme provided by the invention, the generated coding bit stream is reversibly embedded into the translation image by inter-coding the pseudo video consisting of the reconstructed source image and the source image based on the generated countermeasure network, so that the translation image can be ensured to be approximately reversibly and inversely transformed into the reconstructed source image while the source image is transformed into the translation image.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of an approximately reversible image translation method based on joint inter-frame coding and embedding provided by an embodiment of the present invention;
FIG. 2 is a diagram of source images and reconstructed source images corresponding to four pairs of image transform domains and quality evaluation results provided by an embodiment of the present invention;
fig. 3 is a diagram of translated images corresponding to four pairs of image transformation domains, a secret translated image and a quality evaluation result provided in an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to fall within the scope of the invention.
The terms that may be used herein will first be described as follows:
the terms "comprises," "comprising," "includes," "including," "has," "having" or other similar referents are to be construed to cover a non-exclusive inclusion. For example: including a particular feature (e.g., a starting material, component, ingredient, carrier, formulation, material, dimension, part, means, mechanism, apparatus, step, procedure, method, reaction condition, processing condition, parameter, algorithm, signal, data, product or article of manufacture, etc.), should be construed as including not only a particular feature but also other features known in the art that are not explicitly recited.
The following describes a method for translating an approximately reversible image based on joint inter-frame coding and embedding. What is not described in detail in the embodiments of the present invention belongs to the prior art known to those skilled in the art. The specific conditions are not noted in the examples of the present invention and are carried out according to the conditions conventional in the art or suggested by the manufacturer.
As shown in fig. 1, an approximately reversible image translation method based on joint inter-frame coding and embedding includes the following steps:
and step 11, converting the source image into a translation image by using the generated countermeasure network, and inversely converting the translation image into a reconstructed source image based on the generated countermeasure network.
In the embodiment of the invention, the generated type countermeasure network can select a CycleGAN network. The CycleGAN network is capable of transforming a source image in a source domain into a translated image in a target domain and of inversely transforming the translated image into a reconstructed source image based on a generative countermeasure network.
And 12, forming a pseudo video by the reconstructed source image based on the generated type countermeasure network and the source image, performing inter-frame coding on the source image by taking the reconstructed source image based on the generated type countermeasure network as a reference frame based on a joint inter-frame coding and embedding mode, and reversibly embedding the generated coded bit stream into the translation image to generate a secret-carrying translation image.
The preferred embodiment of this step is as follows:
1) The reconstructed source image based on the generated type countermeasure network and the source image form a pseudo video, the source image is inter-coded by taking the reconstructed source image based on the generated type countermeasure network as a reference frame, and the generated coded bit stream can be expressed as:
minimize D S (R),subject to R≤R max
wherein D is S (R) is the source distortion of the encoded source image, R is the encoding bit rate (unit: kbps) of the source image, R max Is the upper bound of the coding bit rate. Source distortion D of encoded source image when lossless inter-coding of source image is performed S (R) is zero.
2) The encoded bitstream is reversibly embedded into the translation image to generate a dense translation image.
The reversible embedding of the coded bitstream into the translated image can be expressed as:
minimize D E (L),subject to L≤C max
wherein D is E (L) is embedding distortion caused by reversibly embedding the encoded bit stream in the translated image, L is the encoded bit stream length, C max For translating the embedded capacity of the image.
It should be noted that, in the embodiment of the present invention, the method of reversibly embedding the encoded bitstream into the translated image is not limited to a specific method of hiding reversible information of the image, and a user may select a suitable method from the existing schemes according to actual requirements.
In the embodiment of the present invention, the relation between the encoding bit rate R of the source image and the encoding bit stream length L involved in inter-frame encoding is expressed as:
wherein F is the frame rate set for video coding, N P Is the number of P frames in the inter-frame coding process, which refers to forward predictive coded image frames.
Based on the above scheme, this step of joint inter-coding and embedding is expressed as:
minimizeωD S (R)+(1-ω)D E (L),subject to L≤C max
wherein ω is the source distortion D for controlling the encoded source image S (R) weight and embedding distortion D E (L) parameters of weights.
Exemplary, N can be set P =1, ω=0.9, but specific values may be set by the user according to actual situations or experience by themselves, and the present invention is not limited to specific values.
And step 13, extracting the coded bit stream from the encrypted translation image to obtain the translation image.
Since the embedding process of the encoded bit stream is reversible in the aforementioned step 12 of the present invention, a lossless recovery translated image can be obtained when the encoded bit stream is extracted from the encrypted translated image.
And step 14, decoding the coded bit stream by taking the reconstructed source image based on the generated type countermeasure network as a reference frame to obtain a final reconstructed source image.
In this step, the reconstructed source image based on the generated type countermeasure network is used as a reference frame to decode the encoded bitstream, and when the translated image can accommodate the encoded bitstream generated by lossless inter-frame encoding, a reconstructed source image with lossless recovery can be obtained, and a specific execution flow can be implemented by a conventional technology, which is not described herein.
According to the scheme, the method for translating the approximate reversible image based on joint inter-frame coding and embedding is used for solving the reversibility problem in image translation, and the coded bit stream is reversibly embedded into the translated image by inter-frame coding of a pseudo video consisting of a reconstructed source image based on a generation type countermeasure network and a source image, so that the source image can be converted into the translated image, and meanwhile, the translated image can be ensured to be approximately and reversibly reversely converted into the reconstructed source image. The scheme overcomes the defect that the reversibility cannot be ensured by the existing image translation method, constructs an approximately reversible image translation technical frame, and improves the image quality of the reconstructed source image in the image translation.
In order to test the reconstructed source image quality (i.e., the final reconstructed source image of step 14) and the load translated image quality of the Proposed near-reversible image translation method based on joint inter-frame coding and embedding, the Proposed near-reversible image translation method is denoted as Proposed. And when the quality of the reconstructed source image is compared, selecting a generated type network opposing CycleGAN method as comparison. When comparing the quality of the encrypted translation image, two image transformation methods based on image block matching and image block residual reversible embedding are selected as comparison and respectively marked as A and B. Wherein B considers the inter-color image channel correlation, whereas a does not consider the inter-color image channel correlation.
Four pairs of image transform domains were considered in the comparison experiment, namely, the transform from Apple to Orange, the transform from Horse to Zebra, the transform from Monet to Photo, the transform from Summer to Winter, and their corresponding inverse transforms. And when the quality of the reconstructed source image is evaluated, the reference image is the source image. And when evaluating the quality of the secret translation image, the reference image is a translation image. The reconstructed source image quality and the dense translation image quality are measured using a peak signal-to-noise ratio, PSNR (unit: dB), and a structural similarity, SSIM.
Fig. 2 shows a source image and a reconstructed source image corresponding to four pairs of image transform domains. In the result shown in fig. 2, the PSNR values of the reconstructed source image of the present invention are all above 45dB, so that it can be seen that the reconstructed source image obtained by the method for translating an approximate reversible image provided by the present invention is approximately reversible.
Fig. 3 shows a translated image and a loaded translated image corresponding to four pairs of image transform domains. As can be seen from the structure shown in FIG. 3, the approximate reversible image translation method provided by the invention can obtain the best loaded translation image quality.
From the description of the above embodiments, it will be apparent to those skilled in the art that the above embodiments may be implemented in software, or may be implemented by means of software plus a necessary general hardware platform. With such understanding, the technical solutions of the foregoing embodiments may be embodied in a software product, where the software product may be stored in a nonvolatile storage medium (may be a CD-ROM, a U-disk, a mobile hard disk, etc.), and include several instructions for causing a computer device (may be a personal computer, a server, or a network device, etc.) to perform the methods of the embodiments of the present invention.
The foregoing is only a preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions easily contemplated by those skilled in the art within the scope of the present invention should be included in the scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.
Claims (3)
1. An approximately reversible image translation method based on joint inter-frame coding and embedding, comprising:
transforming the source image into a translated image using a generative countermeasure network, and inversely transforming the translated image into a reconstructed source image based on the generative countermeasure network;
forming a pseudo video by the reconstructed source image based on the generated type countermeasure network and the source image, performing inter-frame coding on the source image by taking the reconstructed source image based on the generated type countermeasure network as a reference frame based on a joint inter-frame coding and embedding mode, and reversibly embedding a generated coding bit stream into the translation image to generate a secret-carrying translation image;
extracting the coding bit stream from the encrypted translation image to obtain a translation image;
decoding the coded bit stream by taking the reconstructed source image based on the generated type countermeasure network as a reference frame to obtain a final reconstructed source image;
inter-coding the source image by taking the reconstructed source image based on the generated countermeasure network as a reference frame, wherein the generated coded bit stream is expressed as:
minimizeD S (R),subject to R≤R max
wherein D is S (R) is the source distortion of the encoded source image, R is the encoding bit rate of the source image, R max For the upper bound of the coding bit rate, the source distortion D of the encoded source image is the source distortion D when the source image is losslessly inter-coded S (R) is zero;
reversibly embedding the encoded bitstream into the translated image, expressed as:
minimizeD E (L),subject to L≤C max
wherein D is E (L) is embedding distortion caused by reversibly embedding the encoded bit stream in the translated image, L is the encoded bit stream length, C max Embedding capacity for translation image;
joint inter-coding and embedding is expressed as:
minimizeωD S (R)+(1-ω)D E (L),subject to L≤C max
wherein ω is the source distortion D for controlling the encoded source image S (R) weight and embedding distortion D E (L) parameters of weights.
2. The method for translating an approximately reversible image based on joint inter-frame coding and embedding according to claim 1, wherein the relation between the coding bit rate R of the source image involved in inter-frame coding and the coding bit stream length L is expressed as:
wherein F is the frame rate set for video coding, N P Is the number of P frames in the inter-frame coding process, which refers to forward predictive coded image frames.
3. The method for approximately reversible image translation based on joint inter-coding and embedding according to claim 1, wherein decoding the encoded bitstream with the reconstructed source image based on the generated type countermeasure network as a reference frame, to obtain a final reconstructed source image comprises:
and decoding the coded bit stream by taking the reconstructed source image based on the generated countermeasure network as a reference frame, and obtaining the reconstructed source image with lossless recovery when the translated image can accommodate the coded bit stream generated by lossless inter-frame coding.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111055395.1A CN113743518B (en) | 2021-09-09 | 2021-09-09 | Approximate reversible image translation method based on joint inter-frame coding and embedding |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111055395.1A CN113743518B (en) | 2021-09-09 | 2021-09-09 | Approximate reversible image translation method based on joint inter-frame coding and embedding |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113743518A CN113743518A (en) | 2021-12-03 |
| CN113743518B true CN113743518B (en) | 2024-04-02 |
Family
ID=78737736
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111055395.1A Active CN113743518B (en) | 2021-09-09 | 2021-09-09 | Approximate reversible image translation method based on joint inter-frame coding and embedding |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113743518B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2011064945A1 (en) * | 2009-11-30 | 2011-06-03 | 日本電気株式会社 | Video encoding device and video decoding device |
| CN102342127A (en) * | 2009-01-28 | 2012-02-01 | 诺基亚公司 | Method and apparatus for video coding and decoding |
| CN111050021A (en) * | 2019-12-17 | 2020-04-21 | 中国科学技术大学 | Image privacy protection method based on two-dimensional code and reversible visual watermark |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050008240A1 (en) * | 2003-05-02 | 2005-01-13 | Ashish Banerji | Stitching of video for continuous presence multipoint video conferencing |
-
2021
- 2021-09-09 CN CN202111055395.1A patent/CN113743518B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102342127A (en) * | 2009-01-28 | 2012-02-01 | 诺基亚公司 | Method and apparatus for video coding and decoding |
| WO2011064945A1 (en) * | 2009-11-30 | 2011-06-03 | 日本電気株式会社 | Video encoding device and video decoding device |
| CN111050021A (en) * | 2019-12-17 | 2020-04-21 | 中国科学技术大学 | Image privacy protection method based on two-dimensional code and reversible visual watermark |
Non-Patent Citations (2)
| Title |
|---|
| 基于双通道循环生成对抗网络的无人车夜视红外视频彩色化;李佳豪;孙韶媛;吴雪平;李大威;;激光与光电子学进展(09);全文 * |
| 视频图像DCT域压缩感知编码与重构;潘丽峰;;湖南第一师范学院学报(03);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113743518A (en) | 2021-12-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP3933691B2 (en) | Hybrid waveform and model-based encoding and decoding of image signals | |
| US8223837B2 (en) | Learning-based image compression | |
| Qiu et al. | Reversible data hiding in encrypted images using adaptive reversible integer transformation | |
| CN101203882B (en) | Method and apparatus for noise filtering in video coding | |
| Lu et al. | An improved lossless data hiding scheme based on image VQ-index residual value coding | |
| Su et al. | Detection of double-compression in MPEG-2 videos | |
| Hong et al. | Lossless steganography for AMBTC-compressed images | |
| Goel et al. | An optimized un-compressed video watermarking scheme based on SVD and DWT | |
| Ramkumar et al. | Uncompressed digital video watermarking using stationary wavelet transform | |
| Kumar et al. | An improved data-hiding approach using skin-tone detection for video steganography | |
| CN113743518B (en) | Approximate reversible image translation method based on joint inter-frame coding and embedding | |
| Kaul et al. | Audio in image steganography based on wavelet transform | |
| Li et al. | Bi-stretch reversible data hiding algorithm for absolute moment block truncation coding compressed images | |
| Mallikarjuna et al. | Image compression and reconstruction using discrete Rajan transform based spectral sparsing | |
| CN112714230A (en) | Robust video steganography method and device based on audio side channel | |
| Malik et al. | A high capacity data hiding scheme using modified AMBTC compression technique. | |
| JP4945541B2 (en) | Digital watermark embedding detection method using degraded host signal | |
| Kim et al. | End-to-end learnable multi-scale feature compression for vcm | |
| Peng et al. | Optimal audio watermarking scheme using genetic optimization | |
| CN1277412C (en) | Video frequency data copying control system and method | |
| CN114598887A (en) | Anti-recompression video watermarking method for controlling bit rate increase | |
| CN113079274A (en) | Encryption image reversible information hiding method of adaptive parameter binary tree mark | |
| JP5351094B2 (en) | Image coding method, image coding apparatus, and image coding program | |
| JP4685825B2 (en) | Moving picture decoding apparatus, method and program, and moving picture encoding apparatus, method and program | |
| Ma et al. | Large capacity and high quality reversible data hiding method based on enhanced side match vector quantization |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |