US9271006B2 - Coding and decoding method for images or videos - Google Patents

Coding and decoding method for images or videos Download PDF

Info

Publication number
US9271006B2
US9271006B2 US14/534,780 US201414534780A US9271006B2 US 9271006 B2 US9271006 B2 US 9271006B2 US 201414534780 A US201414534780 A US 201414534780A US 9271006 B2 US9271006 B2 US 9271006B2
Authority
US
United States
Prior art keywords
visual
specific object
image
visual word
word
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
US14/534,780
Other versions
US20150131921A1 (en
Inventor
Tiejun HUANG
Wen Gao
Siwei Ma
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Peking University
Original Assignee
Peking University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Peking University filed Critical Peking University
Assigned to PEKING UNIVERSITY reassignment PEKING UNIVERSITY ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: GAO, WEN, HUNAG, TIEJUN, MA, SIWEI
Publication of US20150131921A1 publication Critical patent/US20150131921A1/en
Application granted granted Critical
Publication of US9271006B2 publication Critical patent/US9271006B2/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/20—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • G06T9/008—Vector quantisation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/20—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding
    • H04N19/23—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding with coding of regions that are present throughout a whole video segment, e.g. sprites, background or mosaic
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/593—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial prediction techniques
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
    • H04N19/94—Vector quantisation

Definitions

  • the present invention is related to computer coding and decoding technology, especially related to a coding and decoding method for images or videos.
  • a new coding and decoding method for images or videos are provided by embodiments of the present invention to further improve coding or decoding efficiency.
  • a coding method for images or videos provided includes:
  • the visual dictionary includes one or more visual words
  • a decoding method for images or videos provided includes:
  • the decoding process must refer to the visual dictionary, in this case, even the code stream is captured, the code stream still cannot be decoded without the corresponding visual dictionary, thus the safety of the code stream is guaranteed.
  • FIG. 1 illustrates a flow chart of a coding method for images or videos.
  • FIG. 2 illustrates a flow chart of a feature matching method or videos.
  • FIG. 3 illustrates a framework of a coding method for images or videos.
  • FIG. 4 illustrates a flow chart of a decoding method for images or videos.
  • the term “or” is an inclusive “or” operator, and is equivalent to the term “and/or,” unless the context clearly dictates otherwise.
  • the term “based on” is not exclusive and allows for being based on additional factors not described, unless the context clearly dictates otherwise.
  • the meaning of “a,” “an,” and “the” include plural references.
  • the meaning of “in” includes “in” and “on”.
  • the term “coupled” implies that the elements may be directly connected together or may be coupled through one or more intervening elements. Further reference may be made to an embodiment where a component is implemented and multiple like or identical components are implemented.
  • the order of the steps in the present embodiment is exemplary and is not intended to be a limitation on the embodiments of the present invention. It is contemplated that the present invention includes the process being practiced in other orders and/or with intermediary steps and/or processes.
  • a visual dictionary is established to include those visual objects appearing with high frequency, and each visual object corresponds to a standard visual word in the visual dictionary.
  • the size of data content in a video stream is further reduced and coding efficiency is improved.
  • FIG. 1 illustrates a flow chart of a coding method for images or videos. As shown in FIG. 1 , the method includes following steps.
  • Step 100 a visual dictionary is established, wherein, the visual dictionary includes one or more visual words, and each visual word includes a visual object or a texture object, and corresponding features thereof
  • the visual object or texture object in the visual dictionary may be represented by an image.
  • the visual object is Tiananmen Square
  • an image of Tiananmen Square and corresponding features of the image are stored in the visual dictionary.
  • the corresponding features may include local features and/or global features.
  • the global features may describe color histograms, color matrixes or co-occurrence matrixes of gray level; or may be obtained by combining local features.
  • These global features only represent global information of the image, and cannot represent objects contained in the image.
  • the local features have sufficient description and distinction ability to describe image features.
  • the local features usually include one or more lower-layer expressions, which may be expressions describing one or more circular areas, and the local features cannot visually describe visual objects.
  • Step 101 features are extracted from a specific object in an image to be coded.
  • the image to be coded is different with the image of a visual object or texture object in the visual dictionary.
  • Step 102 a feature matching method is used to determine whether there is a visual word in the visual dictionary matching the specific object of the image to be coded.
  • Step 103 the index of the visual word matched and a geometric relationship between the specific object and the visual word matched are obtained, and the geometric relationship is represented by a project parameter; the project parameter may include magnification, deflation, rotation, affine, relative position and so on.
  • Step 104 differences between the image and all of visual words matched are calculated.
  • each visual object or textual object of a visual word is projected to a corresponding position of a blank image which has the same size with the image to be coded; and then the projected image is subtracted from the image to be coded to obtain the differences.
  • Step 105 the differences are coded by using a sparse coding method or a traditional coding method to obtain residuals.
  • Step 106 the project parameter and the index of the visual word matched, both of which are obtained in Step 103 , and the residuals obtained in Step 105 are entropy coded.
  • the entropy coding method may be based on a prior coding standard, which includes fixed length coding, variable length coding or arithmetic coding, etc.
  • a feature matching method may be used to determine whether there is a visual word in a visual dictionary matching a specific object of the image to be coded.
  • the method includes following steps.
  • Step 201 local features are extracted from the specific object in the image.
  • SIFT algorithm may be used to extract the local features of the specific object.
  • Step 202 the extracted local features of the specific object are compared with local features of a visual word in the visual dictionary to obtain a local feature pair.
  • the local feature pair includes two identical or similar local features respectively extracted from the specific object and obtained from the visual word. The two local features which similarity degree is within a threshold range would be considered as similar.
  • Step 203 geometric distributions of the local features corresponding to the local feature pair are calculated respectively in the specific object and the visual word.
  • Step 204 it is determined whether the geometric distributions of the local features corresponding to the local feature pair, respectively in the specific object and the visual word, are consistent; if the two geometric distributions are consistent, the visual word is considered matching the specific object, and it is further considered that the image to be coded contains the visual object or the texture object corresponding to the visual word.
  • 1000 local features are extracted from a specific object and 800 local features are obtained from a visual word, and 200 local feature pairs are obtained through feature comparisons. Then geometric distributions of the local features corresponding to each of the 200 local feature pairs are calculated respectively in the specific object and the visual word. If the geometric distributions of the local features corresponding to each of the 200 local feature pairs, respectively in the specific object and in the visual word, are considered as consistent, it is considered that the specific object includes an object corresponding to the visual word.
  • the geometric distributions of the local features corresponding to the local feature pairs are considered as consistent.
  • local features of each specific object may be combined to obtain a global feature; in the same way, local features of each visual word may be combined to obtain a global feature too. Then the visual dictionary is searched for one or more candidate visual words with the most similar global feature with that of the specific object; then local features of the specific object are compared with that of the one or more candidate visual words respectively.
  • the feature matching efficiency can be further improved.
  • FIG. 3 illustrates a framework of a coding method for images. As shown in FIG. 3 , coding an image of “Beijing University Weiming Lake (used as “Lake” for simplicity)” is used as an example to illustrate the coding process provided by an embodiment of the present invention.
  • Visual words including visual objects such as the sky, Beijing University learned tower (a tower located by the side of the Lake, used as “tower” for simplicity), a Stele, and their corresponding local features are stored in a visual dictionary in advance.
  • Visual words including textual object such as trees, water, gravel road, and their corresponding local features are also stored in the visual dictionary.
  • the specific objects of the image are compared with the visual words in the visual dictionary one-by-one firstly, then visual words such as the sky, tower, Stele, trees, water and gravel road are found, and then indexes of the visual words matched and their corresponding project parameters are obtained.
  • the image of “Lake” is compared with the visual words matched to obtain differences; the differences are coded by using a sparse coding method or a traditional coding method to obtain residuals. Finally, the indexes of the visual words matched, the corresponding project parameters and the residuals are entropy coded instead.
  • FIG. 4 illustrates a flow chart of a decoding method for images. As shown in FIG. 4 , the method includes following steps.
  • Step 401 a code stream of an image is entropy decoded to obtain an index of a visual word, a project parameter and residuals.
  • the entropy decoding method corresponds to the entropy coding method illustrated in Step 106 .
  • Step 402 an image of a visual object is obtained from a visual dictionary according to the index of the visual word, and then the image of the visual object is adjusted with reference to the project parameter.
  • the image of the visual object obtained from the visual dictionary is adjusted by being projected to a corresponding position of a blank image, which has the same size with the image to be decoded.
  • the image of the visual object stored in the visual dictionary and used to represent the visual object is different from the limitation “image” referring to the image to be coded or decoded in the embodiments of the present invention.
  • Step 403 the residuals are reversely decoded to obtain differences between the image to be decoded and the visual word.
  • Step 404 the adjusted images of the visual objects and the differences are overlapped to obtain a decoded image.
  • Step 402 and Step 403 are exchangeable.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Image Analysis (AREA)

Abstract

A coding and decoding method for images or videos is provided by embodiments of the present invention to improve coding and decoding efficiency. The method includes: establishing a visual dictionary, wherein, the visual dictionary includes one or more visual words; extracting features from a specific object in an image; determining whether there is a visual word in the visual dictionary matching the specific object by using a feature matching method; obtaining the index of the visual word matched and a geometric relationship between the specific object and the visual word matched, wherein, the geometric relationship is represented by a project parameter; entropy coding the index of the visual word matched and the project parameter instead of entropy coding the specific object.

Description

CROSS-REFERENCE TO RELATED APPLICATION
This application claims priority from CN Patent Application Serial No.201310551681.6, filed on Nov. 7 2013, the entire contents of which are incorporated herein by reference for all purposes.
FIELD OF THE INVENTION
The present invention is related to computer coding and decoding technology, especially related to a coding and decoding method for images or videos.
BACKGROUND OF THE INVENTION
In the prior art, most of the coding or decoding method and coder or decoder thereof are based on analysis of the code of images and videos themselves, and further redundant image pixels are compressed to improve coding or decoding efficiency.
With the development of local feature technology of images and videos, another coding or decoding method appears in the prior art. Instead of compressing image pixels, image features are extracted and compressed; and at a decoding side, images are then reconstructed with reference to the image features and a large-scaled image feature database.
However, even image features are used to code or decode images, the size of data content is still very large.
SUMMARY OF THE INVENTION
A new coding and decoding method for images or videos are provided by embodiments of the present invention to further improve coding or decoding efficiency.
In an embodiment of the present invention, a coding method for images or videos provided includes:
establishing a visual dictionary, wherein, the visual dictionary includes one or more visual words;
extracting features from a specific object in an image;
determining whether there is a visual word in the visual dictionary matching the specific object, by using a feature matching method;
obtaining the index of the visual word matched and a geometric relationship between the specific object and the visual word matched; wherein, the geometric relationship is represented by a project parameter;
entropy coding the index of the visual word matched and the project parameter instead of entropy coding the specific object.
In an embodiment of the present invention, a decoding method for images or videos provided includes:
entropy decoding a code stream to obtain an index and a project parameter of a visual word;
obtaining an image of a visual object from a visual dictionary according to the index of the visual word;
adjusting the image of the visual object with reference to the project parameter;
overlapping all of the adjusted images of the visual objects to obtain a decoded image.
By using the technical scheme of the present invention, only the index of a specific object in a visual dictionary and corresponding geometric relationship information are included in a code stream of an image, so that the size of data content in the code stream is greatly reduced. Moreover, the decoding process must refer to the visual dictionary, in this case, even the code stream is captured, the code stream still cannot be decoded without the corresponding visual dictionary, thus the safety of the code stream is guaranteed.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 illustrates a flow chart of a coding method for images or videos.
FIG. 2 illustrates a flow chart of a feature matching method or videos.
FIG. 3 illustrates a framework of a coding method for images or videos.
FIG. 4 illustrates a flow chart of a decoding method for images or videos.
 DETAILED DESCRIPTION OF THE INVENTION
The embodiments of the present invention are described more fully hereinafter with reference to the accompanying drawings, which form a part hereof, and which show, by way of illustration, specific exemplary embodiments by which the invention may be practiced. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be through and complete, and will fully convey the scope of the invention to those skilled in the art. Among other things, the present invention may be embodied as systems, methods or devices. The following detailed description should not to be taken in a limiting sense.
Throughout the specification and claims, the following terms take the meanings explicitly associated herein, unless the context clearly dictates otherwise. The phrase “in one embodiment” as used herein does not necessarily refer to the same embodiment, though it may. Furthermore, the phrase “in another embodiment” as used herein does not necessarily refer to a different embodiment, although it may. Thus, as described below, various embodiments of the invention may be readily combined, without departing from the scope or spirit of the invention.
In addition, as used herein, the term “or” is an inclusive “or” operator, and is equivalent to the term “and/or,” unless the context clearly dictates otherwise. The term “based on” is not exclusive and allows for being based on additional factors not described, unless the context clearly dictates otherwise. In addition, throughout the specification, the meaning of “a,” “an,” and “the” include plural references. The meaning of “in” includes “in” and “on”. The term “coupled” implies that the elements may be directly connected together or may be coupled through one or more intervening elements. Further reference may be made to an embodiment where a component is implemented and multiple like or identical components are implemented.
While the embodiments make reference to certain events this is not intended to be a limitation of the embodiments of the present invention and such is equally applicable to any event where goods or services are offered to a consumer.
Further, the order of the steps in the present embodiment is exemplary and is not intended to be a limitation on the embodiments of the present invention. It is contemplated that the present invention includes the process being practiced in other orders and/or with intermediary steps and/or processes.
In a coding method for images provided by an embodiment of the present invention, a visual dictionary is established to include those visual objects appearing with high frequency, and each visual object corresponds to a standard visual word in the visual dictionary. When an image is to be coded, it is determined that whether the image includes a visual word; if the image includes a visual word, the image is coded with reference to the index of the visual word and the relationship between the visual word and the image.
By using a coding method for images or videos provided by an embodiment of the present invention, the size of data content in a video stream is further reduced and coding efficiency is improved.
A coding process for images or videos provided by an embodiment of the present invention is described in detail as follows. FIG. 1 illustrates a flow chart of a coding method for images or videos. As shown in FIG. 1, the method includes following steps.
Step 100: a visual dictionary is established, wherein, the visual dictionary includes one or more visual words, and each visual word includes a visual object or a texture object, and corresponding features thereof
In an embodiment of the present invention, the visual object or texture object in the visual dictionary may be represented by an image. For example, if the visual object is Tiananmen Square, then an image of Tiananmen Square and corresponding features of the image are stored in the visual dictionary.
The corresponding features may include local features and/or global features. Specifically, the global features may describe color histograms, color matrixes or co-occurrence matrixes of gray level; or may be obtained by combining local features. These global features only represent global information of the image, and cannot represent objects contained in the image. The local features have sufficient description and distinction ability to describe image features. The local features usually include one or more lower-layer expressions, which may be expressions describing one or more circular areas, and the local features cannot visually describe visual objects.
Step 101: features are extracted from a specific object in an image to be coded.
It should be noted that, the image to be coded is different with the image of a visual object or texture object in the visual dictionary.
Step 102: a feature matching method is used to determine whether there is a visual word in the visual dictionary matching the specific object of the image to be coded.
Step 103: the index of the visual word matched and a geometric relationship between the specific object and the visual word matched are obtained, and the geometric relationship is represented by a project parameter; the project parameter may include magnification, deflation, rotation, affine, relative position and so on.
Those skilled in the art can understand, there may not one or more visual words in the visual dictionary matching the specific object or specific objects of the image to be coded. The indexes of all of the visual words found and the geometric relationships between the specific object and each of its corresponding visual words are obtained.
Step 104: differences between the image and all of visual words matched are calculated.
Specifically, according to the project parameter obtained, in order to form a projected image, each visual object or textual object of a visual word is projected to a corresponding position of a blank image which has the same size with the image to be coded; and then the projected image is subtracted from the image to be coded to obtain the differences.
Step 105: the differences are coded by using a sparse coding method or a traditional coding method to obtain residuals.
Step 106: the project parameter and the index of the visual word matched, both of which are obtained in Step 103, and the residuals obtained in Step 105 are entropy coded.
The entropy coding method may be based on a prior coding standard, which includes fixed length coding, variable length coding or arithmetic coding, etc.
Those skilled in the art can understand that, in the coding method described above, orders of some steps are changeable, and the changes of the orders will not affect effect of the present invention.
In an embodiment of the present invention, a feature matching method, as shown in FIG. 2, may be used to determine whether there is a visual word in a visual dictionary matching a specific object of the image to be coded. The method includes following steps.
Step 201: local features are extracted from the specific object in the image. Herein, SIFT algorithm may be used to extract the local features of the specific object.
Step 202: the extracted local features of the specific object are compared with local features of a visual word in the visual dictionary to obtain a local feature pair. The local feature pair includes two identical or similar local features respectively extracted from the specific object and obtained from the visual word. The two local features which similarity degree is within a threshold range would be considered as similar.
Step 203: geometric distributions of the local features corresponding to the local feature pair are calculated respectively in the specific object and the visual word.
Step 204: it is determined whether the geometric distributions of the local features corresponding to the local feature pair, respectively in the specific object and the visual word, are consistent; if the two geometric distributions are consistent, the visual word is considered matching the specific object, and it is further considered that the image to be coded contains the visual object or the texture object corresponding to the visual word.
For example, 1000 local features are extracted from a specific object and 800 local features are obtained from a visual word, and 200 local feature pairs are obtained through feature comparisons. Then geometric distributions of the local features corresponding to each of the 200 local feature pairs are calculated respectively in the specific object and the visual word. If the geometric distributions of the local features corresponding to each of the 200 local feature pairs, respectively in the specific object and in the visual word, are considered as consistent, it is considered that the specific object includes an object corresponding to the visual word. In an embodiment of the present invention, only when the number of the local feature pairs, which have a consistent relationship of projective transformation (such as magnification, deflation, rotation, affine, etc.) in the visual word or the specific object, reaches a certain threshold, the geometric distributions of the local features corresponding to the local feature pairs are considered as consistent.
In an embodiment of the present invention, in order to improve feature matching efficiency, local features of each specific object may be combined to obtain a global feature; in the same way, local features of each visual word may be combined to obtain a global feature too. Then the visual dictionary is searched for one or more candidate visual words with the most similar global feature with that of the specific object; then local features of the specific object are compared with that of the one or more candidate visual words respectively. By using this method, the feature matching efficiency can be further improved.
FIG. 3 illustrates a framework of a coding method for images. As shown in FIG. 3, coding an image of “Beijing University Weiming Lake (used as “Lake” for simplicity)” is used as an example to illustrate the coding process provided by an embodiment of the present invention.
Following visual words including visual objects such as the sky, Beijing University learned tower (a tower located by the side of the Lake, used as “tower” for simplicity), a Stele, and their corresponding local features, are stored in a visual dictionary in advance. Visual words including textual object such as trees, water, gravel road, and their corresponding local features are also stored in the visual dictionary. When the image of “Lake” is to be coded, the specific objects of the image are compared with the visual words in the visual dictionary one-by-one firstly, then visual words such as the sky, tower, Stele, trees, water and gravel road are found, and then indexes of the visual words matched and their corresponding project parameters are obtained. Then the image of “Lake” is compared with the visual words matched to obtain differences; the differences are coded by using a sparse coding method or a traditional coding method to obtain residuals. Finally, the indexes of the visual words matched, the corresponding project parameters and the residuals are entropy coded instead.
FIG. 4 illustrates a flow chart of a decoding method for images. As shown in FIG. 4, the method includes following steps.
Step 401: a code stream of an image is entropy decoded to obtain an index of a visual word, a project parameter and residuals.
The entropy decoding method corresponds to the entropy coding method illustrated in Step 106.
Step 402: an image of a visual object is obtained from a visual dictionary according to the index of the visual word, and then the image of the visual object is adjusted with reference to the project parameter.
Specifically, according to the project parameter obtained, the image of the visual object obtained from the visual dictionary is adjusted by being projected to a corresponding position of a blank image, which has the same size with the image to be decoded.
It should be noticed that, the image of the visual object stored in the visual dictionary and used to represent the visual object, is different from the limitation “image” referring to the image to be coded or decoded in the embodiments of the present invention.
Step 403: the residuals are reversely decoded to obtain differences between the image to be decoded and the visual word.
Step 404: the adjusted images of the visual objects and the differences are overlapped to obtain a decoded image.
Those skilled in the art can understand that, the orders of Step 402 and Step 403 are exchangeable.
The above embodiments are only preferred embodiments of the present invention and cannot be used to limit the protection scope of the present invention. Those skilled in the art can understand that, the technical scheme of the embodiment may still be modified or partly equivalently substituted; and the modification or substitution should be considered within the spirit and protection scope of the present invention.

Claims (9)

The invention claimed is:
1. A coding method for images or videos, comprising:
establishing a visual dictionary, wherein, the visual dictionary comprises one or more visual words;
extracting features from a specific object in an image;
determining whether there is a visual word in the visual dictionary matching the specific object by using a feature matching method;
obtaining the index of the visual word matched and a geometric relationship between the specific object and the visual word matched;
wherein, the geometric relationship is represented by a project parameter;
entropy coding the index of the visual word matched and the project parameter instead of entropy coding the specific object.
2. The method of claim 1, further comprising:
calculating differences between the image and the visual word matched;
coding the differences by using a sparse coding method or a traditional coding method to obtain residuals;
entropy coding the residuals with the index of the visual word matched and the project parameter.
3. The method of claim 1, wherein, each visual word comprises a visual object or a texture object, and corresponding features thereof.
4. The method of claim 1, wherein, the project parameter comprises magnification, deflation, rotation, affine, relative position.
5. The method of claim 1, wherein, determining whether there is a visual word in the visual dictionary matching the specific object comprises:
comparing extracted local features of the specific object with local features of a visual word in the visual dictionary to obtain a local feature pair which comprises two identical or similar local features respectively extracted from the specific object and obtained from the visual word;
calculating geometric distributions of the local features corresponding to the local feature pair, respectively in the specific object and in the visual word;
determining whether the geometric distributions of the local features corresponding to the local feature pair, respectively in the specific object and the visual word, are consistent; considering the visual word as matching the specific object if the two geometric distributions are consistent.
6. The method of claim 5, wherein, before comparing extracted local features of the specific object with local features of a visual word in a visual dictionary, the method further comprises:
combining the local features of each specific object to obtain a global feature;
searching the visual dictionary for a candidate visual word with the most similar global feature with that of the specific object.
7. The method of claim 6, wherein, SIFT algorithm is used to extract the local features of the specific object.
8. A decoding method for images or videos, comprising:
entropy decoding a code stream of an image to obtain an index and a project parameter of a visual word;
obtaining an image of a visual object from a visual dictionary according to the index of the visual word;
adjusting the image of the visual object with reference to the project parameter;
overlapping adjusted images of all of visual objects to obtain a decoded image.
9. The method of claim 8, further comprising:
entropy decoding the code stream to obtain residuals;
reversely decoding the residuals to obtain differences between the image to be decoded and the visual word;
overlapping the adjusted image of all of the visual objects and the differences to obtain a decoded image.
US14/534,780 2013-11-07 2014-11-06 Coding and decoding method for images or videos Active US9271006B2 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
CN201310551681 2013-11-07
CN201310551681.6A CN103561276B (en) 2013-11-07 2013-11-07 A kind of image/video decoding method
CN201310551681/6 2013-11-07

Publications (2)

Publication Number Publication Date
US20150131921A1 US20150131921A1 (en) 2015-05-14
US9271006B2 true US9271006B2 (en) 2016-02-23

Family

ID=50015411

Family Applications (1)

Application Number Title Priority Date Filing Date
US14/534,780 Active US9271006B2 (en) 2013-11-07 2014-11-06 Coding and decoding method for images or videos

Country Status (2)

Country Link
US (1) US9271006B2 (en)
CN (1) CN103561276B (en)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10930019B2 (en) 2019-01-15 2021-02-23 More Than Halfway, L.L.C. Encoding and decoding visual information
US11159808B2 (en) 2018-11-06 2021-10-26 Beijing Bytedance Network Technology Co., Ltd. Using inter prediction with geometric partitioning for video processing
US11265541B2 (en) 2018-11-06 2022-03-01 Beijing Bytedance Network Technology Co., Ltd. Position dependent storage of motion information
US11659180B2 (en) 2019-11-30 2023-05-23 Beijing Bytedance Network Technology Co., Ltd Simplified inter prediction with geometric partitioning
US11671586B2 (en) 2018-12-28 2023-06-06 Beijing Bytedance Network Technology Co., Ltd. Modified history based motion prediction
US11677941B2 (en) 2018-11-22 2023-06-13 Beijing Bytedance Network Technology Co., Ltd Construction method for inter prediction with geometry partition
US11700378B2 (en) 2019-12-24 2023-07-11 Beijing Bytedance Network Technology Co., Ltd High level syntax for inter prediction with geometric partitioning
US11758143B2 (en) 2019-10-10 2023-09-12 Beijing Bytedance Network Technology Co., Ltd Motion vector handling in geometry partition mode
US11956431B2 (en) 2018-12-30 2024-04-09 Beijing Bytedance Network Technology Co., Ltd Conditional application of inter prediction with geometric partitioning in video processing

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104918046B (en) * 2014-03-13 2019-11-05 中兴通讯股份有限公司 A kind of local description compression method and device
US9215468B1 (en) * 2014-08-07 2015-12-15 Faroudja Enterprises, Inc. Video bit-rate reduction system and method utilizing a reference images matrix
US10074161B2 (en) * 2016-04-08 2018-09-11 Adobe Systems Incorporated Sky editing based on image composition
CN108184113B (en) * 2017-12-05 2021-12-03 上海大学 Image compression coding method and system based on inter-image reference
US11895308B2 (en) * 2020-06-02 2024-02-06 Portly, Inc. Video encoding and decoding system using contextual video learning

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5668897A (en) * 1994-03-15 1997-09-16 Stolfo; Salvatore J. Method and apparatus for imaging, image processing and data compression merge/purge techniques for document image databases
US20030115219A1 (en) * 2001-12-19 2003-06-19 International Business Machines Corporation Method, system, and program for storing data in a data store
US6683993B1 (en) * 1996-11-08 2004-01-27 Hughes Electronics Corporation Encoding and decoding with super compression a via a priori generic objects
US7043094B2 (en) * 2001-06-07 2006-05-09 Commissariat A L'energie Atomique Process for the automatic creation of a database of images accessible by semantic features
US7643033B2 (en) * 2004-07-20 2010-01-05 Kabushiki Kaisha Toshiba Multi-dimensional texture mapping apparatus, method and program
US7889926B2 (en) * 2004-04-12 2011-02-15 Fuji Xerox Co., Ltd. Image dictionary creating apparatus, coding apparatus, image dictionary creating method
US8165215B2 (en) * 2005-04-04 2012-04-24 Technion Research And Development Foundation Ltd. System and method for designing of dictionaries for sparse representation
US20150294194A1 (en) * 2012-10-12 2015-10-15 Commissariat A L'energie Atomique Et Aux Energies Alternatives Method of classifying a multimodal object

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5826747B2 (en) * 2009-06-26 2015-12-02 トムソン ライセンシングＴｈｏｍｓｏｎ Ｌｉｃｅｎｓｉｎｇ Video encoding and decoding method and apparatus using adaptive geometric partitioning
KR101873609B1 (en) * 2010-10-14 2018-08-02 톰슨 라이센싱 Methods and apparatus for video encoding and decoding using motion matrix
CN102368237B (en) * 2010-10-18 2013-03-27 中国科学技术大学 Image retrieval method, device and system
US8767835B2 (en) * 2010-12-28 2014-07-01 Mitsubishi Electric Research Laboratories, Inc. Method for coding videos using dictionaries

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5668897A (en) * 1994-03-15 1997-09-16 Stolfo; Salvatore J. Method and apparatus for imaging, image processing and data compression merge/purge techniques for document image databases
US6683993B1 (en) * 1996-11-08 2004-01-27 Hughes Electronics Corporation Encoding and decoding with super compression a via a priori generic objects
US7043094B2 (en) * 2001-06-07 2006-05-09 Commissariat A L'energie Atomique Process for the automatic creation of a database of images accessible by semantic features
US20030115219A1 (en) * 2001-12-19 2003-06-19 International Business Machines Corporation Method, system, and program for storing data in a data store
US7889926B2 (en) * 2004-04-12 2011-02-15 Fuji Xerox Co., Ltd. Image dictionary creating apparatus, coding apparatus, image dictionary creating method
US7643033B2 (en) * 2004-07-20 2010-01-05 Kabushiki Kaisha Toshiba Multi-dimensional texture mapping apparatus, method and program
US8165215B2 (en) * 2005-04-04 2012-04-24 Technion Research And Development Foundation Ltd. System and method for designing of dictionaries for sparse representation
US20150294194A1 (en) * 2012-10-12 2015-10-15 Commissariat A L'energie Atomique Et Aux Energies Alternatives Method of classifying a multimodal object

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11665344B2 (en) 2018-11-06 2023-05-30 Beijing Bytedance Network Technology Co., Ltd. Multiple merge lists and orders for inter prediction with geometric partitioning
US11611763B2 (en) 2018-11-06 2023-03-21 Beijing Bytedance Network Technology Co., Ltd. Extensions of inter prediction with geometric partitioning
US11265541B2 (en) 2018-11-06 2022-03-01 Beijing Bytedance Network Technology Co., Ltd. Position dependent storage of motion information
US11431973B2 (en) 2018-11-06 2022-08-30 Beijing Bytedance Network Technology Co., Ltd. Motion candidates for inter prediction
US11457226B2 (en) 2018-11-06 2022-09-27 Beijing Bytedance Network Technology Co., Ltd. Side information signaling for inter prediction with geometric partitioning
US11570450B2 (en) 2018-11-06 2023-01-31 Beijing Bytedance Network Technology Co., Ltd. Using inter prediction with geometric partitioning for video processing
US11159808B2 (en) 2018-11-06 2021-10-26 Beijing Bytedance Network Technology Co., Ltd. Using inter prediction with geometric partitioning for video processing
US11677941B2 (en) 2018-11-22 2023-06-13 Beijing Bytedance Network Technology Co., Ltd Construction method for inter prediction with geometry partition
US11924421B2 (en) 2018-11-22 2024-03-05 Beijing Bytedance Network Technology Co., Ltd Blending method for inter prediction with geometry partition
US11671586B2 (en) 2018-12-28 2023-06-06 Beijing Bytedance Network Technology Co., Ltd. Modified history based motion prediction
US11956431B2 (en) 2018-12-30 2024-04-09 Beijing Bytedance Network Technology Co., Ltd Conditional application of inter prediction with geometric partitioning in video processing
US10930019B2 (en) 2019-01-15 2021-02-23 More Than Halfway, L.L.C. Encoding and decoding visual information
US11704836B2 (en) 2019-01-15 2023-07-18 More Than Halfway, L.L.C. Encoding and decoding visual information
US11758143B2 (en) 2019-10-10 2023-09-12 Beijing Bytedance Network Technology Co., Ltd Motion vector handling in geometry partition mode
US11659180B2 (en) 2019-11-30 2023-05-23 Beijing Bytedance Network Technology Co., Ltd Simplified inter prediction with geometric partitioning
US11700378B2 (en) 2019-12-24 2023-07-11 Beijing Bytedance Network Technology Co., Ltd High level syntax for inter prediction with geometric partitioning

Also Published As

Publication number Publication date
CN103561276B (en) 2017-01-04
US20150131921A1 (en) 2015-05-14
CN103561276A (en) 2014-02-05

Similar Documents

Publication Publication Date Title
US9271006B2 (en) Coding and decoding method for images or videos
Chen et al. Automatic detection of object-based forgery in advanced video
Redondi et al. Compress-then-analyze vs. analyze-then-compress: Two paradigms for image analysis in visual sensor networks
US8787692B1 (en) Image compression using exemplar dictionary based on hierarchical clustering
US20140254936A1 (en) Local feature based image compression
CN102750339B (en) Positioning method of repeated fragments based on video reconstruction
Zhong et al. Dense moment feature index and best match algorithms for video copy-move forgery detection
CN113744153B (en) Double-branch image restoration forgery detection method, system, equipment and storage medium
Kumar et al. Near lossless image compression using parallel fractal texture identification
Wang et al. Scalable facial image compression with deep feature reconstruction
CN102663398A (en) Color image color feature extraction method and device thereof
Vázquez et al. Using normalized compression distance for image similarity measurement: an experimental study
US9549206B2 (en) Media decoding method based on cloud computing and decoder thereof
CN107203763B (en) Character recognition method and device
CN103020138A (en) Method and device for video retrieval
Zhao et al. Detecting deepfake video by learning two-level features with two-stream convolutional neural network
US20140133550A1 (en) Method of encoding and decoding flows of digital video frames, related systems and computer program products
Roka et al. Deep stacked denoising autoencoder for unsupervised anomaly detection in video surveillance
Ma et al. MSFNET: multi-stage fusion network for semantic segmentation of fine-resolution remote sensing data
CN105989063A (en) Video retrieval method and device
CN105224619B (en) A kind of spatial relationship matching process and system suitable for video/image local feature
KR20240006667A (en) Point cloud attribute information encoding method, decoding method, device and related devices
Rad et al. Digital image forgery detection by edge analysis
Wang et al. Content-based image retrieval using H. 264 intra coding features
CN114298992A (en) Video frame duplication removing method and device, electronic equipment and storage medium

Legal Events

Date Code Title Description
AS Assignment

Owner name: PEKING UNIVERSITY, CHINA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:HUNAG, TIEJUN;GAO, WEN;MA, SIWEI;REEL/FRAME:034122/0305

Effective date: 20141104

STCF Information on status: patent grant

Free format text: PATENTED CASE

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YR, SMALL ENTITY (ORIGINAL EVENT CODE: M2551); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY

Year of fee payment: 4

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YR, SMALL ENTITY (ORIGINAL EVENT CODE: M2552); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY

Year of fee payment: 8