CN109493304B - Image data processing method and device - Google Patents

Image data processing method and device Download PDF

Info

Publication number
CN109493304B
CN109493304B CN201710824066.6A CN201710824066A CN109493304B CN 109493304 B CN109493304 B CN 109493304B CN 201710824066 A CN201710824066 A CN 201710824066A CN 109493304 B CN109493304 B CN 109493304B
Authority
CN
China
Prior art keywords
picture
layered
user terminal
picture data
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201710824066.6A
Other languages
Chinese (zh)
Other versions
CN109493304A (en
Inventor
应玉龙
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Priority to CN201710824066.6A priority Critical patent/CN109493304B/en
Priority to PCT/CN2018/105131 priority patent/WO2019052452A1/en
Publication of CN109493304A publication Critical patent/CN109493304A/en
Application granted granted Critical
Publication of CN109493304B publication Critical patent/CN109493304B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/40—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H04N19/423—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation characterised by memory arrangements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/24—Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—3D [Three Dimensional] image rendering

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

The embodiment of the invention discloses an image data processing method and a device, wherein the method comprises the following steps: sending a first layered picture acquisition request carrying the size information of the display area to a server so that the server selects a target layered picture matched with the size information according to the first layered picture acquisition request; receiving base layer picture data in a target layered picture sent by a server, decoding the base layer picture data through a layered picture decoder, and displaying the decoded base layer picture data; and receiving enhancement layer picture data in the target layered picture sent by the server, decoding the enhancement layer picture data through a layered picture decoder, synthesizing the decoded enhancement layer picture data and the decoded base layer picture data, and displaying the synthesized picture data. By adopting the invention, the picture display area can be prevented from being in a blank state for a long time, and the picture display effect can also be improved.

Description

Image data processing method and device
Technical Field
The present invention relates to the field of internet technologies, and in particular, to an image data processing method and apparatus.
Background
With the development of the mobile internet, the download traffic of the terminal device is greatly increased, and the picture traffic occupies a large proportion in the download traffic of the user. The large number of pictures also puts a great strain on the network transmission bandwidth load. Therefore, if the user downloads a picture with high quality when the network state is not very stable, the picture data transmission time is too long, and the picture display area of the user terminal is in a blank state for a long time. Although the problem of too long picture data transmission time can be solved by downloading the pictures with low quality, the pictures with low quality may cause the picture contents to be too fuzzy, and the display effect is seriously influenced.
Disclosure of Invention
The embodiment of the invention provides an image data processing method and device, which can prevent a picture display area from being in a blank state for a long time and can improve the picture display effect.
A first aspect of the present invention provides an image data processing method, including:
sending a first hierarchical picture acquisition request carrying size information of a display area to a server, so that the server selects a target hierarchical picture matched with the size information according to the first hierarchical picture acquisition request;
receiving base layer picture data in the target layered picture sent by the server, decoding the base layer picture data through a layered picture decoder, and displaying the decoded base layer picture data;
and receiving enhancement layer picture data in the target layered picture sent by the server, decoding the enhancement layer picture data through the layered picture decoder, synthesizing the decoded enhancement layer picture data and the decoded base layer picture data, and displaying the synthesized picture data.
Wherein, still include:
acquiring operating system information of a user terminal, and sending a decoder acquisition request carrying the operating system information of the user terminal to the server, so that the server searches a layered picture decoder matched with the operating system information of the user terminal according to the decoder acquisition request;
and receiving a layered picture decoder which is sent by the server and matched with the operating system information of the user terminal.
Before the sending the first hierarchical picture acquiring request carrying the size information of the display area to the server, the method further comprises:
acquiring size information of a display area, and judging whether the size information meets the self-adaptive condition of the layered picture;
if the judgment result is that the hierarchical picture data is not satisfied, sending a second hierarchical picture acquisition request to the server so that the server sequentially sends the base layer picture data and the enhancement layer picture data in the hierarchical picture with the maximum resolution according to the second hierarchical picture acquisition request;
and if the judgment result is yes, executing the step of sending the first hierarchical picture acquisition request carrying the size information of the display area to the server.
Before the obtaining of the size information of the display area, the method further includes:
judging whether a layered picture format is supported or not;
if the picture is judged to be unsupported, sending an original picture acquisition request to the server so as to enable the server to return to the original picture according to the original picture acquisition request;
decoding the received original picture through an original decoder, and displaying the decoded original picture;
and if the judgment result is that the display area is supported, the step of acquiring the size information of the display area is executed.
A second aspect of the present invention provides an image data processing method, including:
if a first layered picture acquisition request sent by a user terminal is received, selecting a target layered picture matched with size information according to the size information of a display area of the user terminal carried by the first layered picture acquisition request;
sending the base layer picture data in the target layered picture to the user terminal so that the user terminal decodes the base layer picture data through a layered picture decoder and displays the decoded base layer picture data;
and sending the enhancement layer picture data in the target layered picture to the user terminal so that the user terminal decodes the enhancement layer picture data through the layered picture decoder, synthesizes the decoded enhancement layer picture data with the decoded base layer picture data, and displays the synthesized picture data.
If a first layered picture acquisition request sent by a user terminal is received, selecting a target layered picture matched with size information according to the size information of a display area of the user terminal carried by the first layered picture acquisition request, including:
if a first layered picture acquisition request sent by a user terminal is received, extracting size information of a display area of the user terminal carried by the first layered picture acquisition request;
searching at least one layered picture with the pixel width larger than the picture width in the size information from a plurality of layered pictures with different resolutions as at least one layered picture to be selected;
and determining the layered picture with the lowest resolution in the at least one layered picture to be selected as the target layered picture matched with the size information.
Wherein, still include:
if a second layered picture acquisition request sent by the user is received, selecting a layered picture with the maximum resolution from a plurality of layered pictures with different resolutions according to the second layered picture acquisition request, and sequentially sending the base layer picture data and the enhancement layer picture data in the layered picture with the maximum resolution to the user terminal;
if an original picture acquiring request sent by the user is received, sending an original picture to the user terminal according to the original picture acquiring request;
wherein the second hierarchical picture acquisition request is sent by the user terminal upon detecting that the size information does not satisfy a hierarchical picture adaptation condition; the original picture acquisition request is sent by the user terminal when detecting that the layered picture format is not supported; the first hierarchical picture acquisition request is sent by the user terminal upon detecting that the size information satisfies a hierarchical picture adaptation condition.
Wherein, still include:
receiving a decoder acquisition request sent by the user terminal; the decoder acquires the operating system information of the user terminal carried by the request;
and searching a layered picture decoder matched with the operating system information of the user terminal according to the decoder acquisition request, and sending the searched layered picture decoder matched with the operating system information of the user terminal to the user terminal.
Wherein, still include:
acquiring an original picture, and decoding the original picture through a layered picture decoder matched with operating system information of a server to obtain decoded original picture data;
encoding the decoded original picture data into a layered picture of at least one resolution by a layered picture encoder; each layered picture includes base layer picture data and at least one enhancement layer picture data.
Wherein, still include:
setting a hierarchical picture decoder corresponding to at least one type of operating system information respectively;
optimizing prediction modes in the layered picture decoder and the layered picture encoder; the prediction mode comprises a coding unit division mode, a prediction unit division mode, a transformation unit division mode and an intra-frame prediction direction selection mode;
and performing assembly optimization and code structure optimization on the hierarchical picture decoder according to the corresponding operating system information.
Wherein the optimizing a prediction algorithm in the hierarchical picture decoder according to the corresponding operating system information comprises:
setting a storage space selection optimization rule in the hierarchical picture decoder, wherein the storage space selection optimization rule comprises a rule of setting a register as a temporary storage space with the highest priority, and the register is associated with operating system information corresponding to the hierarchical picture decoder;
setting a computational optimization rule in the hierarchical picture decoder, the computational optimization rule comprising a rule that converts a looping logic of a matrix operation into an operation based on a vector instruction set, the vector instruction set being associated with operating system information corresponding to the hierarchical picture decoder.
A third aspect of the present invention provides an image data processing method, including:
a user terminal sends a first hierarchical picture acquisition request carrying size information of a display area to a server;
the server selects a target layered picture matched with the size information according to the first layered picture acquisition request, and sends basic layer picture data in the target layered picture to the user terminal;
the user terminal decodes the base layer picture data through a layered picture decoder and displays the decoded base layer picture data;
the server sends enhancement layer picture data in the target layered picture to the user terminal;
and the user terminal decodes the enhancement layer picture data through the layered picture decoder, synthesizes the decoded enhancement layer picture data with the decoded base layer picture data, and displays the synthesized picture data.
Wherein, still include:
the user terminal acquires operating system information of the user terminal and sends a decoder acquisition request carrying the operating system information of the user terminal to the server;
and the server searches a layered picture decoder matched with the operating system information of the user terminal according to the decoder acquisition request and sends the layered picture decoder matched with the operating system information of the user terminal to the user terminal.
Before the user terminal sends a first hierarchical picture acquisition request carrying size information of a display area to a server, the method further comprises:
the user terminal acquires size information of a display area and judges whether the size information meets a layered picture self-adaptive condition or not;
if the judgment result is not satisfied, the user terminal sends a second hierarchical picture acquisition request to the server;
the server sequentially sends the base layer picture data and the enhancement layer picture data in the layered picture with the maximum resolution according to the second layered picture acquisition request;
and if the judgment result is yes, executing a step that the user terminal sends a first hierarchical picture acquisition request carrying the size information of the display area to a server.
Before the user terminal acquires the size information of the display area, the method further comprises the following steps:
the user terminal judges whether a layered picture format is supported or not;
if the user terminal does not support the image, the user terminal sends an original image acquisition request to the server;
the server returns the original picture according to the original picture acquisition request;
the user terminal decodes the received original picture through an original decoder and displays the decoded original picture;
and if the judgment is that the display area is supported, executing the step that the user terminal obtains the size information of the display area.
Wherein, the server selects a target layered picture matched with the size information according to the first layered picture acquisition request, and the method comprises the following steps:
the server extracts the size information of the display area of the user terminal carried by the first hierarchical picture acquisition request;
searching at least one layered picture with the pixel width larger than the picture width in the size information from a plurality of layered pictures with different resolutions as at least one layered picture to be selected;
and determining the layered picture with the lowest resolution in the at least one layered picture to be selected as the target layered picture matched with the size information.
Wherein, still include:
the server acquires an original picture, and the original picture is decoded by a layered picture decoder matched with the operating system information of the server to obtain decoded original picture data;
the server encodes the decoded original picture data into a layered picture with at least one resolution through a layered picture encoder; each layered picture includes base layer picture data and at least one enhancement layer picture data.
Wherein, still include:
the server sets a layered picture decoder corresponding to at least one type of operating system information respectively;
the server optimizes the prediction modes in the layered picture decoder and the layered picture encoder; the prediction mode comprises a coding unit division mode, a prediction unit division mode, a transformation unit division mode and an intra-frame prediction direction selection mode;
and the server performs assembly optimization and code structure optimization on the hierarchical picture decoder according to the corresponding operating system information.
Wherein, the server optimizes the prediction algorithm in the layered picture decoder according to the corresponding operating system information, including:
setting a storage space selection optimization rule in the hierarchical picture decoder, the storage space selection optimization rule comprising a rule of setting a register to a temporary storage space of a highest priority, the register being associated with operating system information corresponding to the hierarchical picture decoder;
setting a computational optimization rule in the hierarchical picture decoder, the computational optimization rule comprising a rule that converts a looping logic of a matrix operation into an operation based on a vector instruction set, the vector instruction set being associated with operating system information corresponding to the hierarchical picture decoder.
A fourth aspect of the present invention provides an image data processing apparatus comprising:
the device comprises a sending module, a receiving module and a display module, wherein the sending module is used for sending a first hierarchical picture acquisition request carrying size information of a display area to a server so that the server selects a target hierarchical picture matched with the size information according to the first hierarchical picture acquisition request;
the receiving module is used for receiving the basic layer picture data in the target layered picture sent by the server;
the decoding display module is used for decoding the base layer picture data through a layered picture decoder and displaying the decoded base layer picture data;
the receiving module is further configured to receive enhancement layer picture data in the target layered picture sent by the server;
the decoding and displaying module is further configured to decode the enhancement layer picture data through the layered picture decoder, synthesize the decoded enhancement layer picture data and the decoded base layer picture data, and display the synthesized picture data.
The sending module is further configured to obtain operating system information of the user terminal, and send a decoder obtaining request carrying the operating system information of the user terminal to the server, so that the server searches for a layered picture decoder matched with the operating system information of the user terminal according to the decoder obtaining request;
the receiving module is further configured to receive a layered picture decoder that is sent by the server and matches with the operating system information of the user terminal.
Wherein, still include:
the first judgment module is used for acquiring the size information of the display area and judging whether the size information meets the self-adaptive condition of the layered picture;
the sending module is further configured to send a second layered picture acquisition request to the server if the first determining module determines that the first layered picture is not satisfied, so that the server sequentially sends, according to the second layered picture acquisition request, base layer picture data and enhancement layer picture data in a layered picture with a maximum resolution;
and the notification module is used for notifying the sending module to send a first hierarchical picture acquisition request carrying the size information of the display area to a server if the first judgment module judges that the first hierarchical picture acquisition request is satisfied.
Wherein, still include:
the second judging module is used for judging whether the hierarchical picture format is supported or not;
the sending module is further configured to send an original picture obtaining request to the server if the second determining module determines that the second determination module does not support the original picture, so that the server returns the original picture according to the original picture obtaining request;
the decoding display module is further configured to decode the received original picture through an original decoder, and display the decoded original picture;
the notifying module is further configured to notify the first determining module to acquire size information of a display area if the second determining module determines that the display area is supported, and determine whether the size information satisfies a layered picture adaptive condition.
A fifth aspect of the present invention provides an image data processing apparatus comprising:
the device comprises a selection module, a display module and a display module, wherein the selection module is used for selecting a target layered picture matched with size information according to the size information of a display area of a user terminal carried by a first layered picture acquisition request if the first layered picture acquisition request sent by the user terminal is received;
a sending module, configured to send base layer picture data in the target layered picture to the user terminal, so that the user terminal decodes the base layer picture data through a layered picture decoder, and displays the decoded base layer picture data;
the sending module is further configured to send enhancement layer picture data in the target layered picture to the user terminal, so that the user terminal decodes the enhancement layer picture data through the layered picture decoder, synthesizes the decoded enhancement layer picture data with the decoded base layer picture data, and displays the synthesized picture data.
Wherein the selection module comprises:
the device comprises an extraction unit, a display unit and a control unit, wherein the extraction unit is used for extracting the size information of a display area of the user terminal carried by a first layered picture acquisition request if the first layered picture acquisition request sent by the user terminal is received;
the searching unit is used for searching at least one layered picture with the pixel width larger than the picture width in the size information from a plurality of layered pictures with different resolutions as at least one layered picture to be selected;
and the determining unit is used for determining the layered picture with the lowest resolution in the at least one layered picture to be selected as the target layered picture matched with the size information.
The selection module is further configured to select, if a second layered picture acquisition request sent by the user is received, a layered picture with a maximum resolution from among a plurality of layered pictures with different resolutions according to the second layered picture acquisition request;
the sending module is further configured to send the base layer picture data and the enhancement layer picture data in the layered picture with the maximum resolution to the user terminal in sequence;
the sending module is further configured to send an original picture to the user terminal according to the original picture obtaining request if the original picture obtaining request sent by the user is received;
wherein the second hierarchical picture acquisition request is sent by the user terminal upon detecting that the size information does not satisfy a hierarchical picture adaptation condition; the original picture acquisition request is sent by the user terminal when detecting that the layered picture format is not supported; the first hierarchical picture acquisition request is sent by the user terminal upon detecting that the size information satisfies a hierarchical picture adaptation condition.
Wherein, still include:
a receiving module, configured to receive a decoder acquisition request sent by the user terminal; the decoder acquires the operating system information of the user terminal carried by the request;
and the decoder searching module is used for searching the layered picture decoder matched with the operating system information of the user terminal according to the decoder obtaining request and sending the searched layered picture decoder matched with the operating system information of the user terminal to the user terminal.
Wherein, still include:
the decoding module is used for acquiring an original picture, and decoding the original picture through a layered picture decoder matched with the operating system information of the server to obtain decoded original picture data;
an encoding module, configured to encode the decoded original picture data into a layered picture of at least one resolution by a layered picture encoder; each layered picture includes base layer picture data and at least one enhancement layer picture data.
Wherein, still include:
the decoder setting module is used for setting a hierarchical picture decoder corresponding to at least one type of operating system information;
the first optimization setting module is used for optimizing prediction modes in the layered picture decoder and the layered picture encoder; the prediction mode comprises a coding unit division mode, a prediction unit division mode, a transformation unit division mode and an intra-frame prediction direction selection mode;
and the second optimization setting module is used for performing assembly optimization and code structure optimization on the hierarchical picture decoder according to the corresponding operating system information.
Wherein, the second optimization setting module comprises:
a first rule setting unit, configured to set a storage space selection optimization rule in the hierarchical picture decoder, where the storage space selection optimization rule includes a rule that a register is set as a temporary storage space of a highest priority, and the register is associated with operating system information corresponding to the hierarchical picture decoder;
a second rule setting unit configured to set a calculation optimization rule in the hierarchical picture decoder, the calculation optimization rule including a rule for converting a loop logic of a matrix operation into an operation based on a vector instruction set, the vector instruction set being associated with operating system information corresponding to the hierarchical picture decoder.
A sixth aspect of the present invention provides an image data processing apparatus comprising: a processor, memory, and a network interface;
the processor is connected with the memory and the network interface, wherein the network interface is used for connecting the server, the memory is used for storing program codes, and the processor is used for calling the program codes to execute the following operations:
sending a first layered picture acquisition request carrying size information of a display area to the server, so that the server selects a target layered picture matched with the size information according to the first layered picture acquisition request;
receiving base layer picture data in the target layered picture sent by the server, decoding the base layer picture data through a layered picture decoder, and displaying the decoded base layer picture data;
and receiving enhancement layer picture data in the target layered picture sent by the server, decoding the enhancement layer picture data through the layered picture decoder, synthesizing the decoded enhancement layer picture data and the decoded base layer picture data, and displaying the synthesized picture data.
A seventh aspect of the present invention provides an image data processing apparatus comprising: a processor, a memory, and a network interface;
the processor is connected with a memory and a network interface, wherein the network interface is used for connecting a user terminal, the memory is used for storing program codes, and the processor is used for calling the program codes to execute the following operations:
if a first layered picture acquisition request sent by the user terminal is received, selecting a target layered picture matched with the size information according to the size information of the display area of the user terminal carried by the first layered picture acquisition request;
sending the base layer picture data in the target layered picture to the user terminal so that the user terminal decodes the base layer picture data through a layered picture decoder and displays the decoded base layer picture data;
and sending the enhancement layer picture data in the target layered picture to the user terminal so that the user terminal decodes the enhancement layer picture data through the layered picture decoder, synthesizes the decoded enhancement layer picture data with the decoded base layer picture data, and displays the synthesized picture data.
An eighth aspect of the present invention provides a computer storage medium storing a computer program comprising program instructions that, when executed by a processor, perform the method of the first aspect of an embodiment of the present invention.
A ninth aspect of the present invention provides a computer storage medium storing a computer program comprising program instructions which, when executed by a processor, perform the method of the second aspect in an embodiment of the present invention.
The embodiment of the invention sends a first layered picture acquisition request carrying the size information of a display area to a server, so that the server selects a target layered picture matched with the size information according to the first layered picture acquisition request, receives base layer picture data in the target layered picture sent by the server, decodes the base layer picture data through a layered picture decoder, displays the decoded base layer picture data, receives enhancement layer picture data in the target layered picture sent by the server, decodes the enhancement layer picture data through the layered picture decoder, synthesizes the decoded enhancement layer picture data with the decoded base layer picture data, and displays the synthesized picture data. Therefore, even under the condition that the network state is unstable, the base layer picture data can be still obtained quickly, so that the display area can firstly display the content of the base layer picture data, the picture display area is prevented from being in a blank state for a long time, the enhancement layer picture data can be further received subsequently, the enhancement layer picture data and the base layer picture data are synthesized, a picture with high quality is obtained, and the picture display effect can be improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a schematic diagram of a network architecture provided by an embodiment of the present invention;
FIG. 2 is a timing diagram illustrating an image data processing method according to an embodiment of the present invention;
fig. 3a is a schematic diagram of a picture display of a user terminal according to an embodiment of the present invention;
fig. 3b is a schematic diagram of a picture display of another ue according to an embodiment of the present invention;
fig. 3c is a schematic diagram of a picture display of another user terminal according to an embodiment of the present invention;
FIG. 4 is a timing diagram illustrating another image data processing method according to an embodiment of the present invention;
fig. 5 is a flowchart illustrating an optimization method of a decoder according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a layered picture decoder according to an embodiment of the present invention;
FIG. 7 is a flowchart illustrating an image data processing method according to an embodiment of the present invention;
FIG. 8 is a flow chart illustrating another image data processing method according to an embodiment of the present invention;
FIG. 9 is a schematic diagram of an image data processing apparatus according to an embodiment of the present invention;
FIG. 10 is a schematic structural diagram of another image data processing apparatus according to an embodiment of the present invention;
FIG. 11 is a schematic structural diagram of another image data processing apparatus according to an embodiment of the present invention;
fig. 12 is a schematic structural diagram of another image data processing apparatus according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without inventive step based on the embodiments of the present invention, are within the scope of protection of the present invention.
Fig. 1 is a schematic diagram of a network architecture according to an embodiment of the present invention. As shown in fig. 1, the network architecture may include: a server 100, a server 200, and a plurality of user terminals 300. The server 100 may be connected to the plurality of user terminals 300 via the internet, and the server 200 may be connected to the plurality of user terminals 300 via the internet. As shown in fig. 1, the plurality of user terminals 300 may include terminal devices such as a computer, a tablet computer, a smart phone, a notebook computer, a palm computer, a digital television, and a Mobile Internet Device (MID), or may be clients in the terminal devices. The server 100 may be configured to set hierarchical picture decoders corresponding to multiple operating systems, and bring the set hierarchical picture decoders online, so that multiple user terminals 300 may acquire the hierarchical picture decoders matching the respective operating systems through the server 100. The server 200 may encode an original picture into a plurality of layered pictures with different resolutions, and when a user terminal 300 requests to acquire a layered picture, the server 200 may select a layered picture with a corresponding resolution according to size information of a display area in the user terminal 300, and first send base layer picture data in the selected layered picture, so that the user terminal 300 may first decode the base layer picture data through a layered picture decoder acquired from the server 100, and then the user terminal 300 may first display the base layer picture data with lower quality, so as to avoid that the picture display area is in a blank state for a long time; then, the server 200 may continue to send the enhancement layer picture data in the selected layered picture, so that the user terminal 300 decodes the enhancement layer picture data through the layered picture decoder obtained from the server 100, synthesizes the decoded enhancement layer picture data with the decoded base layer picture data, obtains a picture with higher quality, and displays the picture with higher quality, so as to improve the picture display effect.
The layered picture may include base layer picture data and at least one enhancement layer picture data, the base layer picture data may be low quality picture data capable of approximately displaying picture content, that is, the data amount of the base layer picture data is small, so that the base layer picture data may be transmitted to the user terminal 300 relatively quickly, and the enhancement layer picture data may be a difference amount between the high quality picture data and the base layer picture data, so that higher quality picture data may be obtained by superimposing the enhancement layer picture data and the base layer picture data. When a plurality of enhancement layer picture data exist, the quality of the picture can be improved by one level when one enhancement layer picture data is superposed, namely the picture is displayed in a gradual change process from fuzzy to clear. The layered picture decoder may be configured to decode a layered picture, that is, the layered picture decoder may decode base layer picture data, may also decode enhancement layer picture data, and may also synthesize the enhancement layer picture data and the base layer picture data.
Alternatively, the server 100 and the server 200 in fig. 1 may be combined into a same server, that is, the functions performed by the server 100 and the server 200, respectively, may be implemented on the same server.
Fig. 2 is a schematic timing diagram of an image data processing method according to an embodiment of the present invention, where the method includes:
s201, a user terminal sends a first layered picture acquisition request carrying size information of a display area to a server;
specifically, before step S201, the user terminal may send a decoder acquisition request carrying operating system information of the user terminal to the server, so that the server may search for a layered picture decoder matching the operating system information of the user terminal according to the decoder acquisition request, and then the server may send the searched layered picture decoder to the user terminal. The operating system information may include any one of microsoft system information, apple system information, android system information and other operating system information. It can be seen that the server may preset the layered picture decoders corresponding to the multiple operating systems, wherein the server may include the functions of the server 100 and the server 200 in the embodiment corresponding to fig. 1. Optionally, the server may further perform assembly optimization and code structure optimization on the hierarchical picture decoder, when the user terminal decodes using the optimized hierarchical picture decoder, the optimized hierarchical picture decoder will use as many registers of the operating system of the user terminal as possible as the temporary storage space, and the optimized hierarchical picture decoder will use as many vector operation instructions of the operating system of the user terminal as possible for decoding calculation. Therefore, different assembly optimization and code structure optimization are performed for different operating systems, so that the optimized layered picture decoder can be used by user terminals of various operating systems, and the decoding efficiency of the layered picture decoder is improved.
Further, after the user terminal acquires the corresponding layered picture decoder, the user terminal can further detect the size information of the display area and send a first layered picture acquisition request carrying the size information of the display area to the server to request the corresponding layered picture; among them, the layered picture may also be referred to as an X5 picture.
S202, the server selects a target layered picture matched with the size information according to the first layered picture acquisition request;
specifically, after the server acquires the first layered picture acquisition request, the server may extract size information of a display area of the user terminal in the first layered picture acquisition request, search for at least one layered picture having a pixel width larger than a picture width in the size information from a plurality of preset layered pictures with different resolutions, use the at least one layered picture as at least one layered picture to be selected, and determine a layered picture with a lowest resolution in the at least one layered picture to be selected as a target layered picture matched with the size information. The server may encode an original picture into layered pictures with multiple resolutions in advance, and the target layered picture is a layered picture with one of the resolutions. The server may determine, as the target layered picture, a layered picture having a pixel width greater than that of the size information and being adjacent to a numerical value of the size information among the layered pictures of the plurality of resolutions. For example, a certain original picture corresponds to a layered picture with 3 resolutions, which are a layered picture with a width of 810 pixels, a layered picture with a width of 610 pixels, and a layered picture with a width of 410 pixels, respectively, and if the size information is 800 pixels, the layered picture with a width of 810 pixels is determined as a target layered picture; if the size information is 600 pixels wide, determining the layered picture with the 610 pixels wide as a target layered picture; if the size information is 400 pixels wide, the layered picture with the width of 410 pixels is determined as the target layered picture.
The specific process of the server encoding a certain original picture into layered pictures with multiple resolutions in advance may be as follows: the server decodes the original picture through a layered picture decoder corresponding to the operating system information of the server to obtain original picture data which can support a layered picture format, namely the decoded original picture data, and then codes the decoded original picture data through a layered picture coder corresponding to the operating system of the server to generate layered pictures with various resolutions. The layered picture decoder and the layered picture encoder are both used for coding and decoding based on the video coding standard H.265.
S203, the server sends the basic layer picture data in the target layered picture to the user terminal;
specifically, after the server selects the target layered picture, the server may send the base layer picture data in the target layered picture to the user terminal, where the time length for transmitting the base layer picture data is shorter because the data size of the base layer picture data is smaller (that is, the saw teeth of the picture content are more obvious), so that the user may display the base layer picture data on the display area of the user terminal faster, and the display area is prevented from being in a blank state for a long time.
S204, the user terminal decodes the base layer picture data through a layered picture decoder and displays the decoded base layer picture data;
specifically, after receiving the base layer picture data, the user terminal may decode the base layer picture data through the obtained layered picture decoder. Because the hierarchical picture decoder performs assembly optimization and code structure optimization on the server side, the register corresponding to the operating system of the user terminal can be used as much as possible as a temporary storage space in the decoding process of the user terminal by using the hierarchical picture decoder, and the vector operation instruction corresponding to the operating system of the user terminal can be used as much as possible for decoding calculation, so that the decoding speed can be effectively improved.
After the user terminal decodes the base layer picture data, the decoded base layer picture data can be directly put into a display area for display. Because the data size of the picture data of the basic layer is small (namely, the saw teeth of the picture content are obvious), the time for transmitting the picture data of the basic layer is short, so that a user can display the picture data of the basic layer on a display area of a user terminal quickly, and the display area is prevented from being in a blank state for a long time.
S205, the server sends enhancement layer picture data in the target layered picture to the user terminal;
specifically, after the server finishes sending the base layer picture data in the target layered picture, the server can continue to send the enhancement layer picture data in the target layered picture to the user terminal. Or, after receiving the base layer picture data, the user terminal may return a reception confirmation instruction to the server, so that the server triggers execution of the transmission operation of the enhancement layer picture data according to the reception confirmation instruction. Since the enhancement layer picture data is a difference between the high-quality picture data and the base layer picture data, the data amount of the enhancement layer picture data is also small, and thus the transmission time for transmitting the enhancement layer picture data can also be shortened.
S206, the user terminal decodes the enhancement layer picture data through the layered picture decoder, synthesizes the decoded enhancement layer picture data with the decoded base layer picture data, and displays the synthesized picture data;
specifically, after receiving the enhancement layer picture data, the user terminal may decode the enhancement layer picture data through the layered picture decoder, and may also decode the enhancement layer picture data through a register and a vector operation instruction corresponding to an operating system of the user terminal in the decoding process, so as to improve the decoding speed. After decoding the enhancement layer picture data, the user terminal may synthesize the decoded enhancement layer picture data and the decoded base layer picture data, and display the synthesized picture data, where a definition of the synthesized picture data is higher than that of the base layer picture data.
Optionally, if the number of the enhancement layer picture data in the target layered picture in the server side is greater than 1, the server may continue to send the next enhancement layer picture data to the user terminal after sending the first enhancement layer picture data, so that the user terminal continues to synthesize the second enhancement layer picture data with the displayed picture to obtain a clearer synthesized picture, and by analogy, the server may sequentially send all the enhancement layer picture data, and the user terminal may sequentially synthesize all the enhancement layer picture data into the displayed picture, so that the picture displayed in the display area of the user terminal has a gradually clear trend of change.
Further, please refer to fig. 3a to fig. 3c together, which are schematic diagrams illustrating a picture display of a user terminal according to an embodiment of the present invention. As shown in fig. 3a, the picture displayed in the display area 500 of the ue 400 is the base layer picture data 600 in the target layered picture, and as can be seen from fig. 3a, the display effect of the base layer picture data 600 has a more obvious jaggy phenomenon, wherein the resolution of the target layered picture is 310 × 250, and the target layered picture includes one base layer picture data 600 and two enhancement layer picture data, then the resolution of the base layer picture data 600 displayed in the display area 500 is also 310 × 250. After the server has sent the base layer picture data 600, it may continue to send one of the enhancement layer picture data to the user terminal 400, and after the user terminal 400 decodes the enhancement layer picture data, it may synthesize the decoded enhancement layer picture data with the displayed base layer picture data 600 to obtain the synthesized picture 700 shown in fig. 3b, and update the base layer picture data 600 in fig. 3a to the synthesized picture 700, i.e., obtain the synthesized picture 700 displayed in the display area 500 shown in fig. 3b, where the resolution of the synthesized picture 700 is also 310 × 250, but the definition of the synthesized picture 700 is higher than that of the base layer picture data 600. After the server has sent one of the enhancement layer picture data, it may continue to send another enhancement layer picture data to the user terminal 400, and after the user terminal 400 decodes the enhancement layer picture data, it may synthesize the decoded enhancement layer picture data with the displayed composite picture 700 to obtain the final picture 800 shown in fig. 3c, and update the composite picture 700 in fig. 3b to the final picture 800, i.e., obtain the final picture 800 displayed in the display area 500 shown in fig. 3c, where the resolution of the final picture 800 is also 310 × 250, but the definition of the final picture 800 is higher than that of the composite picture 700. It can be seen that in the display area 500, the sharpness of the picture is gradually changed, i.e. the picture displayed in the display area 500 is changed from the base layer picture data 600 to the composite picture 700 and then to the final picture 800.
The embodiment of the invention sends a first layered picture acquisition request carrying the size information of a display area to a server, so that the server selects a target layered picture matched with the size information according to the first layered picture acquisition request, receives base layer picture data in the target layered picture sent by the server, decodes the base layer picture data through a layered picture decoder, displays the decoded base layer picture data, receives enhancement layer picture data in the target layered picture sent by the server, decodes the enhancement layer picture data through the layered picture decoder, synthesizes the decoded enhancement layer picture data with the decoded base layer picture data, and displays the synthesized picture data. Therefore, even under the condition that the network state is unstable, the base layer picture data can be still obtained quickly, so that the display area can firstly display the content of the base layer picture data, the picture display area is prevented from being in a blank state for a long time, the enhancement layer picture data can be further received subsequently, the enhancement layer picture data and the base layer picture data are synthesized, a picture with high quality is obtained, and the picture display effect can be improved. And the server performs assembly optimization and code structure optimization on the hierarchical picture decoders corresponding to the information of the operating systems respectively, so that the user terminals of the operating systems can use the optimized hierarchical picture decoders, and the decoding efficiency of the hierarchical picture decoders is improved.
Referring to fig. 4, a timing diagram of another image data processing method according to an embodiment of the present invention is shown, where the method includes:
s401, a user terminal acquires operating system information of the user terminal and sends a decoder acquisition request carrying the operating system information of the user terminal to a server;
specifically, after the user terminal starts an application program which can be used for displaying pictures, the operating system information of the user terminal can be further acquired, and a decoder acquisition request carrying the operating system information of the user terminal is sent to the server; the operating system information may include any one of microsoft system information, apple system information, android system information and other operating system information.
S402, the server searches a layered picture decoder matched with the operating system information of the user terminal according to the decoder acquisition request, and sends the layered picture decoder matched with the operating system information of the user terminal to the user terminal;
specifically, the server searches for a layered picture decoder matched with the operating system information of the user terminal according to the decoder acquisition request, and sends the layered picture decoder matched with the operating system information of the user terminal to the user terminal. It can be seen that the server may preset the layered picture decoders corresponding to the multiple operating systems, wherein the server may include the functions of the server 100 and the server 200 in the embodiment corresponding to fig. 1. Optionally, the server may further perform assembly optimization and code structure optimization on the hierarchical picture decoder, when the user terminal decodes using the optimized hierarchical picture decoder, the optimized hierarchical picture decoder will use as many registers of the operating system of the user terminal as possible as the temporary storage space, and the optimized hierarchical picture decoder will use as many vector operation instructions of the operating system of the user terminal as possible for decoding calculation. Therefore, different assembly optimization and code structure optimization are performed for different operating systems, so that the optimized layered picture decoder can be used by user terminals of various operating systems, and the decoding efficiency of the layered picture decoder is improved.
S403, the user terminal judges whether a layered picture format is supported;
specifically, after the user terminal acquires the layered picture decoder, it may be further determined whether the user terminal supports the layered picture format, that is, whether the user terminal can decode the layered picture through the layered picture decoder. The layered picture format may be a picture format including base layer picture data and at least one enhancement layer picture data, that is, a layered picture may include base layer picture data and at least one enhancement layer picture data, the base layer picture data may be low-quality picture data capable of substantially displaying picture content, and the enhancement layer picture data may be a difference between the high-quality picture data and the base layer picture data.
Optionally, before the step S401, the user terminal may first determine whether the user terminal supports the layered picture format, and if the user terminal supports the layered picture format, then execute S401-S402, and continue to execute the steps after S407; if not, the steps S404-S406 are directly executed. Therefore, the execution order of step S403 is not limited by the embodiment of the present invention.
S404, if the picture is judged not to be supported, the user terminal sends an original picture obtaining request to a server;
specifically, if it is determined in S403 that the image is not supported, the user terminal may further send an original image obtaining request to the server.
S405, the server returns the original picture according to the original picture acquisition request;
specifically, the server returns the original picture according to the original picture acquisition request, and further, the server may send the original decoder to the user terminal according to the original picture acquisition request. Alternatively, the server may transmit the layered picture decoder to the user terminal simultaneously with transmitting the original decoder to the user terminal. Alternatively, the user terminal itself carries the original decoder, and therefore, the server may not need to send the original decoder to the user terminal. The original decoder is a decoder that can be used to decode pictures in formats such as PNG (Portable Network Graphics), JPEG (Joint Photographic Experts Group), GIF (Graphics Interchange Format), and the like.
S406, the user terminal decodes the received original picture through an original decoder and displays the decoded original picture;
specifically, after the user terminal acquires the original picture and the original decoder, the received original picture can be decoded by the original decoder, and the decoded original picture is displayed on the display area.
S407, if the judgment is supporting, the user terminal acquires the size information of the display area and judges whether the size information meets the self-adaptive condition of the layered picture;
specifically, if the determination in S403 is that the layered picture is supported, that is, the user terminal can decode the layered picture, the user terminal can further obtain the size information of the display area, and determine whether the size information satisfies the layered picture adaptive condition. The specific process of determining whether the layered picture adaptive condition is satisfied may be: and detecting whether the size information is larger than a preset size threshold, if so, determining that the size information meets the self-adaptive condition of the layered pictures, otherwise, determining that the size information does not meet the self-adaptive condition of the layered pictures, and jumping to the steps of S414-S415.
Optionally, before determining whether the size information satisfies the layered picture adaptation condition, it may further detect whether the user terminal has configured an adaptive protocol, and if it is detected that the user terminal has not configured the adaptive protocol, the steps of S414-S415 may be directly performed; if the adaptive protocol is detected to be configured, the step of judging whether the size information meets the adaptive condition of the layered picture can be executed. The adaptive protocol may refer to a protocol commonly deployed between the user terminal and the server at the same time, and the user terminal configured with the adaptive protocol may trigger execution of the step of determining whether the size information satisfies the layered picture adaptive condition, and further trigger the step of S408; and the server configured with the adaptive protocol may identify and respond to the first hierarchical picture acquisition request in step S408, thereby triggering step S409.
S408, if judging that the self-adaptive condition is met, the user terminal sends a first hierarchical picture acquisition request carrying the size information of the display area to a server;
s409, the server selects a target layered picture matched with the size information according to the first layered picture acquisition request;
s410, the server sends the basic layer picture data in the target layered picture to the user terminal;
s411, the user terminal decodes the base layer picture data through a layered picture decoder and displays the decoded base layer picture data;
s412, the server sends the enhancement layer picture data in the target layered picture to the user terminal.
S413, the user terminal decodes the enhancement layer picture data through the layered picture decoder, synthesizes the decoded enhancement layer picture data with the decoded base layer picture data, and displays the synthesized picture data;
the specific implementation manner of steps S408-S413 may refer to steps S201-S206 in the corresponding embodiment of fig. 2, which is not described herein again.
S414, if the self-adaptive condition is judged not to be met, the user terminal sends a second hierarchical picture acquisition request to the server;
specifically, if it is determined in S407 that the size information does not satisfy the adaptive condition, or the user terminal detects that the adaptive protocol is not configured, the user terminal may further send a second layered picture acquisition request to the server, where the second layered picture acquisition request does not carry the size information.
S415, the server sequentially sends the base layer picture data and the enhancement layer picture data in the layered picture with the maximum resolution according to the second layered picture acquisition request.
Specifically, the server may obtain, according to the second layered picture obtaining request, a layered picture with a maximum resolution from among layered pictures with a plurality of preset resolutions, and send base layer picture data in the layered picture with the maximum resolution to the user terminal, so that the user terminal decodes and displays the base layer picture data; after the server sends the base layer picture data in the layered picture with the maximum resolution, the server can continue to send the enhancement layer picture data in the layered picture with the maximum resolution to the user terminal, so that the user terminal decodes the enhancement layer picture data, and synthesizes the decoded enhancement layer picture data with the displayed base layer picture data to display the final high-quality picture. The specific process of decoding and displaying the base layer picture data and the enhancement layer picture data in the layered picture with the maximum resolution by the user terminal can refer to steps S408 to S413.
The embodiment of the invention sends a first layered picture acquisition request carrying the size information of a display area to a server, so that the server selects a target layered picture matched with the size information according to the first layered picture acquisition request, receives base layer picture data in the target layered picture sent by the server, decodes the base layer picture data through a layered picture decoder, displays the decoded base layer picture data, receives enhancement layer picture data in the target layered picture sent by the server, decodes the enhancement layer picture data through the layered picture decoder, synthesizes the decoded enhancement layer picture data with the decoded base layer picture data, and displays the synthesized picture data. Therefore, even under the condition that the network state is unstable, the base layer picture data can be still obtained quickly, so that the display area can display the content of the base layer picture data firstly, the picture display area is prevented from being in a blank state for a long time, the enhancement layer picture data can be further received subsequently, the enhancement layer picture data and the base layer picture data are synthesized, a picture with high quality is obtained, and the picture display effect can be improved. And the server performs assembly optimization and code structure optimization on the hierarchical picture decoders corresponding to the information of the operating systems respectively, so that the user terminals of the operating systems can use the optimized hierarchical picture decoders, and the decoding efficiency of the hierarchical picture decoders is improved.
Further, referring to fig. 5 together, a flowchart of a method for optimizing a decoder according to an embodiment of the present invention is shown, where the method may be executed before S201 in the embodiment corresponding to fig. 2, or may be executed before S401 in the embodiment corresponding to fig. 4, and the method may include:
s501, setting a hierarchical picture decoder corresponding to at least one operating system information respectively;
specifically, the server may preset a hierarchical picture decoder corresponding to at least one type of operating system information. The server can also preset a layered picture encoder corresponding to the operating system of the server, so that the server can pre-encode the original picture into layered pictures with various resolutions through the layered picture encoder.
S502, optimizing prediction modes in a layered picture decoder and a preset layered picture encoder; the prediction mode comprises a coding unit division mode, a prediction unit division mode, a transformation unit division mode and an intra-frame prediction direction selection mode;
specifically, the server may further optimize prediction modes in each layered picture decoder and a preset layered picture encoder, where the prediction modes include a coding unit division mode, a prediction unit division mode, a transform unit division mode, and an intra prediction direction selection mode.
The process of optimizing the coding unit division mode by the server may be: the optimal Coding size of a current CU (Coding Unit) is quickly judged by using depth information of Coding units related in time and space, the optimal extraction of the depth information of the CU at the same position of a previous frame and around the current frame is carried out, a judgment interval of CU partition is reduced, the partition depth of the current CU is predicted, the prediction speed of the CU partition depth is improved, unnecessary calculation is avoided, and the calculation complexity is reduced.
The process of optimizing the prediction unit partition mode by the server may be: before rate distortion optimization is carried out on a CU layer, main factors influencing mode selection of a PU (Prediction Unit) are obtained and analyzed, and therefore several modes with the highest probability are selected as candidate PU dividing modes. Where PU is the basic unit of prediction, a CU may contain one or more PUs, and PU maximum size is the size of the CU.
Among them, the TU (Transform Unit) distribution of the residual block with different texture characteristics exhibits a certain rule: when the texture of the residual block is complex and the motion vector changes violently, the possibility that the TU selects a large size as the best segmentation is very small; conversely, when the texture of the residual block is simpler, the TU always selects the larger partition. Therefore, the process of optimizing the transformation unit division manner by the server may be as follows: in the TU decision algorithm, the texture of the residual block is analyzed, if the texture is simpler, the segmentation of the TU is terminated in advance, and the optimization can greatly reduce the coding and decoding time. Wherein, TU is the basic unit of transform and quantization, and a CU may contain one or more TUs, and TU also adopts a quadtree recursive partitioning structure, which may be larger than PU but not exceeding the size of CU.
The process of optimizing the intra-frame prediction direction selection mode by the server may be as follows: and predicting the possible division of the current block according to the division condition of the coded block adjacent to the current region to be coded, and removing unnecessary coding unit size and prediction mode decision process. The optimization can effectively reduce the complexity of coding and decoding computation and greatly improve the coding and decoding operation efficiency under the condition of ensuring that the coding quality and the compression efficiency are basically unchanged.
Therefore, by optimizing the prediction modes of the layered picture decoder and the layered picture encoder, the encoding efficiency can be improved when the layered picture encoder encodes a layered picture, and the decoding efficiency can also be improved when the layered picture decoder decodes the layered picture.
S503, performing assembly optimization and code structure optimization on the layered picture decoder according to the corresponding operating system information;
specifically, the server may further perform assembly optimization and code structure optimization on the layered picture decoders according to the corresponding operating system information, that is, perform different assembly optimization and code structure optimization on the layered picture decoders corresponding to different operating system information. The specific process of assembly optimization and code structure optimization can be as follows: setting a storage space selection optimization rule in the hierarchical picture decoder, wherein the storage space selection optimization rule comprises a rule of setting a register as a temporary storage space with the highest priority, and the register is associated with the operating system information corresponding to the hierarchical picture decoder; and setting a calculation optimization rule in the layered picture decoder, wherein the calculation optimization rule comprises a rule for converting the circulation logic of the matrix operation into operation based on a vector instruction set, and the vector instruction set is associated with the operating system information corresponding to the layered picture decoder. Therefore, when the user terminal decodes by using the hierarchical picture decoder provided with the storage space selection optimization rule and the calculation optimization rule, the register corresponding to the operating system of the user terminal can be used as a temporary storage space as much as possible, and the vector operation instruction corresponding to the operating system of the user terminal can be used as much as possible for decoding calculation, so that the decoding speed can be effectively improved. The vector operation instruction may include a vector data load instruction, a vector data interleave instruction, a vector data swap instruction, and a vector data store instruction.
Optionally, the server may perform assembly optimization on the layered picture encoder, in a manner similar to that of the layered picture decoder.
Further, please refer to fig. 6 together, which is a schematic structural diagram of a layered picture decoder according to an embodiment of the present invention, as shown in fig. 6, the layered picture decoder may include an Entropy Decoding (Entropy Decoding), an IQ/IDCT (Inverse Quantization/Inverse Cosine Transform), a Deblocking Filter (Deblocking Filtering), an Intra Prediction (Intra Prediction), an Intra/Inter Mode Selection (Intra/Inter Mode Selection), an SAO Filtering (Sample Adaptive Offset Filtering), a Motion Compensation (MC), and a picture buffering (picture buffering). Wherein, the control Decoding may be used to read the picture data and parse out a data structure, which may include a macroblock type, a motion vector, a reference frame, a residual, and the like; IQ/IDCT may be used to transform the entropy decoded data to obtain values of the spatial domain; the Deblocking Filter can be used for filtering data; the Intra Prediction may be used for performing Intra Prediction computations; the Intra/Inter Mode Selection may be used to select the corresponding Intra prediction and Inter prediction modes; the SAO filtering can be used for reducing the sampling distortion of the image and increasing the definition of the image; motion Compensation can be used to perform Motion Compensation processing on subsequent frame images based on existing frame image data, optimizing the subsequently decoded images. Wherein, picture buffering may refer to a decoded image buffer. With the progress of image decoding, new images are generated continuously, so that the decoded images can be put into picture buffering, and further, when the images are output and displayed, image data can be extracted from the picture buffering according to the sequence, so as to render and output the images on a mobile phone screen. When decoding an image, the Intra Prediction (i.e., the Intra Prediction calculation) may be performed first, and then the Intra/Inter Mode Selection calculation may be performed according to the Intra Prediction data and the image data in the picture buffering. For example, the control Decoding processing may be performed on the base layer picture data first, and other Decoding processing may be performed in the layered picture decoder to generate decoded base layer picture data, and the decoded base layer picture data is further placed in picture buffering, and the decoded base layer picture data is output to a mobile phone screen for display at the same time; after the enhancement layer picture data is subsequently decoded, the decoded enhancement layer picture data and the decoded base layer picture data in picture buffering can be merged, and the decoded base layer picture data in the mobile phone screen is replaced by the merged picture data.
The assembly optimization performed on the hierarchical picture decoder may specifically refer to deep assembly optimization performed on modules called with high frequency (such as control Decoding, IQ/IDCT, deblocking Filter, intra Prediction, intra/Inter Mode Selection, SAO filtering, motion Compensation, and the like).
The embodiment of the invention can effectively improve the coding and decoding speed by optimizing the prediction modes of the layered picture decoder and the layered picture encoder; and further performing assembly optimization and code structure optimization on the layered picture decoder, so that registers corresponding to an operating system of the user terminal can be used as a temporary storage space as much as possible in the decoding process, and decoding calculation can be performed as much as possible by using vector operation instructions corresponding to the operating system of the user terminal, thereby further improving the decoding speed.
Referring to fig. 7, a flowchart of an image data processing method according to an embodiment of the present invention is shown, where the method includes:
s701, acquiring operating system information of a user terminal, and sending a decoder acquisition request carrying the operating system information of the user terminal to the server, so that the server searches a layered picture decoder matched with the operating system information of the user terminal according to the decoder acquisition request;
s702, receiving a layered picture decoder which is sent by the server and matched with the operating system information of the user terminal;
s703, judging whether a layered picture format is supported or not;
s704, if the picture is judged not to be supported, sending an original picture obtaining request to the server so that the server returns the original picture according to the original picture obtaining request;
s705, decoding the received original picture through an original decoder, and displaying the decoded original picture;
s706, if the judgment is supported, acquiring the size information of the display area, and judging whether the size information meets the self-adaptive condition of the layered picture;
s707, if the judgment result is yes, sending a first layered picture acquisition request carrying the size information of the display area to a server, so that the server selects a target layered picture matched with the size information according to the first layered picture acquisition request;
s708, receiving base layer picture data in the target layered picture sent by the server, decoding the base layer picture data through a layered picture decoder, and displaying the decoded base layer picture data;
s709, receiving the enhancement layer picture data in the target layered picture sent by the server, decoding the enhancement layer picture data by the layered picture decoder, synthesizing the decoded enhancement layer picture data with the decoded base layer picture data, and displaying the synthesized picture data;
s710, if the determination result is that the first layered picture data is not satisfied, sending a second layered picture acquisition request to the server, so that the server sequentially sends, according to the second layered picture acquisition request, base layer picture data and enhancement layer picture data in a layered picture with a maximum resolution;
the steps S701 to S710 describe all operations at the side of the ue, and specific implementation manners of the steps S701 to S710 may refer to the specific descriptions of the ue in S401 to S415 in the embodiment corresponding to fig. 4, which is not described herein again.
The embodiment of the invention sends a first layered picture acquisition request carrying the size information of a display area to a server, so that the server selects a target layered picture matched with the size information according to the first layered picture acquisition request, receives base layer picture data in the target layered picture sent by the server, decodes the base layer picture data through a layered picture decoder, displays the decoded base layer picture data, receives enhancement layer picture data in the target layered picture sent by the server, decodes the enhancement layer picture data through the layered picture decoder, synthesizes the decoded enhancement layer picture data with the decoded base layer picture data, and displays the synthesized picture data. Therefore, even under the condition that the network state is unstable, the base layer picture data can be still obtained quickly, so that the display area can firstly display the content of the base layer picture data, the picture display area is prevented from being in a blank state for a long time, the enhancement layer picture data can be further received subsequently, the enhancement layer picture data and the base layer picture data are synthesized, a picture with high quality is obtained, and the picture display effect can be improved.
Referring to fig. 8, a schematic flow chart of another image data processing method according to an embodiment of the present invention is shown, where the method includes:
s801, acquiring an original picture, and decoding the original picture through a layered picture decoder matched with operating system information of a server to obtain decoded original picture data;
s802, encoding the decoded original picture data into a layered picture with at least one resolution through a layered picture encoder; each layered picture comprises base layer picture data and at least one enhancement layer picture data;
specifically, the server decodes the original picture through a layered picture decoder corresponding to the operating system information of the server to obtain original picture data capable of supporting a layered picture format, that is, decoded original picture data, and then the server encodes the decoded original picture data through a layered picture encoder corresponding to the operating system of the server to generate layered pictures with multiple resolutions. The layered picture decoder and the layered picture encoder are both used for coding and decoding based on the video coding standard H.265.
S803, receiving a decoder acquisition request sent by the user terminal; the decoder acquires the operating system information of the user terminal carried by the request;
s804, searching a layered picture decoder matched with the operating system information of the user terminal according to the decoder acquisition request, and sending the searched layered picture decoder matched with the operating system information of the user terminal to the user terminal;
s805, if a first layered picture acquisition request sent by a user terminal is received, selecting a target layered picture matched with size information according to the size information of a display area of the user terminal carried by the first layered picture acquisition request;
if the server receives the first layered picture acquisition request sent by the user terminal, it may also indicate that the server has reached a preset agreement with the user terminal, where the preset agreement may refer to an agreement that the user terminal may request a self-adaptive layered picture.
S806, sending the base layer picture data in the target layered picture to the user terminal, so that the user terminal decodes the base layer picture data through a layered picture decoder, and displays the decoded base layer picture data;
s807, sending the enhancement layer picture data in the target layered picture to the user terminal, so that the user terminal decodes the enhancement layer picture data through the layered picture decoder, synthesizes the decoded enhancement layer picture data with the decoded base layer picture data, and displays the synthesized picture data;
s808, if a second layered picture acquisition request sent by the user is received, selecting a layered picture with the maximum resolution from a plurality of layered pictures with different resolutions according to the second layered picture acquisition request, and sequentially sending the base layer picture data and the enhancement layer picture data in the layered picture with the maximum resolution to the user terminal;
s809, if receiving an original picture acquisition request sent by the user, sending an original picture to the user terminal according to the original picture acquisition request;
wherein the second hierarchical picture acquisition request is sent by the user terminal upon detecting that the size information does not satisfy a hierarchical picture adaptation condition; the original picture acquisition request is sent by the user terminal when detecting that the layered picture format is not supported; the first hierarchical picture acquisition request is sent by the user terminal upon detecting that the size information satisfies a hierarchical picture adaptation condition.
The specific implementation manner of steps S803 to S809 may refer to the specific description of the server in S401 to S415 in the corresponding embodiment of fig. 4, and is not described herein again.
Optionally, before the step of S801, the steps S501 to S503 in the embodiment corresponding to fig. 5 may also be executed.
The embodiment of the invention sends a first layered picture acquisition request carrying the size information of a display area to a server, so that the server selects a target layered picture matched with the size information according to the first layered picture acquisition request, receives base layer picture data in the target layered picture sent by the server, decodes the base layer picture data through a layered picture decoder, displays the decoded base layer picture data, receives enhancement layer picture data in the target layered picture sent by the server, decodes the enhancement layer picture data through the layered picture decoder, synthesizes the decoded enhancement layer picture data with the decoded base layer picture data, and displays the synthesized picture data. Therefore, even under the condition that the network state is unstable, the base layer picture data can be still obtained quickly, so that the display area can firstly display the content of the base layer picture data, the picture display area is prevented from being in a blank state for a long time, the enhancement layer picture data can be further received subsequently, the enhancement layer picture data and the base layer picture data are synthesized, a picture with high quality is obtained, and the picture display effect can be improved. Moreover, the server is used for carrying out assembly optimization and code structure optimization on the hierarchical picture decoders corresponding to the information of the operating systems respectively, so that the user terminals of the operating systems can use the optimized hierarchical picture decoders, and the decoding efficiency of the hierarchical picture decoders is improved.
Referring to fig. 9, which is a schematic structural diagram of an image data processing apparatus according to an embodiment of the present invention, the image data processing apparatus 1 may include: a sending module 11, a receiving module 12 and a decoding display module 13;
the sending module 11 is configured to send a first hierarchical picture acquisition request carrying size information of a display area to a server, so that the server selects a target hierarchical picture matched with the size information according to the first hierarchical picture acquisition request;
the receiving module 12 is configured to receive base layer picture data in the target layered picture sent by the server;
the decoding display module 13 is configured to decode the base layer picture data through a layered picture decoder, and display the decoded base layer picture data;
the receiving module 12 is further configured to receive enhancement layer picture data in the target layered picture sent by the server;
the decoding and displaying module 13 is further configured to decode the enhancement layer picture data through the layered picture decoder, synthesize the decoded enhancement layer picture data and the decoded base layer picture data, and display the synthesized picture data.
The sending module 11 is further configured to obtain operating system information of the user terminal, and send a decoder obtaining request carrying the operating system information of the user terminal to the server, so that the server searches for a layered picture decoder matched with the operating system information of the user terminal according to the decoder obtaining request;
the receiving module 12 is further configured to receive a layered picture decoder that is sent by the server and matches with the operating system information of the user terminal.
Further, as shown in fig. 9, the image data processing apparatus 1 may further include: a first judging module 14, a second judging module 15 and a notification module 16;
the first judging module 14 is configured to obtain size information of a display area, and judge whether the size information meets a layered picture self-adaptive condition;
the sending module 11 is further configured to send a second layered picture obtaining request to the server if the first determining module 14 determines that the first layered picture is not satisfied, so that the server sequentially sends, according to the second layered picture obtaining request, the base layer picture data and the enhancement layer picture data in the layered picture with the maximum resolution;
the notifying module 16 is configured to notify the sending module 11 to send a first hierarchical picture acquiring request carrying size information of a display area to a server if the first determining module 14 determines that the size information is satisfied.
The second judging module 15 is configured to judge whether a layered picture format is supported;
the sending module 11 is further configured to send an original picture obtaining request to the server if the second determining module 15 determines that the original picture is not supported, so that the server returns the original picture according to the original picture obtaining request;
the decoding display module 13 is further configured to decode the received original picture through an original decoder, and display the decoded original picture;
the notifying module 16 is further configured to notify the first determining module 14 to obtain size information of a display area if the second determining module 15 determines that the display area is supported, and determine whether the size information satisfies a layered picture adaptive condition.
For specific functional implementation manners of the sending module 11, the receiving module 12, the decoding and displaying module 13, the first determining module 14, the second determining module 15, and the notifying module 16, reference may be made to the specific description of the user terminal in S401-S415 in the embodiment corresponding to fig. 4, which is not described herein again.
The embodiment of the invention sends a first layered picture acquisition request carrying the size information of a display area to a server, so that the server selects a target layered picture matched with the size information according to the first layered picture acquisition request, receives base layer picture data in the target layered picture sent by the server, decodes the base layer picture data through a layered picture decoder, displays the decoded base layer picture data, receives enhancement layer picture data in the target layered picture sent by the server, decodes the enhancement layer picture data through the layered picture decoder, synthesizes the decoded enhancement layer picture data with the decoded base layer picture data, and displays the synthesized picture data. Therefore, even under the condition that the network state is unstable, the base layer picture data can be still obtained quickly, so that the display area can display the content of the base layer picture data firstly, the picture display area is prevented from being in a blank state for a long time, the enhancement layer picture data can be further received subsequently, the enhancement layer picture data and the base layer picture data are synthesized, a picture with high quality is obtained, and the picture display effect can be improved.
Referring to fig. 10, which is a schematic structural diagram of another image data processing apparatus according to an embodiment of the present invention, the image data processing apparatus 2 may include: a selection module 21 and a sending module 22;
the selection module 21 is configured to, if a first hierarchical picture acquisition request sent by a user terminal is received, select a target hierarchical picture matched with size information according to the size information of the display area of the user terminal carried by the first hierarchical picture acquisition request;
the sending module 22 is configured to send the base layer picture data in the target layered picture to the user terminal, so that the user terminal decodes the base layer picture data through a layered picture decoder, and displays the decoded base layer picture data;
the sending module 22 is further configured to send enhancement layer picture data in the target layered picture to the user terminal, so that the user terminal decodes the enhancement layer picture data through the layered picture decoder, synthesizes the decoded enhancement layer picture data with the decoded base layer picture data, and displays the synthesized picture data.
The selecting module 21 is further configured to, if a second layered picture acquiring request sent by the user is received, select, according to the second layered picture acquiring request, a layered picture with a maximum resolution from among a plurality of layered pictures with different resolutions;
the sending module 22 is further configured to send, to the user terminal, the base layer picture data and the enhancement layer picture data in the layered picture with the maximum resolution in sequence;
the sending module 22 is further configured to send an original picture to the user terminal according to the original picture obtaining request if the original picture obtaining request sent by the user is received;
wherein the second hierarchical picture acquisition request is sent by the user terminal upon detecting that the size information does not satisfy a hierarchical picture adaptation condition; the original picture acquisition request is sent by the user terminal when detecting that the layered picture format is not supported; the first hierarchical picture acquisition request is sent by the user terminal upon detecting that the size information satisfies a hierarchical picture adaptation condition.
Further, as shown in fig. 10, the image data processing apparatus 2 may further include: the device comprises a receiving module 25, a decoder searching module 26, a decoding module 23, an encoding module 24, a decoder setting module 27, a first optimization setting module 28 and a second optimization setting module 29;
the receiving module 25 is configured to receive a decoder acquisition request sent by the user terminal; the decoder acquires the operating system information of the user terminal carried by the request;
the decoder searching module 26 is configured to search, according to the decoder obtaining request, a layered picture decoder matched with the operating system information of the user terminal, and send the searched layered picture decoder matched with the operating system information of the user terminal to the user terminal.
The decoding module 23 is configured to acquire an original picture, and decode the original picture through a layered picture decoder that matches with the operating system information of the server to obtain decoded original picture data;
the encoding module 24 is configured to encode the decoded original picture data into a layered picture with at least one resolution through a layered picture encoder; each layered picture includes base layer picture data and at least one enhancement layer picture data.
The decoder setting module 27 is configured to set a hierarchical picture decoder corresponding to at least one operating system information;
the first optimization setting module 28 is configured to optimize prediction modes in the layered picture decoder and the layered picture encoder; the prediction mode comprises a coding unit division mode, a prediction unit division mode, a transformation unit division mode and an intra-frame prediction direction selection mode;
the second optimization setting module 29 is configured to perform assembly optimization and code structure optimization on the hierarchical picture decoder according to the corresponding operating system information.
For specific functional implementation manners of the selecting module 21, the sending module 22, the receiving module 25, the decoder searching module 26, the decoding module 23, and the encoding module 24, reference may be made to the specific description of the server in S401-S415 in the embodiment corresponding to fig. 4, which is not described herein again. For specific functional implementation manners of the decoder setting module 27, the first optimization setting module 28, and the second optimization setting module 29, reference may be made to S501 to S503 in the embodiment corresponding to fig. 5, which is not described herein again.
Further, as shown in fig. 10, the second optimization setting module 29 may include: a first rule setting unit 291, a second rule setting unit 292;
the first rule setting unit 291, configured to set a storage space selection optimization rule in the hierarchical picture decoder, where the storage space selection optimization rule includes a rule that a register is set as a temporary storage space with a highest priority, where the register is associated with operating system information corresponding to the hierarchical picture decoder;
the second rule setting unit 292 is configured to set a calculation optimization rule in the hierarchical picture decoder, where the calculation optimization rule includes a rule for converting a loop logic of a matrix operation into an operation based on a vector instruction set, and the vector instruction set is associated with operating system information corresponding to the hierarchical picture decoder.
For specific functional implementation manners of the first rule setting unit 291 and the second rule setting unit 292, reference may be made to S503 in the embodiment corresponding to fig. 5, which is not described herein again.
Further, as shown in fig. 10, the selecting module 21 may include: an extraction unit 211, a search unit 212, and a determination unit 213;
the extracting unit 211 is configured to, if a first hierarchical picture acquisition request sent by a user terminal is received, extract size information of a display area of the user terminal, where the size information is carried by the first hierarchical picture acquisition request;
the searching unit 212 is configured to search, among a plurality of layered pictures with different resolutions, at least one layered picture with a pixel width larger than the picture width in the size information as at least one layered picture to be selected;
the determining unit 213 is configured to determine a layered picture with a lowest resolution in the at least one layered picture to be selected as a target layered picture matched with the size information.
For specific functional implementation manners of the extracting unit 211, the searching unit 212, and the determining unit 213, reference may be made to S202 in the corresponding embodiment of fig. 2, which is not described herein again.
The embodiment of the invention sends a first layered picture acquisition request carrying the size information of a display area to a server, so that the server selects a target layered picture matched with the size information according to the first layered picture acquisition request, receives base layer picture data in the target layered picture sent by the server, decodes the base layer picture data through a layered picture decoder, displays the decoded base layer picture data, receives enhancement layer picture data in the target layered picture sent by the server, decodes the enhancement layer picture data through the layered picture decoder, synthesizes the decoded enhancement layer picture data with the decoded base layer picture data, and displays the synthesized picture data. Therefore, even under the condition that the network state is unstable, the base layer picture data can be still obtained quickly, so that the display area can display the content of the base layer picture data firstly, the picture display area is prevented from being in a blank state for a long time, the enhancement layer picture data can be further received subsequently, the enhancement layer picture data and the base layer picture data are synthesized, a picture with high quality is obtained, and the picture display effect can be improved. And the server performs assembly optimization and code structure optimization on the hierarchical picture decoders corresponding to the information of the operating systems respectively, so that the user terminals of the operating systems can use the optimized hierarchical picture decoders, and the decoding efficiency of the hierarchical picture decoders is improved.
Referring to fig. 11, which is a schematic structural diagram of another image data processing apparatus according to an embodiment of the present invention, as shown in fig. 11, the image data processing apparatus 1000 may be applied to the user terminal 100 in the embodiment corresponding to fig. 1, and the image data processing apparatus 1000 may include: the processor 1001, the memory 1005, and the network interface 1004, and the image data processing apparatus 1000 may further include: a user interface 1003, and at least one communication bus 1002. The communication bus 1002 is used to implement connection communication among these components. The optional user interface 1003 may include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory (non-volatile memory), such as at least one disk memory. The memory 1005 may optionally be at least one memory device located remotely from the processor 1001. As shown in fig. 11, the memory 1005, which is a kind of computer storage medium, may include therein an operating system, a network communication module, a user interface module, and a device control application program.
In the image data processing apparatus 1000 shown in fig. 11, the network interface 1004 may provide a network communication function, and the network interface 1004 may be connected to a server; the user interface 1003 is an interface for providing input to a user; and the processor 1001 may be configured to invoke the device control application stored in the memory 1005 to implement:
sending a first hierarchical picture acquisition request carrying size information of a display area to a server, so that the server selects a target hierarchical picture matched with the size information according to the first hierarchical picture acquisition request;
receiving base layer picture data in the target layered picture sent by the server, decoding the base layer picture data through a layered picture decoder, and displaying the decoded base layer picture data;
and receiving enhancement layer picture data in the target layered picture sent by the server, decoding the enhancement layer picture data through the layered picture decoder, synthesizing the decoded enhancement layer picture data and the decoded base layer picture data, and displaying the synthesized picture data.
In one embodiment, the processor 1001 may further perform the following steps:
acquiring operating system information of a user terminal, and sending a decoder acquisition request carrying the operating system information of the user terminal to the server, so that the server searches a layered picture decoder matched with the operating system information of the user terminal according to the decoder acquisition request;
and receiving a layered picture decoder which is sent by the server and matched with the operating system information of the user terminal.
In one embodiment, before the processor 1001 sends the first hierarchical picture acquisition request carrying the size information of the display area to the server, the following steps may be further performed:
acquiring size information of a display area, and judging whether the size information meets a layered picture self-adaption condition;
if the judgment result is that the hierarchical picture data is not satisfied, sending a second hierarchical picture acquisition request to the server so that the server sequentially sends the base layer picture data and the enhancement layer picture data in the hierarchical picture with the maximum resolution according to the second hierarchical picture acquisition request;
and if the judgment result is yes, executing the step of sending the first hierarchical picture acquisition request carrying the size information of the display area to the server.
In one embodiment, before the processor 1001 obtains the size information of the display area, the following steps may be further performed:
judging whether a layered picture format is supported or not;
if the original picture is judged not to be supported, sending an original picture acquisition request to the server so that the server returns the original picture according to the original picture acquisition request;
decoding the received original picture through an original decoder, and displaying the decoded original picture;
and if the judgment result is that the display area is supported, the step of acquiring the size information of the display area is executed.
The embodiment of the invention sends a first layered picture acquisition request carrying the size information of a display area to a server, so that the server selects a target layered picture matched with the size information according to the first layered picture acquisition request, receives base layer picture data in the target layered picture sent by the server, decodes the base layer picture data through a layered picture decoder, displays the decoded base layer picture data, receives enhancement layer picture data in the target layered picture sent by the server, decodes the enhancement layer picture data through the layered picture decoder, synthesizes the decoded enhancement layer picture data with the decoded base layer picture data, and displays the synthesized picture data. Therefore, even under the condition that the network state is unstable, the base layer picture data can be still obtained quickly, so that the display area can display the content of the base layer picture data firstly, the picture display area is prevented from being in a blank state for a long time, the enhancement layer picture data can be further received subsequently, the enhancement layer picture data and the base layer picture data are synthesized, a picture with high quality is obtained, and the picture display effect can be improved.
Fig. 12 is a schematic structural diagram of another image data processing apparatus according to an embodiment of the present invention. As shown in fig. 12, the image data processing apparatus 2000 may be applied to a server that may include all the functions of the server 100 and the server 200 in the corresponding embodiment of fig. 1 described above. The image data processing apparatus 2000 may include: a processor 2001, a network interface 2003, and a memory 2004, and the image data processing apparatus 2000 may further include: at least one communication bus 2002. The communication bus 2002 is used to implement connection communication between these components. The network interface 2003 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). Memory 2004 may be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The memory 2004 may alternatively be at least one memory device located remotely from the aforementioned processor 2001. As shown in fig. 12, the memory 2004, which is a type of computer storage medium, may include therein an operating system, a network communication module, a user interface module, and a device control application program.
In the image data processing apparatus 2000 shown in fig. 12, the network interface 2003 is used for communication with a user terminal; and the processor 2001 may be used to invoke the device control application stored in the memory 2004 to implement:
if a first layered picture acquisition request sent by a user terminal is received, selecting a target layered picture matched with size information according to the size information of a display area of the user terminal carried by the first layered picture acquisition request;
sending the base layer picture data in the target layered picture to the user terminal so that the user terminal decodes the base layer picture data through a layered picture decoder and displays the decoded base layer picture data;
and sending the enhancement layer picture data in the target layered picture to the user terminal so that the user terminal decodes the enhancement layer picture data through the layered picture decoder, synthesizes the decoded enhancement layer picture data with the decoded base layer picture data, and displays the synthesized picture data.
In an embodiment, when the processor 2001 performs that, if a first hierarchical picture acquisition request sent by a user terminal is received, and a target hierarchical picture matched with size information of a display area of the user terminal carried by the first hierarchical picture acquisition request is selected according to the size information, the following steps may be specifically performed:
if a first layered picture acquisition request sent by a user terminal is received, extracting size information of a display area of the user terminal carried by the first layered picture acquisition request;
searching at least one layered picture with the pixel width larger than the picture width in the size information from a plurality of layered pictures with different resolutions as at least one layered picture to be selected;
and determining the layered picture with the lowest resolution in the at least one layered picture to be selected as the target layered picture matched with the size information.
In one embodiment, the processor 2001 may also perform the following steps:
if a second layered picture acquisition request sent by the user is received, selecting a layered picture with the maximum resolution from a plurality of layered pictures with different resolutions according to the second layered picture acquisition request, and sequentially sending the base layer picture data and the enhancement layer picture data in the layered picture with the maximum resolution to the user terminal;
if an original picture acquiring request sent by the user is received, sending an original picture to the user terminal according to the original picture acquiring request;
wherein the second hierarchical picture acquisition request is sent by the user terminal upon detecting that the size information does not satisfy a hierarchical picture adaptation condition; the original picture acquisition request is sent by the user terminal when detecting that the layered picture format is not supported; the first hierarchical picture acquisition request is sent by the user terminal upon detecting that the size information satisfies a hierarchical picture adaptation condition.
In one embodiment, the processor 2001 may also perform the following steps:
receiving a decoder acquisition request sent by the user terminal; the decoder acquires the operating system information of the user terminal carried by the request;
and searching a layered picture decoder matched with the operating system information of the user terminal according to the decoder acquisition request, and sending the searched layered picture decoder matched with the operating system information of the user terminal to the user terminal.
In one embodiment, the processor 2001 may also perform the following steps:
acquiring an original picture, and decoding the original picture through a layered picture decoder matched with operating system information of a server to obtain decoded original picture data;
encoding the decoded original picture data into a layered picture of at least one resolution by a layered picture encoder; each layered picture includes base layer picture data and at least one enhancement layer picture data.
In one embodiment, the processor 2001 may also perform the following steps:
setting a hierarchical picture decoder corresponding to at least one type of operating system information respectively;
optimizing prediction modes in the layered picture decoder and the layered picture encoder; the prediction mode comprises a coding unit division mode, a prediction unit division mode, a transformation unit division mode and an intra-frame prediction direction selection mode;
and performing assembly optimization and code structure optimization on the hierarchical picture decoder according to the corresponding operating system information.
In an embodiment, when the processor 2001 performs assembly optimization and code structure optimization on the hierarchical picture decoder according to the corresponding operating system information, the following steps may be specifically performed:
setting a storage space selection optimization rule in the hierarchical picture decoder, the storage space selection optimization rule comprising a rule of setting a register to a temporary storage space of a highest priority, the register being associated with operating system information corresponding to the hierarchical picture decoder;
setting a computational optimization rule in the hierarchical picture decoder, the computational optimization rule comprising a rule that converts a looping logic of a matrix operation into an operation based on a vector instruction set, the vector instruction set being associated with operating system information corresponding to the hierarchical picture decoder.
The embodiment of the invention sends a first layered picture acquisition request carrying the size information of a display area to a server, so that the server selects a target layered picture matched with the size information according to the first layered picture acquisition request, receives base layer picture data in the target layered picture sent by the server, decodes the base layer picture data through a layered picture decoder, displays the decoded base layer picture data, receives enhancement layer picture data in the target layered picture sent by the server, decodes the enhancement layer picture data through the layered picture decoder, synthesizes the decoded enhancement layer picture data with the decoded base layer picture data, and displays the synthesized picture data. Therefore, even under the condition that the network state is unstable, the base layer picture data can be still obtained quickly, so that the display area can display the content of the base layer picture data firstly, the picture display area is prevented from being in a blank state for a long time, the enhancement layer picture data can be further received subsequently, the enhancement layer picture data and the base layer picture data are synthesized, a picture with high quality is obtained, and the picture display effect can be improved. Moreover, the server is used for carrying out assembly optimization and code structure optimization on the hierarchical picture decoders corresponding to the information of the operating systems respectively, so that the user terminals of the operating systems can use the optimized hierarchical picture decoders, and the decoding efficiency of the hierarchical picture decoders is improved.
Furthermore, it is to be noted here that: an embodiment of the present invention further provides a computer storage medium, and the computer storage medium stores the aforementioned computer program executed by the image data processing apparatus 1, and the computer program includes program instructions, and when the processor executes the program instructions, the description of the image data processing method in the embodiment corresponding to fig. 7 can be performed, so that details are not repeated here. In addition, the beneficial effects of the same method are not described in detail. For technical details not disclosed in the embodiment of the computer storage medium related to the present invention, refer to the description of the embodiment of the method of the present invention.
Further, here, it is to be noted that: an embodiment of the present invention further provides another computer storage medium, and the computer storage medium stores the aforementioned computer program executed by the image data processing apparatus 2, and the computer program includes program instructions, and when the processor executes the program instructions, the description of the image data processing method in the embodiment corresponding to fig. 8 can be performed, so that details are not repeated here. In addition, the beneficial effects of the same method are not described in detail. For technical details not disclosed in the embodiments of the computer storage medium to which the present invention relates, reference is made to the description of the method embodiments of the present invention.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above may be implemented by a computer program, which may be stored in a computer readable storage medium and executed by a computer to implement the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
The above disclosure is only for the purpose of illustrating the preferred embodiments of the present invention, and it is therefore to be understood that the invention is not limited by the scope of the appended claims.

Claims (12)

1. An image data processing method characterized by comprising:
when detecting that the size information of a display area meets a hierarchical picture self-adaption condition, sending a first hierarchical picture acquisition request carrying the size information of the display area to a server, so that the server extracts the size information of the display area of a user terminal carried by the first hierarchical picture acquisition request, searching at least one hierarchical picture with a pixel width larger than the picture width in the size information from a plurality of hierarchical pictures with different resolutions as at least one hierarchical picture to be selected, and determining the hierarchical picture with the lowest resolution in the at least one hierarchical picture to be selected as a target hierarchical picture matched with the size information;
receiving base layer picture data in the target layered picture sent by the server, decoding the base layer picture data through a layered picture decoder, and displaying the decoded base layer picture data; the data volume of the base layer picture data is less than that of the high-quality picture data;
receiving enhancement layer picture data in the target layered picture sent by the server, decoding the enhancement layer picture data through the layered picture decoder, synthesizing the decoded enhancement layer picture data and the decoded base layer picture data, and displaying the synthesized picture data; the enhancement layer picture data refers to a delta between the high quality picture data and the base layer picture data;
when the size information is detected not to meet the layered picture self-adaption condition, sending a second layered picture acquisition request to the server, so that the server selects a layered picture with the maximum resolution from a plurality of layered pictures with different resolutions according to the second layered picture acquisition request, and sequentially sending base layer picture data and enhancement layer picture data in the layered picture with the maximum resolution to the user terminal;
and when detecting that the layered picture format is not supported, sending an original picture acquisition request to the server so that the server sends an original picture to the user terminal according to the original picture acquisition request.
2. An image data processing method characterized by comprising:
if a first layered picture acquisition request sent by a user terminal is received, extracting size information of a display area of the user terminal carried by the first layered picture acquisition request, searching at least one layered picture with a pixel width larger than the picture width in the size information from a plurality of layered pictures with different resolutions as at least one layered picture to be selected, and determining the layered picture with the lowest resolution in the at least one layered picture to be selected as a target layered picture matched with the size information;
sending the base layer picture data in the target layered picture to the user terminal so that the user terminal decodes the base layer picture data through a layered picture decoder and displays the decoded base layer picture data; the data volume of the base layer picture data is less than that of the high-quality picture data;
transmitting enhancement layer picture data in the target layered picture to the user terminal so that the user terminal decodes the enhancement layer picture data through the layered picture decoder, synthesizes the decoded enhancement layer picture data with the decoded base layer picture data, and displays the synthesized picture data; the enhancement layer picture data refers to a delta between the high quality picture data and the base layer picture data;
if a second layered picture acquisition request sent by the user terminal is received, selecting a layered picture with the maximum resolution from a plurality of layered pictures with different resolutions according to the second layered picture acquisition request, and sequentially sending base layer picture data and enhancement layer picture data in the layered picture with the maximum resolution to the user terminal;
if an original picture acquiring request sent by the user terminal is received, sending an original picture to the user terminal according to the original picture acquiring request;
wherein the second hierarchical picture acquisition request is sent by the user terminal upon detecting that the size information does not satisfy a hierarchical picture adaptation condition; the original picture acquisition request is sent by the user terminal when detecting that the layered picture format is not supported; the first hierarchical picture acquisition request is sent by the user terminal upon detecting that the size information satisfies a hierarchical picture adaptation condition.
3. The method of claim 2, further comprising:
receiving a decoder acquisition request sent by the user terminal; the decoder acquires the operating system information of the user terminal carried by the request;
and searching a layered picture decoder matched with the operating system information of the user terminal according to the decoder acquisition request, and sending the searched layered picture decoder matched with the operating system information of the user terminal to the user terminal.
4. The method of claim 2, further comprising:
acquiring an original picture, and decoding the original picture through a layered picture decoder matched with the operating system information of the server to obtain decoded original picture data;
encoding the decoded original picture data into a layered picture of at least one resolution by a layered picture encoder; each layered picture includes base layer picture data and at least one enhancement layer picture data.
5. The method of claim 3, further comprising:
setting a hierarchical picture decoder corresponding to at least one type of operating system information respectively;
optimizing prediction modes in the layered picture decoder and the layered picture encoder; the prediction mode comprises a coding unit division mode, a prediction unit division mode, a transformation unit division mode and an intra-frame prediction direction selection mode;
and performing assembly optimization and code structure optimization on the hierarchical picture decoder according to the corresponding operating system information.
6. The method of claim 5, wherein the performing assembly optimization and code structure optimization for the hierarchical picture decoder according to the corresponding operating system information comprises:
setting a storage space selection optimization rule in the hierarchical picture decoder, the storage space selection optimization rule comprising a rule of setting a register to a temporary storage space of a highest priority, the register being associated with operating system information corresponding to the hierarchical picture decoder;
setting a computational optimization rule in the hierarchical picture decoder, the computational optimization rule comprising a rule that converts a looping logic of a matrix operation into an operation based on a vector instruction set, the vector instruction set being associated with operating system information corresponding to the hierarchical picture decoder.
7. An image data processing apparatus characterized by comprising:
the device comprises a sending module and a receiving module, wherein the sending module is used for sending a first hierarchical picture acquisition request carrying the size information of a display area to a server when the size information of the display area is detected to meet a hierarchical picture self-adaption condition, so that the server extracts the size information of the display area of a user terminal carried by the first hierarchical picture acquisition request, at least one hierarchical picture with the pixel width larger than the picture width in the size information is searched in a plurality of hierarchical pictures with different resolutions and is used as at least one hierarchical picture to be selected, and the hierarchical picture with the lowest resolution in the at least one hierarchical picture to be selected is determined as a target hierarchical picture matched with the size information;
the receiving module is used for receiving the basic layer picture data in the target layered picture sent by the server; the data volume of the base layer picture data is less than that of the high-quality picture data;
the decoding display module is used for decoding the base layer picture data through a layered picture decoder and displaying the decoded base layer picture data;
the receiving module is further configured to receive enhancement layer picture data in the target layered picture sent by the server; the enhancement layer picture data refers to a delta between the high quality picture data and the base layer picture data;
the decoding and displaying module is further configured to decode the enhancement layer picture data by the layered picture decoder, synthesize the decoded enhancement layer picture data and the decoded base layer picture data, and display the synthesized picture data;
the sending module is further configured to send a second layered picture acquisition request to the server when it is detected that the size information does not satisfy a layered picture adaptation condition, so that the server selects, according to the second layered picture acquisition request, a layered picture with a maximum resolution from among a plurality of layered pictures with different resolutions, and sequentially sends base layer picture data and enhancement layer picture data in the layered picture with the maximum resolution to the user terminal;
the sending module is further configured to send an original picture obtaining request to the server when it is detected that the layered picture format is not supported, so that the server sends an original picture to the user terminal according to the original picture obtaining request.
8. An image data processing apparatus characterized by comprising:
the device comprises a selection module, a display module and a display module, wherein the selection module is used for extracting size information of a display area of a user terminal carried by a first layered picture acquisition request if the first layered picture acquisition request sent by the user terminal is received, searching at least one layered picture with the pixel width larger than the picture width in the size information from a plurality of layered pictures with different resolutions as at least one layered picture to be selected, and determining the layered picture with the lowest resolution in the at least one layered picture to be selected as a target layered picture matched with the size information;
a sending module, configured to send base layer picture data in the target layered picture to the user terminal, so that the user terminal decodes the base layer picture data through a layered picture decoder, and displays the decoded base layer picture data; the data volume of the base layer picture data is less than that of the high-quality picture data;
the sending module is further configured to send enhancement layer picture data in the target layered picture to the user terminal, so that the user terminal decodes the enhancement layer picture data through the layered picture decoder, synthesizes the decoded enhancement layer picture data with the decoded base layer picture data, and displays the synthesized picture data; the enhancement layer picture data refers to a delta between the high quality picture data and the base layer picture data;
the selection module is further configured to select, if a second layered picture acquisition request sent by the user terminal is received, a layered picture with a maximum resolution from among a plurality of layered pictures with different resolutions according to the second layered picture acquisition request;
the sending module is further configured to send the base layer picture data and the enhancement layer picture data in the layered picture with the maximum resolution to the user terminal in sequence;
the sending module is further configured to send an original picture to the user terminal according to the original picture obtaining request if the original picture obtaining request sent by the user terminal is received;
wherein the second hierarchical picture acquisition request is sent by the user terminal upon detecting that the size information does not satisfy a hierarchical picture adaptation condition; the original picture acquisition request is sent by the user terminal when detecting that the layered picture format is not supported; the first hierarchical picture acquisition request is sent by the user terminal upon detecting that the size information satisfies a hierarchical picture adaptation condition.
9. An image data processing apparatus characterized by comprising: a processor, a memory, and a network interface;
the processor is connected with a memory and a network interface, wherein the network interface is used for connecting a server, the memory is used for storing program codes, and the processor is used for calling the program codes to execute the following operations:
when detecting that the size information of the display area meets a hierarchical picture self-adaption condition, sending a first hierarchical picture acquisition request carrying the size information of the display area to the server, so that the server extracts the size information of the display area of the user terminal carried by the first hierarchical picture acquisition request, searching at least one hierarchical picture with a pixel width larger than the picture width in the size information from a plurality of hierarchical pictures with different resolutions as at least one hierarchical picture to be selected, and determining the hierarchical picture with the lowest resolution in the at least one hierarchical picture to be selected as a target hierarchical picture matched with the size information;
receiving base layer picture data in the target layered picture sent by the server, decoding the base layer picture data through a layered picture decoder, and displaying the decoded base layer picture data; the data volume of the base layer picture data is less than that of the high-quality picture data;
receiving enhancement layer picture data in the target layered picture sent by the server, decoding the enhancement layer picture data through the layered picture decoder, synthesizing the decoded enhancement layer picture data and the decoded base layer picture data, and displaying the synthesized picture data; the enhancement layer picture data refers to a delta between the high quality picture data and the base layer picture data;
when the size information is detected not to meet the layered picture self-adaption condition, sending a second layered picture acquisition request to the server, so that the server selects a layered picture with the maximum resolution from a plurality of layered pictures with different resolutions according to the second layered picture acquisition request, and sequentially sending base layer picture data and enhancement layer picture data in the layered picture with the maximum resolution to the user terminal;
and when detecting that the layered picture format is not supported, sending an original picture acquisition request to the server so that the server sends an original picture to the user terminal according to the original picture acquisition request.
10. An image data processing apparatus characterized by comprising: a processor, a memory, and a network interface;
the processor is connected with a memory and a network interface, wherein the network interface is used for connecting a user terminal, the memory is used for storing program codes, and the processor is used for calling the program codes to execute the following operations:
if a first layered picture acquisition request sent by the user terminal is received, extracting size information of a display area of the user terminal carried by the first layered picture acquisition request, searching at least one layered picture with a pixel width larger than the picture width in the size information from a plurality of layered pictures with different resolutions as at least one layered picture to be selected, and determining the layered picture with the lowest resolution in the at least one layered picture to be selected as a target layered picture matched with the size information;
sending the base layer picture data in the target layered picture to the user terminal so that the user terminal decodes the base layer picture data through a layered picture decoder and displays the decoded base layer picture data; the data volume of the base layer picture data is less than that of the high-quality picture data;
transmitting enhancement layer picture data in the target layered picture to the user terminal so that the user terminal decodes the enhancement layer picture data through the layered picture decoder, synthesizes the decoded enhancement layer picture data with the decoded base layer picture data, and displays the synthesized picture data; the enhancement layer picture data refers to a delta between the high quality picture data and the base layer picture data;
if a second layered picture acquisition request sent by the user terminal is received, selecting a layered picture with the maximum resolution from a plurality of layered pictures with different resolutions according to the second layered picture acquisition request, and sequentially sending base layer picture data and enhancement layer picture data in the layered picture with the maximum resolution to the user terminal;
if an original picture acquiring request sent by the user terminal is received, sending an original picture to the user terminal according to the original picture acquiring request;
wherein the second hierarchical picture acquisition request is sent by the user terminal upon detecting that the size information does not satisfy a hierarchical picture adaptation condition; the original picture acquisition request is sent by the user terminal when detecting that the layered picture format is not supported; the first hierarchical picture acquisition request is sent by the user terminal upon detecting that the size information satisfies a hierarchical picture adaptation condition.
11. A computer storage medium, characterized in that the computer storage medium stores a computer program comprising program instructions which, when executed by the processor, perform the method of claim 1.
12. A computer storage medium, characterized in that the computer storage medium stores a computer program comprising program instructions which, when executed by the processor, perform the method according to any one of claims 2-6.
CN201710824066.6A 2017-09-13 2017-09-13 Image data processing method and device Active CN109493304B (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201710824066.6A CN109493304B (en) 2017-09-13 2017-09-13 Image data processing method and device
PCT/CN2018/105131 WO2019052452A1 (en) 2017-09-13 2018-09-12 Image data processing method, user terminal, server and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710824066.6A CN109493304B (en) 2017-09-13 2017-09-13 Image data processing method and device

Publications (2)

Publication Number Publication Date
CN109493304A CN109493304A (en) 2019-03-19
CN109493304B true CN109493304B (en) 2022-11-08

Family

ID=65687963

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710824066.6A Active CN109493304B (en) 2017-09-13 2017-09-13 Image data processing method and device

Country Status (2)

Country Link
CN (1) CN109493304B (en)
WO (1) WO2019052452A1 (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11646983B1 (en) 2019-07-16 2023-05-09 Whatsapp Llc Methods and systems for transmitting an image with thumbnail data
DE102020118411A1 (en) * 2019-07-16 2021-01-21 Whatsapp Inc. METHODS AND SYSTEMS FOR TRANSMITTING AN IMAGE IN Sentences
CN115914750A (en) * 2022-11-10 2023-04-04 深圳创维－Rgb电子有限公司 Picture display method and device, display equipment and storage medium

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103518374A (en) * 2011-06-10 2014-01-15 联发科技股份有限公司 Method and apparatus of scalable video coding
CN105681818A (en) * 2016-01-07 2016-06-15 中国联合网络通信集团有限公司 Picture cloud storage method and device
CN105847345A (en) * 2016-03-21 2016-08-10 深圳市茁壮网络股份有限公司 Picture merging display method and device

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101098311B (en) * 2006-06-29 2010-10-13 腾讯科技（深圳）有限公司 Picture live transmission method and device in instant communication
US9338204B2 (en) * 2013-01-25 2016-05-10 Mobitv, Inc. Prioritized side channel delivery for download and store media
EP2804375A1 (en) * 2013-02-22 2014-11-19 Thomson Licensing Coding and decoding methods of a picture block, corresponding devices and data stream
KR20140121315A (en) * 2013-04-04 2014-10-15 한국전자통신연구원 Method and apparatus for image encoding and decoding based on multi-layer using reference picture list
WO2015099613A1 (en) * 2013-12-26 2015-07-02 Singapore University Of Technology And Design A method and apparatus for reducing data bandwidth between a cloud server and a thin client

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103518374A (en) * 2011-06-10 2014-01-15 联发科技股份有限公司 Method and apparatus of scalable video coding
CN105681818A (en) * 2016-01-07 2016-06-15 中国联合网络通信集团有限公司 Picture cloud storage method and device
CN105847345A (en) * 2016-03-21 2016-08-10 深圳市茁壮网络股份有限公司 Picture merging display method and device

Also Published As

Publication number Publication date
WO2019052452A1 (en) 2019-03-21
CN109493304A (en) 2019-03-19

Similar Documents

Publication Publication Date Title
US11818389B2 (en) Method and device for encoding/decoding motion vector
CN111066326B (en) Machine learning video processing system and method
CN111681167B (en) Image quality adjusting method and device, storage medium and electronic equipment
US10771813B2 (en) Reference frame encoding method and apparatus, and reference frame decoding method and apparatus
US8644381B2 (en) Apparatus for reference picture resampling generation and method thereof and video decoding system using the same
EP2805499B1 (en) Video decoder, video encoder, video decoding method, and video encoding method
CN109565587B (en) Method and system for video encoding with context decoding and reconstruction bypass
CN109493304B (en) Image data processing method and device
US11310514B2 (en) Encoding method and apparatus using non-encoding region, block-based encoding region, and pixel-based encoding region
JP2014525176A (en) Intensity-based chromaticity intra prediction
JP2022509191A (en) Video decoding control methods, devices, electronic devices and storage media
GB2560548A (en) Video data processing system
EP3873094B1 (en) Reduction of visual artifacts in parallel video coding
CN110679150A (en) Same frame motion estimation and compensation
EP3918784A1 (en) Shape adaptive discrete cosine transform for geometric partitioning with an adaptive number of regions
US20190222858A1 (en) Optimal out of loop inter motion estimation with multiple candidate support
CN111432213A (en) Adaptive tile data size coding for video and image compression
KR102028016B1 (en) Method and apparatus for video coding/decoding using intra prediction
KR102234712B1 (en) Methods For Encoding/Decoding Image And Apparatus For Encoder/Decoder Using The Same
US20210195220A1 (en) Methods and systems for adaptive cropping
CN111108747B (en) Obtaining a target representation of a time sample of a signal
EP3281410B1 (en) Efficient low-complexity video compression
WO2021046692A1 (en) Resolution-adaptive video coding with conditional interpolation filters
CN116405675A (en) Encoding method, apparatus, device and storage medium
KR20210036328A (en) Methods For Encoding/Decoding Image And Apparatus For Encoder/Decoder Using The Same

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant